Gaze Controlled Application for children with Speech and Motor Impairment

Project Overview

The project is an ongoing research on assistive systems for children with Severe Speech and Motor Impairment (SSMI). The project is funded by Microsoft Research under Microsoft AI Accessibility Grant. Initial user studies were done previously. The task assigned to me was understanding the needs identified from the user study and design and develop a gaze controlled application as part of the edutainment systems.

About I3D Lab

I3D Lab is a research lab at Centre for Product Design & Manufacturing (CPDM) in Indian Institute of Science, Bengaluru (IISc Bengaluru). The Intelligent Inclusive Interaction Design (I3D) Lab undertakes research in the field of

  • Human Computer Interaction,
  • Intelligent User Interfaces and
  • Inclusive Design.

Project Goal

To design, develop and test gaze controlled quiz application for children with Severe Speech and Motor Impairment (SSMI) using Microsoft UWP Framework


Project Timeline


The research phase included understanding the needs of the end users and identifying the challenges they face. The user’s challenges was translated to the design challenges which were as follows:

  • Mixed Question Types: Designing and Developing a consistent UI for match of variety of question and answer types for quiz in a limited workspace.
  • Limited Screen Space: Previous studies identified that children with SSMI felt difficulty in accessing certain screen spaces. These spaces were treated as dead spaces and so the application was to be designed withing the available canvas space.
  • Button and Text sizes: The previous studies gave reference to minimum permissible button and text size to be considered throughout the design.
  • Image Orientation: The UI for QG and QGT had to be designed for both image orientations (Landscape & Portrait)


From the identified constraints various layouts were designed. Since the usage of complex shapes, iconography or any distracting elements was discouraged, considering the fact that they might induce certain levels of cognitive stress, the UI was kept very simplified.


The application was developed using C# and XAML for Microsoft UWP. The framework was used considering the ease of integrating gaze technology it provides.

Final application screen.

Implementation & Testing

The application was developed primarily for experimenting the reception and user experience for children with SSMI. It was tested at Spastic Society of India, Chennai during the project timeline and is still in continuous evaluation and iteration phase.

Application testing at Spastic Society of India, Chennai


Personal Challenges & Learning

  • Limited Coding Experience

    Learnt C# for Universal Windows Platform and solely designed and developed the frontend and the backend (Database/ Eye tracking integration)

  • Function over Form

    Learning to keep function over form when required

  • Working in Constraints

    Learning to work in technical and temporal constraints.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Website Built with

Up ↑

%d bloggers like this: