Gaze Controlled Application for children with Speech and Motor Impairment

Project Overview

The project is an ongoing research on assistive systems for children with Severe Speech and Motor Impairment (SSMI). The project is funded by Microsoft Research under Microsoft AI Accessibility Grant. Initial user studies were done previously. The task assigned to me was understanding the needs identified from the user study and design and develop a gaze controlled application as part of the edutainment systems.

About I³D Lab

I³D Lab is a research lab at Centre for Product Design & Manufacturing (CPDM) in Indian Institute of Science, Bengaluru (IISc Bengaluru). The Intelligent Inclusive Interaction Design (I³D) Lab undertakes research in the field of

• Human Computer Interaction,
• Intelligent User Interfaces and
• Inclusive Design.

---

Project Goal

To design, develop and test gaze controlled quiz application for children with Severe Speech and Motor Impairment (SSMI) using Microsoft UWP Framework

Process

Project Timeline

Research

The research phase included understanding the needs of the end users and identifying the challenges they face. The user’s challenges was translated to the design challenges which were as follows:

• Mixed Question Types: Designing and Developing a consistent UI for match of variety of question and answer types for quiz in a limited workspace.

• Limited Screen Space: Previous studies identified that children with SSMI felt difficulty in accessing certain screen spaces. These spaces were treated as dead spaces and so the application was to be designed withing the available canvas space.

• Button and Text sizes: The previous studies gave reference to minimum permissible button and text size to be considered throughout the design.

• Image Orientation: The UI for Q_G and Q_GT had to be designed for both image orientations (Landscape & Portrait)

Design

From the identified constraints various layouts were designed. Since the usage of complex shapes, iconography or any distracting elements was discouraged, considering the fact that they might induce certain levels of cognitive stress, the UI was kept very simplified.

• 

• 

• 

• 

Development

The application was developed using C# and XAML for Microsoft UWP. The framework was used considering the ease of integrating gaze technology it provides.

Final application screen.

Implementation & Testing

The application was developed primarily for experimenting the reception and user experience for children with SSMI. It was tested at Spastic Society of India, Chennai during the project timeline and is still in continuous evaluation and iteration phase.

Application testing at Spastic Society of India, Chennai

Team

Personal Challenges & Learning

• Limited Coding Experience

  Learnt C# for Universal Windows Platform and solely designed and developed the frontend and the backend (Database/ Eye tracking integration)

• Function over Form

  Learning to keep function over form when required

• Working in Constraints

  Learning to work in technical and temporal constraints.

---

Please refer internship certificate hereDownload

---