Now Reading
What Assistive Technologies Can Do To Make STEM More Inclusive

What Assistive Technologies Can Do To Make STEM More Inclusive

Illustration: Cdd20/pixabay.

Bengaluru: Entrepreneur Christina Wallace wrote in 2015, “for most people – myself included – the default [picture of a scientist] is probably a white man in a lab coat, hunched over a microscope.”

Thanks to many individuals, collectives and movements showing the mirror on racial, gender-based and caste-based discrimination in STEM, this stereotype is slowly changing. However, few still have promoted the inclusion of people with a disability in STEM.

Beyond the ideal of egalitarianism, more people means more fresh perspectives and newer ways of approaching old problems. Wanda Diaz Merced, an astronomer who lost her sight in her twenties, put it thus: “Right now, we are missing discoveries because we are only focused on some visual ways of interacting with the data.”

As an intern at NASA in 2005, along with her mentor Robert Candey, Merced developed software that could map astronomical data into sound – its pitch, rhythm and volume. She went on to do her PhD at the University of Glasgow, where she used this tool to make astronomical observations.

“It could help astronomers avoid methodological biases that come with interpreting data only visually,” Merced said in an interview in December 2019.

In India, far from becoming scientists and technologists in their own right, people with disabilities have had to fight for access to higher education in STEM – with little success.

In 2013, the IIT council and the Central Board for Secondary Education (CBSE) denied Kartik Sawhney, the first blind person to study in the science and mathematics stream in classes XI and XII in India – the permission to write the IIT Joint Entrance Examination (JEE) due to his lack of sight.

Sawhney went on to ace the entrance tests of many foreign universities and completed his BS and MS in computer science from Stanford University. In 2019, he co-founded I-STEM, an organisation that aims to provide “equal access to information, resources and opportunities” for people with disabilities.

The story of Vidhya Y., the first visually impaired student from Karnataka to pursue mathematics in classes XI and XII, is similar. She hails from a village in the outskirts of Bengaluru. She realised when she was in class VI that science and mathematics were her favourite subjects. But she was forced to take commerce after class X due to the “practical nature of science”, she said in an interview.

Unswayed, she pursued a combination of subjects in commerce, including mathematics, studying by listening to audio recordings of lectures.

When she applied for jobs with multiple companies after her postgraduate education, she was rejected. So Vidhya co-founded Vision Empower to help visually impaired children access education. At the organisation, she leads work on creating and distributing accessible content (digital material, tactile diagrams and models), educating visually impaired students and training teachers.

“STEM careers are flourishing!” Vidhya told me when I reached out to her. “How can we exclude the blind and deprive them of these opportunities?”

Vembi Technologies, the for-profit arm of Vision Empower, has developed a Braille book-reader called ‘HEXIS’ that allows people to download and access content from online platforms.

Trestle Labs, another start-up in Bengaluru, has also developed a reader that reads out printed texts in real-time.

While these tools are empowering, images and graphs are still tricky. As Vidhya asked, “You can’t really study mathematics and science without these diagrams, right?”

This was also one of the CBSE’s arguments against Sawhney’s attempt to pursue his higher studies in India. According to news reports, the board wasn’t convinced that he could “handle the ‘visual inputs’ – graphs, diagrams, and models – required for science”.

The audio charts project that Sukriti Chadha led could fill this gap. A graduate in electrical engineering and finance from Princeton University, Chadha started working on accessibility about five years ago when her dad lost sight in one of his eyes. She was a developer at Yahoo Finance at the time.

“It was an important issue because many of the platform’s users were older, loyal customers who are likely to have vision problems with age,” Chadha said.

She managed to find a reasonable solution using tones and haptics [footnote]Touch-related feedback, including vibrations[/footnote] to summarise the information in charts. When they tested it on users, about 80% of them were able to recreate the overall pattern themselves.

She also had another idea – to map the information to haptic feedback and draw the pattern on a skin surface. But she couldn’t develop it because today’s smartphones have only one haptic centre, or point of contact.

Although the scope of the application she developed is currently limited to finance, Chadha said it can be expanded to different areas, including education.

“If we are able to use alternate modalities like sound and haptics in traditional interfaces, as well as emerging technologies such as augmented reality and virtual reality effectively, we open a whole new world of possibilities for … users to more fully participate in the digital economy,” Chadha said.

“This includes being able to study science, statistics and interpret data with similar efficiency to what their sighted counterparts have access to.”

But one issue that continues to bother Indian users is the high cost of assistive technologies available in the market.

“Although there are technologies available, many of them are not affordable. Most of them are not built with Indian users in mind,” Vidhya said. (Chadha’s audio charts project is a notable exception: it’s open source.)

The other problem, Vidhya explained, is to establish policies that integrate assistive technologies and make them work in favour of everyone. And as much as we need policies to regulate the use of assistive technologies in specific contexts, we also need regulations that guide their development.

Chadha is involved in designing guidelines for accessible mobile and web applications as part of a working group within the World Wide Web Consortium and Mozilla’s MDN product advisory board. Vidhya is involved in advocacy for policies by the government to use accessible technologies in education.

“Building a specific solution for one product and open-sourcing it solves a targeted problem in innovative ways. It is relatively quick, and the impact is almost instantly observable after it is shipped,” Chadha said. “But thinking about policy is much broader and multi-faceted, in that it takes years to come up with a widely applicable guideline, take into account perspectives from academia, industry and government bodies to solve systemic issues.”

And while this sort of change is slow, she said it’s also powerful.

Although the Indian government launched an accessibility campaign in 2014, the focus was on making government buildings, airports, railway stations etc., physically accessible to people with disabilities. But of late, it has also started promoting assistive technologies through grants, competitions, schemes and partnerships with corporations.

One technology that was nurtured through such a competition was See Sound Live, which enables smartphones to provide visual feedback on speech efforts for deaf users.

Having performed around 300 cochlear implant surgeries in India, Shomeshwar Singh found that older children had a more difficult time speaking as they focused on using their visual sense, and later lost their ability to process auditory inputs permanently.

“I realised that I had a potential solution in my head – to reroute auditory inputs visually by creating a visual avatar of spoken sounds,” Singh said. “This idea shaped out to be the current-day product See Sound Live.”

With this tool, children can add some spoken sounds to their abilities, and Singh said this could be very rewarding when it comes to achieving social inclusion and economic success.

Singh believes that this technology can become for the deaf tomorrow what Braille is for the blind today. His team “anticipates that in the next decade, … a combination of currently evolving See Sound Live technology, speech-to-text technology and artificial intelligence could help people who are deaf develop receptive and expressive language.”

Such solutions coming together could ultimately break the barriers that stand between all children and a good STEM education. As Chadha put it, “It’s a great way to understand how people experience everyday products in different ways.”

Joel P. Joseph is a science writer.

Scroll To Top