![\"Mary](\"https:\/\/www.sickgaming.net\/blog\/wp-content\/uploads\/2020\/10\/shrinking-the-data-desert-making-ai-systems-more-inclusive-of-people-with-disabilities.jpg\")
<\/a>Today, Team Gleason<\/a> announced it is partnering with Microsoft on Project Insight<\/a>, which will create an open dataset of facial imagery of people living with ALS to help advance innovation in computer vision and train those AI models more inclusively.<\/p>\n It\u2019s an industry-wide problem that won\u2019t be solved by one project or organization alone, Microsoft says. But new collaborations are beginning to address the issue.<\/p>\n A research roadmap on AI Fairness and Disability<\/a> published by Microsoft Research and a workshop on Disability, Bias and AI<\/a> hosted last year with the AI Now Institute at New York University found a host of potential areas in which mainstream AI algorithms that aren\u2019t trained on inclusive data either don\u2019t work well for people with disabilities or can actively harm them.<\/p>\n If a self-driving car\u2019s pedestrian detection algorithms haven\u2019t been shown examples of people who use wheelchairs or whose posture or gait is different due to advanced age, for example, they may not correctly identify those people as objects to avoid or estimate how much longer they need to safely cross a street, researchers noted.<\/p>\n AI models used in hiring processes that try to read personalities or interpret sentiment from potential job candidates can misread cues and screen out qualified candidates with autism or who emote differently. Algorithms that read handwriting may not be able to cope with examples from people who have Parkinson\u2019s disease or tremors. Gesture recognition systems may be confused by people with amputated limbs or different body shapes.<\/p>\n It\u2019s fairly common for some people with disabilities to be early adopters of intelligent technologies, yet they\u2019ve often not been adequately represented in the data that informs how those systems work, researchers say.<\/p>\n \u201cWhen technologies are so desired by a community, they\u2019re often willing to tolerate a higher rate of errors,\u201d said Meredith Ringel Morris, senior principal researcher who manages the Microsoft Research Ability Team<\/a>. \u201cSo imperfect AI systems still have value, but they could provide so much more and work so much better if they were trained on more inclusive data.\u201d<\/p>\n Danna Gurari, an AI for Accessibility grantee and assistant professor at the University of Texas at Austin, had that goal in mind when she began developing the VizWiz datasets<\/a>. They include tens of thousands of photographs and questions submitted by people who are blind or have low vision to an app originally developed by researchers at Carnegie Mellon University<\/a>.<\/p>\n The questions run the gamut: What is the expiration date on this milk? What does this shirt say? Do my fingertips look blue? Do these clouds look stormy? Do the charcoal briquettes in this grill look ready? What does the picture on this birthday card look like?<\/p>\n The app originally crowdsourced answers from people across the internet, but Gurari wondered if she could use the data to improve how computer vision algorithms interpret photos taken by people who are blind.<\/p>\n Many of those questions require reading text, such as determining how much of an over-the-counter medicine is safe to take. Computer vision research has often treated that as a separate problem, for example, from recognizing objects or trying to interpret low-quality photos. But successfully describing real-world photos requires an integrated approach, Gurari said.<\/p>\n Moreover, computer vision algorithms typically learn from large image datasets of pictures downloaded from the internet. Most are taken by sighted people and reflect the photographer\u2019s interest, with items that are centered and in focus.<\/p>\n But an algorithm that\u2019s only been trained on perfect images is likely to perform poorly in describing what\u2019s in a photo taken by a person who is blind; it may be blurry, off center or backlit. And sometimes the thing that person wants to know hinges on a detail that a person who is sighted might not think to label, such as whether a shirt is clean or dirty.<\/p>\n \u201cOften it\u2019s not obvious what is meaningful to people, and that\u2019s why it\u2019s so important not just to design for \u2014 but design these technologies with \u2014 people who are in the blind and low vision community,\u201d said Gurari, who also directs the School of Information\u2019s Image and Video Computing Group at the University of Texas at Austin.<\/p>\n Her team undertook the massive task of cleaning up the original VizWiz dataset to make it usable for training machine learning algorithms \u2014 removing inappropriate images, sourcing new labels, scrubbing personal information and even translating audio questions into text to remove the possibility that someone\u2019s voice could be recognized.<\/p>\n Working with Microsoft funding and researchers, Gurari\u2019s team has developed a new public dataset to train, validate and test image captioning algorithms<\/a>. It includes more than 39,000 images taken by blind and low vision participants and five possible captions for each. Her team is also working on algorithms that can recognize right off the bat when an image someone has submitted is too blurry, obscured or poorly lit<\/a> and suggest how to try again.<\/p>\n\u2018Pushing the state of the art\u2019<\/strong><\/h2>\n
![\"Danna](\"https:\/\/www.sickgaming.net\/blog\/wp-content\/uploads\/2020\/10\/shrinking-the-data-desert-making-ai-systems-more-inclusive-of-people-with-disabilities-1.jpg\")
<\/a>