+- Sick Gaming (https://www.sickgaming.net)
+-- Forum: Computers (https://www.sickgaming.net/forum-86.html)
+--- Forum: Windows (https://www.sickgaming.net/forum-89.html)
+--- Thread: Microsoft - How cloud AI can power a prosthetic arm that sees objects (/thread-86421.html)
Microsoft - How cloud AI can power a prosthetic arm that sees objects - xSicKxBot - 09-12-2018
How cloud AI can power a prosthetic arm that sees objects
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/how-cloud-ai-can-power-a-prosthetic-arm-that-sees-objects.png" width="500" height="262" title="" alt="" /></div><div><p><em>Based on “Connected Arms”, a keynote talk at the <a href="https://conferences.oreilly.com/artificial-intelligence">O’Reilly AI Conference</a> delivered by <a href="https://twitter.com/josephsirosh">Joseph Sirosh</a>, CTO for AI at Microsoft. Content reposted from this <a href="https://www.oreilly.com/ideas/connected-arms">O’Reilly Media</a> website. </em></p>
<div class="oembed-wrapper">[embedded content]</div>
<p>There are over 1 million new amputees every year, i.e. one every 30 seconds – a truly shocking statistic.</p>
<p>The World Health Organization estimates that between 30 to 100 million people around the world are living with limb loss today. Unfortunately, only 5-15% of this population has access to prosthetic devices.</p>
<p>Although prostheses have been around since ancient times, their successful use has been severely limited for millennia by several factors, with cost being the major one. Although it is possible to get sophisticated bionic arms today, the cost of such devices runs into tens of thousands of dollars. These devices are just not widely available today. What’s more, having these devices interface satisfactorily with the human body has been a massive issue, partly due to the challenges of working with the human nervous system. Such devices generally need to be tailored to work with each individual’s nervous system, a process that often requires expensive surgery.</p>
<p>Is it possible for a new generation of human beings to finally help us break through these long-standing barriers?</p>
<p>Can prosthetic devices learn to adapt to us, as opposed to the other way around?</p>
<h3>A Personalized Prosthetic Arm for $100?</h3>
<p>In his talk, Joseph informs us about how, using the combination of:</p>
<ul>
<li>Low-cost off-the-shelf electronics,</li>
<li>3D-printing, and</li>
<li>Cloud AI, for intelligent, learned, personalized behavior,</li>
</ul>
<p>it is now becoming possible to deliver prosthetic arms at a price point of around $100.</p>
<p><img width="500" height="262" class="aligncenter size-mediumlarge wp-image-18335" alt="" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/how-cloud-ai-can-power-a-prosthetic-arm-that-sees-objects.png" /></p>
<p>Joseph takes the <a href="https://azure.microsoft.com/en-us/blog/connected-arms-a-closer-look-at-the-winning-idea-from-imagine-cup-2018/">smartARM</a> as an example of such a breakthrough device. A prototype built by two undergraduate students from Canada who recently won the first prize in Microsoft’s Imagine Cup, the smartARM is 3D-printed, has a camera in the palm of its hand and is connected to the cloud. The magic is in the cloud, where a computer vision service recognizes the objects seen by the camera. Deep learning algorithms then generate the precise finger movements needed to grasp the object near the arm. Essentially, the cloud vision service classifies the object and generates the right grip or action, such as a pincer action to pick up a bunch of keys on a ring, or a palmar action to pick up a wineglass. The grip itself is a learned behavior which can be trained and customized.</p>
<p>The user of the prosthetic arm triggers the grip (or its release) by flexing any muscle of their choice on their body, for instance, their upper arm muscle. A myoelectric sensor located in a band that is strapped over that muscle detects the signal and triggers the grip or its release.</p>
<h3>Simple, Adaptable Architecture</h3>
<p>The architecture of this grip classification solution is shown below. The input to the raspberry pi on the smartARM comes from camera and the muscle sensor. These inputs are sent to the <a href="https://azure.microsoft.com/en-us/services/cognitive-services/custom-vision-service/">Azure Custom Vision Service</a>, an API in the cloud which has been trained on grip classifications and is able to output the appropriate grip. This grip is sent back to an Arduino board in the smartARM which can then trigger the servo motors that realize that grip in the physical world, i.e. as soon as the smartARM gets the signal to do so from the muscle sensor.</p>
<p><img width="500" height="270" class="aligncenter size-mediumlarge wp-image-18345" alt="" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/how-cloud-ai-can-power-a-prosthetic-arm-that-sees-objects-1.png" /></p>
<p>This is an adaptable architecture. It can be customized to the kinds of movements you want this arm to generate. For instance, the specific individual using this prosthetic can customize the grips for the objects in their daily life which are the ones they care the most about. The muscle sensor -based trigger could be replaced with a speech trigger, if so desired.</p>
<h3>Summary</h3>
<p>AI is empowering a new generation of developers to explore all sorts of novel ideas and mashups. Through his talk on “<a href="https://www.oreilly.com/ideas/connected-arms">Connected Arms</a>”, Joseph shows us how the future of prosthetic devices can be transformed by the power of the cloud and AI. Imagine a world in which all future assistive devices are empowered with AI in this fashion. Devices would adapt to individuals, rather than the other way around. Assistive devices will become more affordable, intelligent, cloud-powered and personalized.</p>
<p>Cloud AI is letting us build unexpected things that we would scarcely have imagined.</p>
<p>Such as like an arm that can see.</p>
<p>The AI / ML Blog Team</p>
<p> </p>
</div>