Posted on Leave a comment

Empathy Vision Model: AI that can see and talk with us about our world

Microsoft unveils a smartphone app in Japan, featuring Rinna the chatbot with a combination of powerful new AI technologies

Artificial intelligence (AI) that can see and comment on the world around us will soon be interacting much more naturally with people in their daily lives thanks to a powerful combination of new technologies being trialed in Japan through a chatty smartphone app.

Microsoft Japan President Takuya Hirano
Microsoft Japan President Takuya Hirano

The app features Microsoft Japan’s hugely popular Rinna social chatbot. It was unveiled at the Microsoft Tech Summit 2018 in Tokyo on Monday and is still in its developmental stage.

The AI behind the app has enhanced sight, hearing, and speech capabilities to recognize and talk about objects it sees in ways that are similar to how a person would. As such, it represents a significant step towards a future of natural interactions between AI and people. At the heart of the app is the “Empathy Vision Model,” which combines conventional AI image recognition technology emotional responses.

With this technology, Rinna views her surrounding through a smartphone’s camera. She not only recognizes objects and people, she can also describe and comment verbally about them in realtime. Using natural language processing, speech recognition, and speech synthesis technologies – developed by scientists at Microsoft Research – she can engage in natural-like conversations with a phone’s human user.

“A user can hold their smartphone in their hand or place it in a breast pocket while walking around. With the camera switched on, Rinna can see the same scenery, people, and objects as the user and it talk about all that with the user,” Microsoft Japan President Takuya Hirano said.

Unlike other AI vision models, Rinna can describe her impressions of what she is viewing with feeling, rather than just listing off recognition results such as the names, shapes, and colors of the things she sees. Rinna on a smartphone can view the world from the same perspective as a user and can converse with that user about it.

Let’s take the following image to help illustrate the difference:

Dog, father, son and a car behind

Conventional AI vision technology might typically react this way: “I see people. I see a child. I see a dog. I see a car.”

In contrast, Rinna with the Empathy Vision Model might say: “Wow, nice family! Enjoying the weekend, maybe? Oh, there’s a car coming! Watch out!”

As well as the Empathy Vision Model, which generates empathetic comments in real time about what the AI sees, Rinna’s smartphone app also features other cutting-edge features, including “full duplex.” This enables AI to participate in telephone-like natural conversations with a person by anticipating what that person might say next.

This capability helps Rinna make decisions about how and when to respond to someone who is chatting with her, a skill set that is very natural to people, but not common in chatbots. It differs from “half duplex,” which is more like the walkie-talkie experience in which only one party to a conversation can talk at any one time. Full duplex reduces the unnatural lag time that can sometimes make interactions between a person and a with chatbots feel awkward or forced.

Rinna’s smartphone app also incorporates Empathy Chat, which aids independent thinking by the AI. This helps keep a conversation with the user going as long as possible. In other words, the AI selects and uses responses most likely to encourage a person to keep engaged and talking.

It is still in its development stage and the timing of its general release has not been set. But the voice chat function is available as “Voice Chat with Rinna” on Rinna’s official LINE account in Japan.

READ the Latest on artificial intelligence in Asia

Tags: ,

Posted on Leave a comment

Photos from last week’s Future Decoded event in London

Tens of thousands of people attended Microsoft’s Future Decoded in London on October 31 and November 1.

Partners, consumers, experts, journalists and analysts heard from keynote speakers including Microsoft Chief Executive Satya Nadella, UK CEO Cindy Rose, Azure Executive Vice-President Jason Zander, UK Chief Technology Officer Michael Wignall, broadcaster and author Matthew Syed.

During the event, Microsoft announced a partnership with British Gas owner Centrica, a deal with NHSScotland to revolutionise healthcare in the country and cut waiting times, a tie-up with the University of Cambridge to help AI research in the UK, and the expansion of the “defending democracy” program.

Microsoft also released research on AI, showing that nearly half of bosses believe their business model won’t exist by 2023.

Here’s everything that happened during the two days (click pictures to enlarge):

Tags: , , , , , ,

Posted on Leave a comment

Weird and wonderful games from ID@Xbox

A thousand games have launched through the ID@Xbox program – hopefully, you’ve picked that up by now. My name’s Glenn and I’ve been doing marketing for the ID@Xbox program almost from day one, so I’ve been around for the launch of all but a couple of these. (And I have the grey hair to prove it. Seriously, I had no grey hair before I started working on ID.)

Other than my impeccable grammar and stunning fashion sense, one thing I pride myself in is playing (nearly) all the ID@Xbox games. So, when we got to the 1,000 game milestone I thought to myself, “Hey, Self, I should write an article celebrating all the amazing, landmark, award-winning games in the ID@Xbox program!” And then I thought: “Meh – that’s exactly what everybody would expect, and ID@Xbox is about embracing all kinds of games – including new, unique and different games – from independent publishers.” And what I enjoy as much as anything else about working on the ID team is finding strange and usual games that you’d never see anywhere else.

So, here we go – some of the most unusual, strangest, and even flat out weird games that you can find on Xbox One, courtesy of the ID@Xbox program.

Anarcute – It’s cute anarchy. Get it? Your goal is to assemble a group of cute animals to take on the evil corporations ruling and ruining the world. As you march through the city wreaking havoc you recruit cute companions to join your cause Mob rule has never been so cute! And man is it cute. They make civil disobedience unbelievably cute. Even the music’s cute!

Goat Simulator In this paradigm of sublime brilliance from the geniuses at Coffee Stain you play (as you might have guessed) as a goat. And you destroy things. And lick things. And when you lick things your goat tongue sticks to things. And can steal Deadmau5’s head. And jump on trampolines (so many trampolines). And you can morph into a tall goat (giraffe), or fancy goat (penguin), etc. And you earn points for doing things. And I guess there are missions – I don’t know, I was too busy licking things.

Headlander – Don’t lose your head over this one! In this 2D puzzle game with some light combat, you fly around as a head on a jetpack thing and suck off the heads of robots, then stick your head on their bodies, thereby gaining control of each robot’s specific abilities. While the “head sucking” premise alone would earn this game a spot on this list, the groovy sci-fi spin on “Austin Powers” kicks it over the edge – we would expect nothing other than something this unique from our friends at Double Fine, creators of Psychonauts, Broken Age, and others.

Level 22 – You’re late for work and you have to sneak to your desk without being seen by the boss or you’re fired! It’s a stealth game set in the world of work which sounds really simple, but it’s actually a pretty complex puzzle game with the addition of vents to crawl through, doughnuts to distract guards with, newspapers and coffee cups and all manner of office paraphernalia. And some of the achievements in this game are seriously meta. This was one of our earlier ID@Xbox releases and I played this thing into the ground. At work. While I should have been working. Don’t tell the boss!

Maize – I used to live in Iowa and I like corn. There I said it. I’m not sure if that’s why I was drawn to the game or if it was the incredible visuals and beautiful art portraying sentient corn exploring a farm and then a laboratory to unravel the mystery of why sentient corn exists in the first place. And along the way you pick up an obnoxious talking teddy bear robot companion who just simply cannot believe that he’s saddled with being a sidekick to a dumb corn stalk. I mean, can you blame him? If you like adventure games, and corn, and teddy bears, and hysterical writing, and corn, then this is the game for you. And if you don’t like all those things then you really need to put some time aside to reevaluate your life choices.

Roundabout – You’re a limo driver and for some reason your limousine is constantly spinning in circles. Sure, you probably could have stopped to get it fixed at any point – sounds like a transaxle (I have no idea what that means). But you just go with it. With your horribly broken car you pick people up, drop them off and undertake various missions. And it’s all intercut with full motion videos (FMV) of these wacky people you’re picking up being wacky. This one is worth playing for the FMV scenes alone, if not to simply try to find out why somebody would make a game like this.

Thomas Was Alone – Unlike a lot of the other games on this list, Thomas Was Alone isn’t a funny game. It’s a serious game about a group of characters, each with distinctive, well developed personalities and a sophisticated character arc that progresses throughout the story. And they’re blocks. Blocks of different colors and sizes, but basically squares are rectangles. But you really care about those squares and rectangles by the end of the story. You really do. It’s weirdly beautiful.

Soda Drinker Pro – This is the crème-de-la-crème of weird games. This is the Mt. Everest of strangeness. This is the Mona Lisa of bizarre-o land. To describe this game in writing is an affront to the majesty of its absurdity, but with apologies I’ll give it a try. You drink soda. And you’re really good at it. And you drink soda on the beach, in a park, in a city and in various other locations appropriate to the activity of drinking soda. If you have mad skillz you can locate some bonus sodas. Sounds amazing, right? But wait, that’s not the half of it – literally. If you manage to stumble through some texture on one wall in level two you are transported to a completely different game. And that game is… also quite unique. There are some turtles and clocks and… who am I kidding I can’t possible describe it. If you have the chops to be a Soda Drinker – and not an amateur soda drinker – then check it out. (And if you’re from the Midwest and you say “Pop” I guess you’re out of luck.)

And so, that’s my list. There are many, many more strange and weird games that you should check out. Honorable mentions include: I Am Bread, Manual Samuel, Stay, The Magic Circle, Three Fourths Home and so many more. Please, go check out some of the weirdness that the ID@Xbox program delivers!

Posted on Leave a comment

Team Xbox looks lack on their favorite ID@Xbox and indie game memories

All of us here on Team Xbox are tremendous fans of the talent involved in the creation of games, especially those that come into the ID@Xbox program. From cartoon platformers to adventure games to strategy simulators, there’s a robust catalog of titles that deliver countless memories from those gaming worlds. It’s almost unfair that we can only share a few of those with you too, but we’ve taken some of our most memorable moments and captured them here in this feature. Enjoy.

Chris Charla – Senior Director of ID@Xbox

Having been part of ID@Xbox since the beginning, it’s extremely hard to pick one moment from the thousands of great moments and games and people involved with this program on both the Xbox and developer side. But if you made me pick just one, it would be E3 2015, when I was onstage, in the dark, watching Chad and Jared Moldenhauer introduce Cuphead. I knew how hard they were working, and I knew that after the teaser the year before, people were expecting a lot. I also know how much that trailer was going to blow everyone away. That moment when the StudioMDHR logo came up on stage, I got goosebumps like I never have before, and I just could not get the silly grin off my face watching that game own the stage at E3. There have been tons and tons and tons of other fantastic moments and games over the last five years, but that moment will always stay with me!

Brothers Screenshot

Brothers Screenshot

Phil Spencer – Head of Xbox

The game that comes to mind for me is Brothers. I have such a vivid memory of the first time I completed the game as there’s a key moment at the end in which the game mechanic is designed to reinforce the emotion I felt as a player. If you haven’t played it, Brothers is a cooperative, puzzle-based game where each thumbstick controls one of the two brothers on screen simultaneously. In the game, the brothers’ mother has died and their dad is sick and in need of medicine. The game is about the brothers’ quest to get medicine and by the end of the game, the older brother dies as well. As the story draws to a close, the younger brother is alone, trying to get back to his dad and to do so, he must swim across a stream. We learn early in the game that the younger brother does not swim. Until this moment in the game, he’s only crossed water on the back of his older brother.

So there I was, trying to finish the game, pausing and working to solve the puzzle of how to get the younger brother across the water without his older brother. And then the answer just came to me and when I tried it and it worked – it was spine tingling. I felt the loneliness and sadness of the young brother. I don’t think I’ve ever encountered a gameplay mechanic that evoked emotion in the way Brothers did for me and so many other players.

Never Alone

Never Alone

Ashley McKissick – Head of Xbox Game Pass

Four years ago, when my son was six-years-old, we spent our winter vacation playing Never Alone and exploring the Iñupiat culture together.  It was the perfect game to play with my little one. The co-op nature meant that we had to work together, and the controls were so intuitive and well-crafted. The fact that, while playing through the adventure, we also got to learn about this special Alaskan Native community, made it even more special. I also just absolutely loved the art style in Never Alone, especially the environments, and I am so glad that ID@Xbox enabled us to discover this game. Playing this little gem will remain a fond holiday memory for my son and I.

Manual Samuel

Manual Samuel

Craig Duncan – Studio Head, Rare Games

I’ve enjoyed many incredible ID@Xbox games over the years but my favorite moment is quite a recent one while browsing the what’s new in Xbox Game Pass section there was a game called Manual Samuel. I hadn’t heard of it but was drawn to the art of a cartoon death holding a puppet and then spent the next few hours being pulled out of my gaming comfort zone doing what appeared to be normal everyday tasks; without spoiling the plot which everyone should experience spoiler free it made me smile throughout and laugh at some of the dialog and the crazy moments that poor Samuel had to navigate with appropriate well deserved achievements awarded throughout including probably the craziest driving achievement I have ever earned 😊 A really funny, enjoyable and unique little adventure.

Overcooked 2

Overcooked 2

Katie Stone Perez – Principal Program Manager, Mixer

The timer ends, you see the results, and my daughters jump up and cheer “We did it!” We have come together as a family and defeated the unbread. That’s my favorite ID@Xbox moment. My family loves playing games together and Overcooked 2 provides a great level of challenge but when we work together we can always be successful. Each level starts with us coming up with a plan and then the chaos ensues. The game has become an almost nightly ritual for us before bed and I love that we can have that moment of success that unites us as a family.

Chariot

Chariot

Jeff Rubenstein – Xbox Comms and Inside Xbox Host

I may not recall my daughter’s first word, or exactly when she took her first steps. Should we have saved a lock of hair from her first haircut? Yet I vividly remember her first Minecraft world, I showed her how to record her first PotG in Overwatch, and was sitting right next to her during her first Fortnite Victory Royale – the culmination of a carefully laid game education curriculum that began with… Chariot. Yes, Frima Games’ platformer has players dragging a king’s coffin to a more luxurious burial site to placate his demanding spirit (who berates you on your journey), but it’s colorful, kid-friendly, and teaches the mechanics of platforming and cooperative play. More importantly, it sparked a love of gaming and a father-daughter bonding activity that continues to this day. And that’s better than a pair of bronzed baby shoes any day.

Stardew Valley

Stardew Valley

Graeme Boyd – Xbox Live’s AceyBongos and Inside Xbox Host

The day I got married is one of the greatest days of my life. I am, of course, talking about Stardew Valley. My real wedding was pretty good too, I guess. But in Stardew Valley, getting married felt like the culmination of months of hard work and careful relationship building as I carved out my new life on the farm. It felt like acceptance.

That’s the funny thing about Stardew Valley – it starts out like a nice mash-up of Harvest Moon, Animal Crossing, and even Minecraft. But soon enough the rhythms of it – tending your crops, selling your goods, making new friends, exploring the town, the seasons rolling in and out – they get inside you. You feel part of something special. And for me, that experience was one of the most relaxing and fulfilling I’ve ever had in a video game. Stardew Valley became my happy place (I still have a map of the town next to my desk at work and I gaze longingly at it during stressful conference calls). Getting married made it even happier.

Oh, yeah. I got married to Leah. Obviously.

Thomas Was Alone

Thomas Was Alone

Will Tuttle – Editor in Chief of Xbox Wire

I’m a sucker for a great narrative and ID@Xbox has no shortage of amazing narrative-driven games, from the haunting family drama of the excellent What Remains of Edith Finch to the sci-fi horror of Soma. The best of these combine a compelling narrative with unique simple-but-complex gameplay. One of my favorite games of this generation, Thomas Was Alone by Bithell Games, is a pitch-perfect example of that intoxicating blend. At first glance, the game looks like a straightforward, minimalistic platformer starring a group of quadrilateral shapes, but it quickly becomes clear that it’s much more.

Thanks to top-notch writing and some truly wonderful narration by British humorist Danny Wallace (who won a BAFTA for his performance), those little blocks become Thomas, Claire, Chris, John, and Laura. By combining those beautifully humanized shapes with some good old-fashioned puzzle platforming, Thomas Was Alone helped to push the medium forward in unexpectedly new ways.

Cuphead Screenshot

Cuphead Screenshot

Larry Hryb – Xbox Live’s Major Nelson and Inside Xbox Host

As a fan of film, music and obviously video games, I’ve long had an appreciation for content created and produced by independent studios. So, when I heard the ID@Xbox team was celebrating the release of their 1,000th indie title, I naturally considered some of my favorites and it was impossible not to think about Cuphead.

Cuphead is an absolute gem, featuring beautifully crafted visuals reminiscent of sound cartoons from the golden age of animation. The soundtrack is easily some of the best music you will hear in a video game, composed using live musicians playing jazz, early big band, and ragtime music, a favorite genre of mine. But Cuphead doesn’t just look and sound great, it plays great too. My colleagues may heckle me a bit as I praise Cuphead’s gameplay since I am admittedly “challenged” by some of the levels, but the game is just so much fun to play.

As great as Cuphead is though, the real magic of the game is with the number of subtle references (some might even call them Easter Eggs) seamlessly interwoven into the game. I heard mention of a Street Fighter reference when you battle the frogs Ribby and Croaks, and when I encountered that particular scene, it indeed appeared to be a clear tribute to Street Fighter.

Sure, there are the more obvious ones, like a building in the background named after the developers or the name of their studio appearing in different places. But when you dig a little deeper, you’ll discover a seemingly unending link to other cartoon characters, video games and real people including actors, musicians, artists, and cartoonists. It’s fascinating as you play the game when you recognize one of these subtle links and even more fascinating to think of all the hidden connections yet to be discovered.

Posted on Leave a comment

How your company can be a positive force for disability employment

by Jessica Rafuse, Microsoft Accessibility Senior Program Manager

NDEAM is a wrap, what’s next?

As National Disability Employment Awareness Month comes to an end, I reflect on the last month and ask, “Now what?!” Making a positive impact on the unemployment and underemployment rates for people with disabilities is a challenge that must extend beyond NDEAM and must be a collaborative effort. So how do we keep the momentum going?

To help answer the question, we gathered experts and influencers from over 75 different organizations from across the Seattle-area to the second annual Microsoft DisAbility Employment Symposium.

As Bri Sambo from T-Mobile put it, we must “lean on the community we have and collaborate with other companies.” The Symposium generated discussion on common challenges, lessons learned, and innovation in disability employment. We celebrated people with disabilities and committed to our ongoing efforts towards inclusion. Then we said goodbye. As the last guest exited the building, the lights dimmed in the conference room, and I realized, “Now, the real work begins.”

What can your company do to continue the journey to positively impact disability employment? Three things: get Ready, get Set, Hire!

GET READY

1. Nurture a culture of inclusion

A common theme across the Symposium was the importance of nurturing a culture of inclusion. Celebrating people with disabilities within your organization will ignite a sense of disability pride that results in a more inclusive and productive workplace. Here are a few things to consider:

  • Recognize Talent: People with disabilities are an asset to the workplace. From innovation to troubleshooting to project management skills, each individual with a disability brings with them a unique set of skills that they have honed as a result of their disabilities. Remember that 70% of disabilities are non-apparent, so you like have someone in your workplace today who is already adding value to the team.
  • Create opportunity for connection: To build a culture you first need to build a community. Our Disability Employee Resource Group is a long standing group of employees, who have helped to define disability inclusion at Microsoft. If you are looking for a first step in this journey, start here. Just a few passionate employees with common experiences can support your business in the creation of truly inclusive culture.
Disability Employment Symposium Opening Panel – in order left to right: Jessica Rafuse (Microsoft), Keith Clark (Seattle Lighthouse), Megan Mauney (Amazon), Becky Curran (Disability: IN) and Bri Sambo (T-Mobile)

DisAbility Employment Symposium Opening Panel – in order left to right: Jessica Rafuse (Microsoft), Keith Clark (Seattle Lighthouse), Megan Mauney (Amazon), Becky Curran (Disability: IN) and Bri Sambo (T-Mobile)

GET SET

2. Invest in accessibility

Another common theme we saw during the event was companies referencing the importance of embedding accessibility into the fabric of their companies as key to recruiting and retaining talent with disabilities. So how do they do it?

  • Be creative, be frugal, and be resourceful. “Investment” does not always mean “budget” allocation. Utilize resources that are available for free on public platforms. For example, watch Introduction to Disability and Inclusion to learn the basics, and educate your colleagues by using Microsoft Accessibility Training Resources particularly our At a Glance series, which provides bite-sized accessibility trainings.
  • Leverage technology to empower people with disabilities. The role of technology is indisputable in empowering all people, including people with disabilities. From accessible career websites to assistive technologies like screen readers (e.g., Narrator and JAWS) for people who are Blind or Low Vision, accessible technology can make your company more attractive to talent with disabilities. As an individual, you also have a role in making your workplace more accessible with a few simple tricks. Try using the Accessibility Checker before sending an email or turn on Translator within PowerPoint during your next meeting. If you want to know more about accessibility features check out our Accessibility Feature Sway and at the Microsoft Accessibility Site.
  • Accessibility in all levels of your company. Senior leaders are immensely influential as accessibility champions. Ignite your leader’s passion for accessibility by sharing personal stories about how your company’s business has positively impacted the lives of people with disabilities. In support of NDEAM, Microsoft CEO, Satya Nadella, celebrated with six accessibility feature demos: Inclusive Presentations, Seeing AI, Xbox Adaptive Controller, Learning Tools, Soundscape, and Eye Control.

HIRE

3. Hire someone with a disability today!

The timing is now. While nurturing your company culture and integrating accessibility into all that you do, actively seek candidates with disabilities to join in your journey. People with disabilities are just the problem solvers you need to provide feedback on your efforts as you strive for improvement.

  • Transparency in recruiting: candidates appreciate transparency in your company’s inclusion efforts and clear timelines on the accessibility journey. For hiring managers, open communication is key to confront bias and to allow for discussion around your company’s policies for workplace accommodations and benefits.
  • Identifying gaps: partner with your internal talent acquisition team, HR, legal, or other stakeholders to understand the process for working with candidates with disabilities. Ensure that your organization has a process for soliciting and responding to requests for accommodations. Offer trainings to all members of a recruiting ecosystem and co-create resource guides that are specific to your business. Don’t wait for the processes and policies to be perfect, “Hiring talent with disabilities is just common sense, just ask us what we need.”- Nyle DiMarco
  • Resources abound: check out the Disability Equality Index (DEI), which serves as an neutral benchmarking tool to evaluate, measure, and improve your company’s disability inclusion efforts. We are sharing even more of our learnings through our Disability Inclusion Sway and the Microsoft Inclusive Hiring Site.
Jenny Lay-Flurrie and Nyle DiMarco onstage at Disability Employment Symposium

Jenny Lay-Flurrie and Nyle DiMarco onstage at DisAbility Employment Symposium

Thank you again to all the companies, organizations, and disability influencers who participated in our DisAbility Employment Symposium and to those who helped develop these learnings. For those who were not able to join us in-person, Disability:IN can help you to find other businesses in your area that are also committed to disability inclusion. Be curious, be bold, and be collaborative. Hire someone with disabilities and they will guide you along the way.

Posted on Leave a comment

Microsoft’s code-mixing project could help computers handle Spanglish

ENMLP

Communication is a large part of who we are as human beings, and today, technology has allowed us to communicate in new ways and to audiences much larger and wider than ever before. That technology has assumed single-language speech, which — quite often — does not reflect the way people naturally speak. India, like many other parts of the world, is multilingual on a societal level with most people speaking two or more languages. I speak Bengali, English, and Hindi, as do a lot of my friends and colleagues. When we talk, we move fluidly between these languages without much thought.

This mixing of words and phrases is referred to as code-mixing or code-switching, and from it, we’ve gained such combinations as Hinglish and Spanglish. More than half of the world’s population speaks two or more languages, so with as many people potentially code-switching, creating technology that can process it is important in not only creating useful translation and speech recognition tools, but also in building engaging user interface. Microsoft is progressing on that front in exciting ways.

In Project Mélange, we at Microsoft Research India have been building technologies for processing code-mixed speech and text. Through large-scale computational studies, we are also exploring some fascinating linguistic and behavioral questions around code-mixing, such as why and when people code-mix, that are helping us build technology people can relate to. At the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), my colleagues and I have the opportunity to share some of our recent research with our paper “Word Embeddings for Code-Mixed Language Processing.

A data shortage in code-mixed language

Word embeddings — multidimensional vector representation where words similar in meaning or used in similar context are closer to each other — are learnt using deep learning from large language corpora and are valuable in solving a variety of natural language processing tasks using neural techniques. For processing code-mixed language — say, Hinglish — one would ideally need an embedding of words from both Hindi and English in the same space. There are standard methods for obtaining multilingual word embeddings; however, these techniques typically try to map translation equivalents from the two languages (e.g., school and vidyalay) close to each other. This helps in cross-lingual transfer of models. For instance, a sentiment analysis system trained for English can be appropriately transferred to work for Hindi using multilingual embeddings. But it’s not ideal for code-mixed language processing. While school and vidyalay are translation equivalents, in Hinglish, school is far more commonly used than vidyalay; also, these words are used in slightly different contexts. Further, there are grammatical constraints on code-mixing that disallow certain types of direct word substitutions, most notably for verbs in Hinglish. For processing code-mixed language, the word embeddings should ideally be learnt from a corpus of code-mixed text.

It is difficult to estimate the amount of code-mixing that happens in the world. One good proxy is the code-mixing patterns on social media. Approximately 3.5 percent of the tweets on Twitter are code-mixed. The above pie charts show the distribution of monolingual and code-mixed, or code-switched (cs), tweets in seven major European languages: Dutch (nl), English (en), French (fr), German (de), Portuguese (pt), Spanish (es), and Turkish (tr).

The chart above shows the distributions of monolingual and code-mixed tweets for 12 major cities in Europe and the Americas that were found to have very large or very small fractions of code-mixed tweets, represented in the larger pies by the missing white wedge. The smaller pies show the top two code-mixed language pairs, the size being proportionate to their usage. The Microsoft Research India team found that code-mixing is more prevalent in cities where English is not the major language used to tweet.

Even though code-mixing is extremely common in multilingual societies, it happens in casual speech and rarely in text, so we’re limited in the amount of text data available for code-mixed language. What little we do have is from informal speech conversations, such as interactions on social media, where people write almost exactly how they speak. To address this challenge, we developed a technique to generate natural-looking code-mixed data from monolingual text data. Our method is based on a linguistic model known as the equivalence constraint theory of code-mixing, which imposes several syntactic constraints on code-mixing. In building the Spanglish corpus, for example, we used Bing Microsoft Translator to first translate an English sentence into Spanish. Then we aligned the words, identifying which English word corresponded to the Spanish word, and in a process called parsing identified in the sentences the phrases and how they’re related. Then using the equivalence constraint theory, we systematically generated all possible valid Spanglish versions of the input English sentence. A small number of the generated sentences were randomly sampled based on certain criteria that indicated how close they were to natural Spanglish data, and these sentences comprise our artificial Spanglish corpus. Since there is no dearth of monolingual English and Spanish sentences, using this fully automated technique, we can generate as large a Spanglish corpus as we want.

Solving NLP tasks with an artificially generated corpus

Through experiments on parts-of-speech tagging and sentiment classification, we showed that word embeddings learnt from the artificially generated Spanglish corpus were more effective in solving these NLP tasks for code-mixed language than the standard cross-lingual embedding techniques.

The linguistic theory–based generation of code-mixed text has applications beyond word embeddings. For instance, in one of our previous studies published earlier this year, we showed that this technique helps us in learning better language models that can help us build better speech recognition systems for code-mixed speech. We are exploring its application in machine translation to improve the accuracy of mixed-language requests. And imagine a multilingual chatbot that can code-mix depending on who you are, the context of the conversation, and what topic is being discussed, and switch in a natural and appropriate way. That would be true engagement.

Posted on Leave a comment

ID@Xbox Easter eggs you might have missed

We’ve been celebrating ID@Xbox’s incredible milestone of releasing 1,000 games through the program and as a special treat, we wanted to share a handful of sneaky in-game Easter eggs you might have missed hidden within your favorite games through the ID@Xbox program. Take a peek below and let us know if you have a favorite Easter egg of your own!

Cuphead – Try to spot the original 12 Street Fighter II characters when battling Ribby and the Croaks!

“The Ribby and Croaks fight is one giant homage to our love of fighting games. Throughout the battle, there’s a visual reference to each of the original 12 Street Fighter II characters. From the top, Ribby and Croaks’ gi colors reference Ryu and Ken. Croaks’ flaming fire flies represent Dhalsim’s yoga fire while Ribby’s spinning hand attack is from E. Honda and the flying boxing gloves are from Balrog.  In phase two, Ribby claps sonic booms like Guile while Croaks’ spinning is a nod to Zangief’s lariat. When the frogs combine, Ribby rolls into Croaks like Blanka to start phase three. As the slot machine, the snake, tiger, and bull icons denote Vega, Sagat, and M.Bison, while the spiked drums resemble Chun Li’s bracelets.”

Maja Moldenhauer, StudioMDHR

SpeedRunners & No Time to Explain – Can you guess what crossover ties both of these games together?

“There is a fun crossover Easter egg between No Time to Explain and SpeedRunners. The Shark Boss from No Time to Explain has a ship called “SS Royale” in the background. This is the same ship name as in the SpeedRunners level “SS Royale” that takes place on a ship!”

– Alex Nichiporchik, tinyBuild 

Human: Fall Flat – Have you discovered the secret hidden in the Aztec ruins?

“There are countless ways to traverse the weird and wonderful dreamscapes of Human: Fall Flat. Each level’s physics-based puzzles can be solved any way you see fit, and with up to eight players flailing across a level together, you’re bound to come across some of the game’s hidden gems!

One of Human: Fall Flat’s most challenging secrets can be found in the Aztec level, a labyrinth of swinging platforms and perilous walkways reserved only for the most seasoned of Bobs. Once you’ve made your way across the level’s first platform and into the Aztec ruins, ignore the huge opening teasing you forward and turn around. Climb up the cage behind you and use the box above it to access a narrow grass path. Carefully climb to its peak and your reward will be a literal nest of colorful Easter eggs! Pretty clever, huh? Good luck, fellow humans!”

– Jack Gosling, Curve Digital

In case you missed it – we shared some of the Xbox team’s favorite ID@Xbox memories earlier this week, as well as favorite moments shared by our friends across the industry. What’s your favorite ID@Xbox moment? Give us a Tweet and let us know!

Posted on Leave a comment

Microsoft co-signs Human Rights Campaign’s Business Statement on Transgender Equality

Posted on Leave a comment

Voyage Aquatic takes students on new Minecraft coding adventure

More than 50 percent of jobs require technology skills, and in less than a decade that number will grow to 77 percent of jobs.1 Just 40% of schools have classes that teach programming and if you are a girl, black or Hispanic, or live in a rural community you are even less likely to have access.2

Minecraft and Microsoft are committed to helping close the STEM gap and expanding opportunities for students to learn computer science. For the fourth year, we are partnering with Code.org to support Hour of Code, a global movement demystifying computer science and making coding more accessible through one-hour tutorials and events. Hour of Code helps students get ‘Future Ready’ by connecting them with STEM learning experiences and career opportunities. 

Today, we launched a new Minecraft Hour of Code tutorial, the Voyage Aquatic, which takes learners on an aquatic adventure to find treasure and solve puzzles with coding. Voyage Aquatic encourages students to think creatively, try different coding solutions and apply what they learn in mysterious Minecraft worlds. 

Since 2015, learners around the world have completed nearly 100 million Minecraft Hour of Code sessions. The tutorials offer more than 50 puzzles, as well as professional development, facilitator guides and online training to help educators get started teaching computer science.

Dive into the Voyage Aquatic 

Minecraft teamed up with four YouTube creators – AmyLee33, Netty PlaysiBallisticSquid and Tomohawk, with a cumulative following of more than five million – for this year’s Minecraft Hour of Code. These creative YouTubers guide participants along their journey through caves, ruins and underwater reefs to solve puzzles and learn coding concepts. 

Voyage Aquatic presents 12 unique challenges, focusing on how to use loops and conditionals, two fundamental concepts in computer science. The tutorial also includes a ‘free play’ level for participants to apply what they learn in the prior puzzles and use coding to build imaginative underwater creations.  

People of all ages and experience levels can use the Minecraft world to learn the basics of coding. The tutorial is free, open to anyone and available for any device. If your language is not available, you can help contribute to translation here.

Host an Hour of Code

Anyone can host an Hour of Code. Download a free facilitator’s guide to lead your own Minecraft Hour of Code at your school, library, museum, learning center or even at home. 

Learn how to effectively facilitate an Hour of Code with this new Microsoft Education course for educators. Learn about the benefits of Hour of Code for your students, and where to find resources to lead an Hour of Code. 

Let us know about your experience by posting on Facebook or Twitter and make sure to mention #HourofCode. Tag us @playcraftlearn!

Keep coding in Minecraft

You can continue your coding journey in Minecraft: Education Edition (or Minecraft on Windows 10) using Code Builder, a special feature that allows you to code in Minecraft. 

  • Visit education.minecraft.net/cs for trainings, lessons and classroom activities to go beyond Hour of Code with your students. 
  • If you already have a license for Minecraft: Education Edition, click this link to launch a special Voyage Aquatic world in Minecraft. Use code to fill an aquarium with marine life. 
  • If you are not licensed, you can download a free trial of Minecraft: Education Edition for Windows 10, macOS and iPad by visiting aka.ms/download.

Save the date: Computer Science Education Week, December 3-9 

  • Attend an event: Join an Hour of Code or STEM workshop at a Microsoft Store near you. Sign up at microsoft.com/en-us/store/locations/events-for-educators. 
  • Connect coding to careers with Skype in the Classroom: Sign up for free 30-minute Skype in the Classroom broadcasts and live Q&A with professionals who use code to create amazing things, including two Minecraft game designers! Happening December 3-7, introduce your students to nine inspiring ‘Code Creators’ in the worlds of dance, fashion, gaming, animation and artificial intelligence in a series brought to you by Microsoft and Code.org. Get your questions ready as we explore code-powered creativity. Register for free at aka.ms/codecreators.
  • Learn how to bring CS to your school: Microsoft is helping close the skills gap for all youth by increasing access to equitable computer science education. Discover resources at www.microsoft.com/digital-skills. 

The Future of Jobs Employment, Skills and Workforce Strategy for the Fourth Industrial Revolution, World Economic Forum, January 2016.

2 Pioneering Results in the Blueprint of U.S. K-12 Computer Science Education, Google Gallup poll, 2016.

The post Embark on the Voyage Aquatic, a new Minecraft Hour of Code appeared first on Minecraft: Education Edition.

This post was originally published on this site.

Posted on Leave a comment

Devs and manufacturers: Enhance accessibility with Microsoft’s Eye Control

October 31, 2018 | By Microsoft blog editor

Eye Control is an exciting technology in Windows 10 that allows customers to use their eyes to control an on-screen mouse, keyboard and text-to-speech experience. This is a technology that empowers people with limited mobility such as people living with ALS.

As part of the National Disability Employer Awareness Month (NDEAM), Microsoft’s CEO Satya Nadella has shared a demo video showing the capabilities enabled by the APIs (Application Programming Interface) associated with the Eye Control. Thanks to the additional Eye Drive Library, which emulates a joystick via eye tracking, the video shows prototypes of a remote-controlled car and an electric wheelchair, which can then both be controlled by eye tracking. Those are research prototypes designed to illustrate how this technology can be used to empower people, and appeal to developers, innovators and makers to download these open source libraries and see what you can do.

By highlighting the Eye Control features in Windows and its APIs, we encourage developers and eye tracking device manufacturers to harness the power of eye control for their own applications and products to enhance accessibility and utilize a new mode of interacting with a Windows-based computer. We are keen to see where the development of this technology is leading us in the future.

To learn more about Eye Control APIs for Windows and Eye Drive Libraries referenced in the video visit: Eye Control APIs and Eye Drive Library. To learn more about the research team behind this technology visit Microsoft Enable.

If you have any questions about the demo or the technology, please reach out to the Disability Answer Desk.

[embedded content]