Using a Neural Network Architecture and dataset of course! The Microsoft Maps AI team’s network was based on UNet and ResNet and the following papers [U-Net] (https://arxiv.org/abs/1505.04597), [Res U-Net] (https://arxiv.org/pdf/1512.03385.pdf), [Res U-Net] (https://arxiv.org/pdf/1711.10684.pdf). The model was trained using the Keras toolkit with Bing Maps 512×512 images, it is fully convolutional, which allows images of any size (that are divisible by 64) to be processed by the model (constrained to 1088×1088 with 100 cm/pixel resolution by GPU (graphics processing unit) memory in this instance). The dataset consisted of 20k labeled satellite images covering diverse areas worldwide. To achieve a good set representation, the set was also enriched with samples from various areas that included mountains, glaciers, forests, deserts, beaches, coasts, etc.
Once the model was trained, road extraction was done in four stages
Semantic Segmentation – Recognizing road pixels on the aerial image using Convolutional Neural Network (CNN).
Geometry Generation – A series of algorithms and processes transforming output of semantic segmentation into roads in geometry format.
Image postprocessing
Thinning
Connectivity improvement
Graph construction;
Finalizing road shapes and network quality
Stitching roads between neighboring images (where needed)
Conflation & Cutting – Excluding roads and parts of roads that already exist in the road network (OSM).
Classification – A classifier to filter out low-confidence roads and predict a road type.
How do we know if it is good?
The Microsoft Maps AI Team measured intermediate stage metrics to track performance of the models. This focused on Pixel metric measures for the performance of the Convolutional Neural Network and APLS metric (Average Path Length Similarity) to measure overall road connectivity after the road geometry generation stage.
Metric
Precision
Recall
Pixel
85.24%
82.81%
APL
87.53%
79.33%
The “Missing” OSM Data went through a final classifier to ensure that the precision is at least 95%. After classifiers filtered out potentially bad roads, the precision was remeasured and made sure that it is 95% before releasing results.
What does this mean for the future?
The vintage of the roads depends on the vintage of the underlying imagery. As Bing Maps Imagery is a composite of multiple sources it is difficult to know the exact dates for individual pieces of data but the Microsoft Maps AI Team is now connected to the direct imagery update pipeline for Bing Maps so will continue to refine and update this dataset as new imagery is acquired.
How can this data be used?
This set of road network data is useful for a range of different applications. Having an accurate map of rural and urban roads is a necessary condition for effective long-term planning and can save precious time and resources that would otherwise have been used for surveying access options to remote areas.
One of the most popular applications of our Microsoft Maps AI data, both these new roads and building footprints, can been seen in OpenStreetMap. The Humanitarian OpenStreetMap Team is effectively crowdsourcing the identification of potentially vulnerable areas with the use of road network data. Once identified areas are mapped and verified, humanitarian organizations can act quickly in times of need.
What format is the data stored in?
Microsoft’s building footprint and new roads data is easily accessible in the GeoJson format, which is commonly used to encode a range of different geographic data. This file format has been chosen due to its compactness relative to other XML formats, as well as its easy readability. For example, here is a GeoJson file with New York as a reference point.
Just head to Microsoft.com/maps to get started and create a Bing Maps API key to discover further applications of our location data and get started with a developer-friendly mapping experience today!
High On Life is the biggest Game Pass release of 2022, the biggest 3rd party launch in Game Pass history, and the biggest single-player game launch on Game Pass ever.
High On Life is out now for Xbox One, Xbox Series X|S and Windows, and is available as part of Xbox Game Pass and PC Game Pass.
Today, we’re happy to announce that HighOn Life — the comedy-shooter featuring charismatic talking guns from Rick and Morty co-creator Justin Roiland’s Squanch Games — is Xbox Game Pass’ biggest launch of 2022, the biggest 3rd party Game Pass launch of all time, and the biggest release of a single player-only game in the service’s history. Those stats are measured in terms of engagement, based on the number of hours played in the first five days of release.
“This was our first time launching a game with Game Pass,” said Mike Fridley, Studio Director and COO of Squanch Games, “and we’ve been blown away by the response from the players who have made us the most popular game on Game Pass right now. When Squanch Games was first created, it was to make the games we wanted to play – and Game Pass is helping us reach the players that want to play those games too.”
“High On Life is an incredibly unique title that we’ve been excited about for a long time,” said Matt Percy, General Manager of Content Planning at Xbox. “It’s fantastic to see so many Game Pass members jump into it at launch to experience Squanch’s story for themselves ahead of the holidays.”
High On Life tells the story of a regular teenager who gets embroiled in a galactic conflict after an alien cartel discovers humans can be used as drugs. Of course, the only recourse is to become a bounty hunter and take the whole cartel down. A mixture of classic shooting, light RPG mechanics, and outrageous humor, High on Life has proven a huge hit with fans across the world.
In a recent podcast interview with Larry Hryb, Xbox’s Major Nelson, co-creator Justin Roiland explained how much there is to find along the way, and encouraged players to journey “off the golden path” for plenty of deeply buried details and secrets. Inspired by Roiland’s own gaming interests, including Fallout 3, the team went above and beyond to prioritize extra voices, animations, and details that reflect player decisions. To learn more, check out the interview here.
Thank you to the Xbox community and those who have joined us on this quest to save this crazy world from the alien cartel. If you aren’t among the millions of players who are already exploring the world of High On Life yet, you can grab the game right now on Xbox One, Xbox Series X|S and Windows, and with your Game Pass membership.
High On Life
Squanch Games, Inc.
☆☆☆☆☆622
★★★★★
PC Game Pass
Xbox Game Pass
From the Co-Creator of Rick & Morty and Solar Opposites, comes High On Life. Fresh out of high school with no job and no ambition, you’ve really got nothing going for you until an alien cartel that wants to get high off humanity invades Earth. Now, you and a team of charismatic talking guns must answer the hero’s call and become the deadliest intergalactic bounty hunter the cosmos has ever seen. Travel to a variety of biomes and locations across the cosmos, go up against the nefarious Garmantuous and his gang of goons, collect loot, meet unique characters, and more, in the latest comedy adventure from Justin Roiland!
– By Eric Horvitz, Chief Scientific Officer, and Saurabh Tiwary, Corporate VP and Technical Fellow, Microsoft Turing
The AI research community is focusing attention on large language models (LLMs) given their impressive performance with difficult tasks and value in useful applications. The capabilities of the models have stimulated many research questions: How do these models work and what do they learn? How can we leverage their potential for new innovations and discoveries? How can we mitigate potentially harmful behaviors? These and many other questions are inspiring new and challenging directions for AI research and underscoring the need for different perspectives and proficiencies.
To explore what’s ahead for LLMs and discuss the ways industry, academia, and government could work together to advance understanding of these models, we organized a panel titled, “Towards a Healthy Research Ecosystem for Large Language Models.” We were joined by Ahmed Awadallah from Microsoft Research, Erwin Gianchandani from the National Science Foundation, and Percy Liang from Stanford University who each brought fascinating insights and ideas for consideration.
[embedded content]
A key part of the discussion is on the need to expand access to large language models. Building and experimenting with the largest models requires a great deal of data and computing resources, which are often beyond the reach of university-based teams. This challenge was our motivation for launching the Microsoft Turing Academic Program (MS-TAP). Since 2021, we have sought to provide leading academic teams with access to some of the world’s largest language models. The program reflects our belief in the importance of having diverse and talented teams from academia working with these models.
MS-TAP has supported multiple in-depth collaborations with partner universities. With deep engagement from researchers and domain experts in Microsoft Research, Microsoft Turing, and the Office of the Chief Scientific Officer, we work to better understand model behavior, identify novel applications, explore potential risks, develop mitigations, and improve future models. Participants receive unprecedented access to our 530B parameter Natural Language Generation model (T-NLGv2), Natural Language Representation model (T-NLRv5), and 2.5B parameter Universal Image Language Representation model (T-UILRv2) and Azure compute resources to run experiments and evaluations.
University of California, Berkeley and University of California, San Francisco: Leveraging large language models for transfer learning in medical notes
Brown University: The extent to which large language models exacerbate bias when given different types of biased and unbiased inputs
École Polytechnique Fédérale de Lausanne (EPFL): Enhancing the robustness of massive language models via invariant representation learning
Georgia Tech: Analyzing and using large pretrained language models for societal good
University of Washington: Analyzing toxicity, factuality and memory
The second phase is currently underway and focuses on larger and more complex models. We are collaborating with seven universities on nine projects. Additional details about each effort including contributors are available on the MS-TAP Phase 2 Collaborations page. We will share links to forthcoming papers at the close of Phase 2.
Carnegie Mellon University: Large language models for dialog evaluation
The end of the year typically brings with it a small library of reports with predictions for the year ahead. The value in these reports is less in the precise predictions themselves—given how interconnected the world is, no one has a perfect crystal ball. Rather, the forecasts help frame the thinking about the possibilities for the coming year, and what they might mean for you. With that in mind, I would like to share five predictions for 2023 that resonated with me and explain what they could mean for endpoint management in your organization. After reviewing these predictions, I encourage you to review your current endpoint security posture, and how Microsoft Intune can help further improve it in 2023.
1. Strong cloud adoption rates will continue
Macroeconomists may be pessimistic about gross domestic product growth in Europe and the United States in 2023, but even in weak macroeconomic scenarios, cloud growth rates remain stellar.1Gartner® predicts almost 30 percent growth for infrastructure as a service and almost 25 percent growth for platform as a service in 2023, as compared to 2022 in the worldwide public cloud user spending category. A September 2022 survey of chief technology officers (CTOs) by Evercore-ISI asked the top things they would do in response to reduced budgets or inflationary pressure.2 The top answer (from 44 percent of CTOs): increase their use of the cloud. Gartner® predicts that by 2025 more than 90 percent of clients will use cloud-based unified endpoint management (UEM) tools, up from 50 percent in 2022. So, if you have not migrated your UEM to the cloud yet, 2023 is the year to start.
2. Security will remain the top issue for CTOs into 2023
When asked in September about their highest priority project (in terms of incremental spending), 42 percent of CTOs said cloud security. Network security was the second most common response, with analytics third.2 Credit Suisse recently polled CTOs on how different categories in their IT budget would grow.3 In 2021 and 2022, security was ranked top, with an 11 percent increase. Asked to predict the growth in security spending in 2026, security again ranked highest, but the expected increase was even more: 14 percent. Underlying factors provide color to the raw growth numbers. The geopolitical storm continues, and new avenues continue to emerge for hackers. I expect to hear even more about deepfake videos and ransomware as a service in 2023. So, how do chief information security officers (CISOs) strengthen their organization’s defenses in 2023? We would propose two initiatives: first, ensure security software is suitably integrated with a unified console to enable fewer points of vulnerability and more automation. By extension, this might mean consolidating vendors. Second, tackle the human aspect: invest in upskilling staff on how best to be aware of potential attacks.4
3. Worker mobility will increase further
The past few years have changed the model for knowledge workers. 2023 will see several shifts that will add to the hybrid work from anywhere (and hence, protect everywhere) trend. Next year will see mass adoption of 5G capable devices: Juniper Research estimates that there will be 600 million more 5G connections added in 2023 alone.5 Technological trends will be compounded by demographic trends, such as “productivity paranoia,” where workers want to show they are being productive, no matter where they are. What does this mean for CISOs? New working styles, new networks, and new devices mean new attack vectors. In 2023, be ready to protect your workers who are working from anywhere, not just from home.
4. CTOs will need to pay more attention to local factors
There is always a balance between global and local initiatives, but in 2023, we expect that it will be increasingly difficult to just adopt a one-size-fits-all global shortcut. We are seeing an increasing number of national regulations related to data sovereignty, with implications for where that data is stored and secured.6 2023 will see further digital transformation of public sector agencies. These agencies often have more country-specific security or compliance rules compared to their private sector counterparts. As such, CISOs need to ensure their endpoint management solutions (and, indeed, their entire technology architecture) can adapt to handle extra local requirements.
5. Truly transformative technology will rise to the top
My final prediction is that 2023 will see further clarity on the difference between genuinely transformative technology and tech that has been overhyped. One technology that I expect to compare favorably for enterprises in 2023 will be more advanced forms of automation, such as AI. AI start-ups have seen more than USD100 billion in venture capital investment since 2020, in everything from the development of new drugs to new ways to create art and writing (and, perhaps, eventually, transform how blogs are created!).7 Security represents a great opportunity for advanced automation and AI, given the nature of the ongoing problems CISOs must grapple with. As such, while new AI-generated images may garner the headlines, away from the limelight we expect many other enterprise software solutions to benefit from both sophisticated AI and simply greater automation.8 For example, in endpoint management, Gartner® sees that by 2027, UEM and digital employee experience tools will converge—to drive autonomous endpoint management, reducing human effort by at least 40 percent. The more that security tasks are automated, the more time is freed up for more strategic work by your key staff.
Learn more
I hope you found these 2023 trends thought-provoking. I would encourage you to continue to think about what the macro situation might mean specifically for your organization and translate that into an action plan for your Microsoft Intune assets in 2023. In the meantime, I wish you all a safe and thoughtful holiday season and wish you continued success in the new year.
Learn more about how Microsoft Intune can simplify your endpoint management:
To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.
56% agree the passing of video game enjoyment from older to younger generations is a great tradition.
The holiday season is a time filled with togetherness, joy and gift-giving, and this year, you can add gaming to that list! A new global survey, conducted by YouGov and commissioned by Xbox, suggests that families are using video games to better connect during the holidays – across the globe and across generations. Among survey respondents who said they plan to play video games during the holidays, 38% ranked video games as one of their top five family activities.
Not just a pastime for teens anymore, family-friendly games like Minecraft and Forza Horizon 5 accounted for more than 50% of games played during the holidays in 2021, an increase over the last 2 years [Internal Xbox Live data from November 2020 – January 2022].
The research also suggests gaming is helping create more intergenerational connections during the holiday season, for example:
Of the survey respondents who say they plan to play video games with family members during the holidays, 40% plan to game with their kids, 21% plan to game with their parents, 5% plan to game with their grandparents and 10% plan to game with their in-laws
56% of survey respondents agreed that the passing of video game enjoyment from older to younger generations is a great tradition
In addition to bringing people together during the holidays, survey findings suggest gaming’s fantastical virtual worlds provide a much-needed escape from real-world pressures to help people relax. Of the survey respondents who play video games 54% turn to video games to relieve stress, compared to other activities such as social media scrolling (46%), reading a book (45%) exercising (42%) and self-care activities (33%).
This embrace of gaming among families during the holidays also represents a new era of gift-giving. Among the survey respondents who say they are planning to celebrate the holidays and give gifts, 30% say they prefer to give gifts the whole family can enjoy. Jimmy O. Yang, actor, comedian and frequent gamer is partnering with Xbox this holiday season to help amplify great holiday games and deals the whole family can enjoy, including our Xbox Series S Gilded Hunter bundle [This Holiday Season Just Got Better: Announcing the Xbox Series S – Gilded Hunter Bundle Coming November 29 – Xbox Wire]. .
“I grew up playing video games with my family, so I was excited to partner with Xbox this holiday season,” said Yang. “I love keeping up with my friends and family across the world by playing video games, especially while I’m on tour and during the busy holidays. Plus, it keeps me humble to know that my nephew can probably crush me at Fortnite!”
Gaming is a gift that can be shared between generations. With 71% of survey respondents agreeing that the variety of video games available means that everyone in the family – from kids to grandparents – can play, Xbox is thrilled to be celebrating milestone game anniversaries this holiday season, such as Microsoft Flight Simulator (40-year franchise), Age of Empires (25-year franchise), Forza Horizon (10 year franchise) and more celebratory anniversaries. Offering these games, and many more on Xbox Game Pass is a meaningful way to instill nostalgia and shared memories into family gatherings this holiday season.
As families and young people continue to reinvent what the holidays look like for them, Xbox is proud to be a part of it. For more information on the best products and services to celebrate the holidays this year, be sure to check out Microsoft Store’s holiday gift guide and top Black Friday and Cyber Week savings. Shoppers can also browse current PC gaming deals here or visit our Black Friday Hub. For a more seamless experience, gift-givers can also set up a free online appointment with a Microsoft product expert to learn how to set up a new device like an Xbox Series S or subscription like Xbox Game Pass or PC Game Pass so it’s ready to play from the start.
About the survey
Microsoft commissioned YouGov to conduct a nationally representative survey of the age 13+ online population in each of 6 markets: United States, United Kingdom, Australia, Brazil, Germany, and Mexico. The survey focused on understanding consumer behaviors and perceptions around video gaming as it relates to other activities and traditions around the holidays. Final sample sizes were: n=1,404 in the United States, n=1,336 in the United Kingdom, n=1,254 in Australia, n=1,256 in Brazil, n=1,254 in Germany, and n=1,267 in Mexico.
We are excited to announce that Microsoft will be a featured exhibitor at CES 2023, taking place January 5 to 8, 2023 in Las Vegas, Nevada, and virtually. Known as the “most influential tech event in the world,” CES hosts what has become the place to experience innovation in the automotive and mobility industry, with a “vehicle technology” topic area and dedicated exhibition hall specific to this space. Microsoft will present exciting representations of the future of mobility and innovation including:
Metaverse and the future of car buying.
Mixed reality transforming service and customer experiences.
Connected, autonomous, and software-defined mobility.
The Future of Automotive Industry Solutions
Your transformation to smart mobility services starts here.
The theme for CES 2023 is “Human Security for All” and intends to highlight how innovations in sustainability, transportation and mobility, digital health, and the metaverse are addressing the world’s greatest challenges.
In a CES Insider Look video, Sanjay Ravi, General Manager, Automotive, Mobility, and Transportation Industry, highlights some of the key disruptions, and innovations creating what he termed a “once-in-a-century transformation.” The common industry goals of Connected, Autonomous, Shared mobility, and Electrification (CASE) have expanded into more impactful end goals of improving the societal, sustainability, and safety impact of mobility and improving the interactive and connected experiences that make our daily lives more productive and enjoyable.
Microsoft is working closely with our customers, partners, and the community and enabling them to drive the future of mobility. Data and AI are playing an increasing role in mobility. In a recent podcast interview, Sanjay talked with Alex Kendall, CEO of Wayve, who is advancing the safety and testing of autonomous vehicle technology using AI.
Transportation and aviation also featured at CES, are navigating their own roadmap to the future with a keen focus on sustainability. Contributing to more than one-third of all carbon emissions globally, we see nearly all players finding unique ways to reduce this impact, including an Alaska Airlines and Microsoft partnership to help the advancement of Sustainable Aviation Fuel (SAF) in key routes. Along with our customers and partners, we highlight our own emphasis on sustainability to be carbon negative by 2030, and by 2050 to remove our historical emissions since our founding in 1975. In addition, Microsoft is helping our customers fulfill their sustainability mission with Microsoft Cloud for Sustainability.
Innovation must lead from trust, data sovereignty, and cybersecurity and these are the core foundations at the center of empowerment for our customers and partners. Over the next five years, we are investing more than USD20 billion to continue the expansion of our cybersecurity capabilities, an imperative in our increasingly connected world.
Microsoft at CES 2023
The Microsoft CES theme of “Drive the future of mobility” embodies our vision to empower—not compete with—the industry to shape our mobility future. It is often out of duress that innovation sparks and grows. And there’s no doubt that the larger global ecosystem is under great pressure from both macro and micro influences. Organizations and individuals are having to do more when more may not always be readily available.
But as we’ve seen, the human resolve is strong and perhaps more vibrant than ever given our recent challenges. Part of the CES mission is to expose this resolve and technical prowess to the world as innovation will power us forward. Once again, we join the global community to showcase and celebrate the impact of innovation in automotive, mobility, and transportation at CES.
Subject to modifications, we are supporting these industry endeavors across five key areas:
1. Empowered Organization
To lead external transformation, organizations must first transform internally. The pandemic created unique opportunities to improve inter and intra-organizational productivity, ushering in an opportunity to drive efficiencies through a diverse, inclusive, and hybrid workforce. Here we will showcase the impact these trends have on accelerating transformation of productivity, frontline workers, citizen development, and most importantly, security.
Key showcase:
Lynk & Co.: Improving enterprise efficiency isn’t the only area of focus companies have. That efficiency is now extended to the vehicle and leveraged through Microsoft Teams to bring on-the-go, safe, and efficient productivity integration while keeping eyes on the road.
2. Resilient Operations
We have all been directly impacted by the vital links to a global supply chain and operations. Whether as a consumer, supplier, or someplace in between, improving operations is a key focus. Companies must find ways to create sustainable, agile, and collaborative supply chains that improve visibility and resilience. Many may not know that Microsoft operates a significant global manufacturing and supply chain operation. That experience along with our partner ecosystem has enabled our customers to drive rapid transformation in operations.
Factory supply chain transformation with PwC and ZF and the creation of the ZF cloud in partnership with Microsoft as a platform for innovation across factories, products, and services.
Powering the future of electric vehicles (EVs) with Rockwell and advanced battery technologyleveraging the Internet of Things (IoT), mixed reality, and product lifestyle management (PLM) integration to power the electrification of mobility.
3. Accelerate Innovation
At the core of transformation is the need to innovate. Disruptions impacting the industry have created a sense of urgency creating rapid transformation opportunities that are truly moving the industry forward at a rapid pace in many areas.
Here we highlight:
Autonomous simulation with Ansys and Applied Intuition enables advanced driver-assistance systems (ADAS) and autonomous vehicles (AV) engineering and product teams to safely develop, test, and deploy autonomous vehicles at scale.
AI, cognitive services, and the future of in-car interactions with General Motors (with the all-electric Cadillac LYRIQ in-car demonstration at the General Motors booth).
Developing the future of software-defined vehicles with Eclipse Foundation.
4. Customer Experience
A benefit of transformation at its core is creating engaging experiences that build long-term value for the customer. After all, the customer is the ultimate beneficiary of technological advances in mobility. Whether it is integration with entertainment, productivity, sales, or service—opportunities are being created that offer unique, connected, and immersive experiences.
Examples include:
5. Mobility Services
Traditional automotive, transportation, and logistics providers see themselves as part of a larger mobility ecosystem, not just creators of a product to move throughout it. Mobility is vibrant with blurred lines between old and new business models, and the creation of entirely new categories that address the future needs for the movement of people and goods.
A significant enabler of mobility services is the introduction of Microsoft Connect Fleets. A new reference architecture that creates an interconnected ecosystem of partners and makes use of common architecture, data models, and business applications from the Microsoft Cloud. Look for this announcement at CES and visit the booth to learn more.
Here we represent:
Turning connected vehicle data intelligence into actionable insights and business system integration for efficient fleet management operations with Connected Fleets, a Microsoft reference architecture. Learn how Accenture utilized Connected Fleets with the campus Connector transportation system at Microsoft campus headquarters to improve mobility options and efficiency.
Enhanced visibility of vehicle data and insights through Connected Cars and Wejo, offering insights from more than 13 million active vehicles and 18 trillion data points.
EV charging management as demonstrated by HCLTech, enhanced by the Connected Fleets reference architecture.
Learn more
If you’ll be in Las Vegas for CES 2023, stop by and visit us in booth #6017 in the Las Vegas Convention Center West Hall for hands-on, immersive experiences and an opportunity to talk with mobility experts from many of our customers and partners.
“I think with some confidence I can say that 2023 is going to be the most exciting year that the AI community has ever had,” writes Kevin Scott, chief technology officer at Microsoft, in a Q&A on the company’s AI blog. He acknowledges that he also thought 2022 was the most exciting year for AI, but he believes that the pace of innovation is only increasing. This is particularly true with generative AI, which doesn’t simply analyze large data sets but is a tool people can use to create entirely new works. We can already see its promise in systems like GPT-3, which can do anything from helping copyedit and summarize text to providing inspiration, and DALL-E 2, which can create useful and arresting works of art based on text inputs. Here are some of Scott’s predictions about how AI will change the way we work and play.
1. It Will Unleash Our Creativity
As generative AI becomes more popular and accessible, more people will be able to use the technology for creative expression, whether it’s helping them produce sophisticated artworks or write moving poetry. In his blog post, Scott describes how new AI tools are democratizing access to design. “An AI system such as DALL-E 2 doesn’t turn ordinary people into professional artists, but it gives a ton of people a visual vocabulary that they didn’t have before—a new superpower they didn’t think they would ever have.” DALL-E 2 already shows up in tools like Microsoft Designer, but there’s exciting potential for it to help many more people unleash creative ideas in ways that were once only available to trained professionals.
2. It Will Make Coding Much More Accessible
Generative AI innovations like GitHub Copilot, an AI pair programmer built using OpenAI’s Codex AI system, can translate natural human language into programming code, essentially turning our practical intentions into complex pieces of software. Among Copilot users, 40 percent of the code in some popular programming languages is being generated by Copilot, a figure that is set to increase. In a recent talk at the Fortune Brainstorm AI conference, Scott pointed to the example of people noodling around with the capabilities of ChatGPT (which is powered by GPT-3.5) to hint at the future potential. “It really opens up the aperture of who can actually use AI now,” he says. “We’ll need new sorts of specialties in the future, but you don’t need to have a PhD in computer science anymore to build an AI application, which I think is really, really exciting.”
The upshot is an acceleration of the iterative cycle, as human beings tweak and refine the AI’s work in a virtuous, back-and-forth collaborative process.
3. It Will Become Our Copilot in Other Ways Too
In an essay for Wired UK, Scott sketches a scenario in which AI helps us do our jobs better. Like coding assistance with GitHub Copilot, industries from construction to healthcare, technology to law, could potentially benefit from a form of AI assistance. “The applications are potentially endless, limited only by one’s ability to imagine scenarios in which productivity-assisting software could be applied to complex cognitive work, whether that be editing videos, writing scripts, designing new molecules for medicines, or creating manufacturing recipes from 3D models,” he writes. While there’s concern about how AI will impact human jobs, Scott describes in his post how, with thoughtful application, these AI tools have the potential to augment and amplify human capability, enabling people to spend less time on repetitive tasks. These models will also “democratize access to AI,” he writes, so “you’ll have a more diverse group of people being able to participate in the creation of technology.”
4. It Will Unlock Faster Iteration
Generative AI may significantly reduce the legwork of the creative process by helping designers iterate on product concepts and helping writers generate first drafts of press releases, essays, and scripts, assisting with graphic design–heavy posters, video edits, and more. Scott notes in his Wired essay that it has the potential to “allow knowledge workers to spend their time on higher order cognitive tasks, and effectively transforming how a great many of us interact with technology to get things done.” The upshot is an acceleration of the iterative cycle, as human beings tweak and refine the AI’s work in a virtuous, back-and-forth collaborative process. We will become adept at developing techniques to edit and modify generated images, text, drawings, and even molecules or proteins to be used in new medicines, creating better results more quickly through careful collaboration with AI.
5. It Will Make Work More Enjoyable
In the AI blog post, Scott observes that AI tools for programmers have the potential to vastly improve the overall work experience. “People now have new and interesting and fundamentally more effective tools than they’ve had before,” he notes. “This is exactly what we’re seeing with the experiences developers are having with Copilot; they are reporting that Copilot helps them stay in the flow and keeps their minds sharper during what used to be boring and repetitive tasks.” This also extends to low-code and no-code tools in products like Power Platform that are opening new potential across job functions, roles, and processes. “We did a study that found using no-code or low-code tools led to more than an 80 percent positive impact on work satisfaction, overall workload, and morale.”
Generative AI has the capacity to profoundly alter the working practices of a range of vocations, giving rise to new professions and transforming established ones. With ethical and thoughtful deployment, it is a tool that could help precipitate a revolution in creativity—one that enables everyone to better express their humanity.
2022 has seen remarkable progress in foundational technologies that have helped to advance human knowledge and create new possibilities to address some of society’s most challenging problems. Significant advances in AI have also enabled Microsoft to bring new capabilities to customers through our products and services, including GitHub Copilot, an AI pair programmer capable of turning natural language prompts into code, and a preview of Microsoft Designer, a graphic design app that supports the creation of social media posts, invitations, posters, and one-of-a-kind images.
These offerings provide an early glimpse of how new AI capabilities, such as large language models, can enable people to interact with machines in increasingly powerful ways. They build on a significant, long-term commitment to fundamental research in computing and across the sciences, and the research community at Microsoft plays an integral role in advancing the state of the art in AI, while working closely with engineering teams and other partners to transform that progress into tangible benefits.
In 2022, Microsoft Research established AI4Science, a global organization applying the latest advances in AI and machine learning toward fundamentally transforming science; added to and expanded the capabilities of the company’s family of foundation models; worked to make these models and technologies more adaptable, collaborative, and efficient; further developed approaches to ensure that AI is used responsibly and in alignment with human needs; and pursued different approaches to AI, such as causal machine learning and reinforcement learning.
We shared our advances across AI and many other disciplines during our second annual Microsoft Research Summit, where members of our research community gathered virtually with their counterparts across industry and academia to discuss how emerging technologies are being explored and deployed to bring the greatest possible benefits to humanity.
Plenary sessions at the event focused on the transformational impact of deep learning on the way we practice science, research that empowers medical practitioners and reduces inequities in healthcare, and emerging foundations for planet-scale computing. Further tracks and sessions over three days provided deeper dives into the future of the cloud; efficient large-scale AI; amplifying human productivity and creativity; delivering precision healthcare; building user trust through privacy, identity, and responsible AI; and enabling a resilient and sustainable world.
In June, the Microsoft Climate Research Initiative (MCRI) announced its first phase of collaborations among multidisciplinary researchers working together to accelerate cutting-edge research and transformative innovation in climate science and technology.
In May, researchers across Microsoft published the New Future of Work Report 2022, which summarizes important recent research developments related to hybrid work. It highlights themes that have emerged in the findings of the past year and resurfaces older research that has become newly relevant.
In this blog post, we look back at some of the key achievements and notable work in AI and highlight other advances across our diverse, multidisciplinary, and global organization.
Advancing AI foundations and accelerating progress
Researchers from Microsoft Research Asia and the Microsoft Turing team also introduced BEiT-3, a general-purpose multimodal foundation model that achieves state-of-the-art transfer performance on both vision and vision-language tasks. In BEiT-3, researchers introduce Multiway Transformers for general-purpose modeling, where the modular architecture enables both deep fusion and modality-specific encoding. Based on the shared backbone, BEiT-3 performs masked “language” modeling on images (Imglish), texts (English), and image-text pairs (“parallel sentences”) in a unified manner. The code and pretrained models will be available at GitHub.
One of the most crucial accelerators of progress in AI is the ability to optimize training and inference for large-scale models. In 2022, the DeepSpeed team made a number of breakthroughs to improve mixture of experts (MoE) models, making them more efficient, faster, and less costly. Specifically, they were able to reduce training cost by 5x, reduce MoE parameter size by up to 3.7x, and reduce MoE inference latency by 7.3x while offering up to 4.5x faster and 9x cheaper inference for MoE models compared to quality-equivalent dense models.
Transforming scientific discovery and adding societal value
Our ability to comprehend and reason about the natural world has advanced over time, and the new AI4Science organization, announced in July, represents another turn in the evolution of scientific discovery. Machine learning is already being used in the natural sciences to model physical systems using observational data. AI4Science aims to dramatically accelerate our ability to model and predict natural phenomena by creating deep learning emulators that learn by using computational solutions to fundamental equations as training data.
This new paradigm can help scientists gain greater insight into natural phenomena, right down to their smallest components. Such molecular understanding and powerful computational tools can help accelerate the discovery of new materials to combat climate change, and new drugs to help support the prevention and treatment of disease.
For instance, AI4Science’s Project Carbonix is working on globally accessible, at-scale solutions for decarbonizing the world economy, including reverse engineering materials that can pull carbon out of the environment and recycling carbon into materials. Collaborating on these efforts through the Microsoft Climate Research Initiative (MCRI) are domain experts from academia, industry, and government. Announced in June, MCRI is focused on areas such as carbon accounting, climate risk assessments, and decarbonization.
As part of the Generative Chemistry project, Microsoft researchers have been working with the global medicines company Novartis to develop and execute machine learning tools and human-in-the-loop approaches to enhance the entire drug discovery process. In April, they introduced MoLeR, a graph-based generative model for designing compounds that is more reflective of how chemists think about the process and is more efficient and practical than an earlier generative model the team developed.
Making AI more adaptable, collaborative, and efficient
To help accelerate the capabilities of large-scale AI while building a landscape in which everyone can benefit from it, the research community at Microsoft aimed to drive progress in three areas: adaptability, collaboration, and efficiency.
To provide consistent value, AI systems must respond to changes in task and environment. Research in this area includes multi-task learning with task-aware routing of inputs, knowledge-infused decoding, model repurposing with data-centric ML, pruning and cognitive science or brain-inspired AI. A good example of our work toward adaptability is GODEL, or Grounded Open DialogueLanguage Model, which ushers in a new class of pretrained language models that enable chatbots to help with tasks and then engage in more general conversations.
Microsoft’s research into more collaborative AI includes AdaTest, which leverages human expertise alongside the generative power of large language models to help people more efficiently find and correct bugs in natural language processing models. Researchers have also explored expanding the use of AI in creative processes, including a project in which science fiction writer Gabrielle Loisel used OpenAI’s GPT-3 to co-author a novella and other stories.
To enable more people to make use of AI in an efficient and sustainable way, Microsoft researchers are pursuing several new architectures and training paradigms. This includes new modular architectures and novel techniques, such as DeepSpeed Compression, a composable library for extreme compression and zero-cost quantization, and Z-Code Mixture of Experts models, which boost translation efficiency and were deployed in Microsoft Translator in 2022.
In December, researchers unveiled AutoDistil, a new technique that leverages knowledge distillation and neural architecture search to improve the balance between cost and performance when generating compressed models. They also introduced AdaMix, which improves the fine-tuning of large pretrained models for downstream tasks using mixture of adaptations modules for parameter-efficient model tuning. And vision-language model compression research on the lottery ticket hypothesis showed that pretrained language models can be significantly compressed without hurting their performance.
Cloud Intelligence/AIOps is a rapidly emerging technology trend and an interdisciplinary research direction across system, software engineering, and AI/ML communities. In this blog post from November, the researchers behind Microsoft’s AIOps work outline a research vision to make the cloud more autonomous, proactive, and manageable.
Building and deploying AI responsibly
Building AI that maximizes its benefit to humanity, and does so equitably, requires considering both the opportunities and risks that come with each new advancement in line with our guiding principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.
Helping to put these principles into practice is Microsoft’s Responsible AI Standard, which the company made publicly available in June. The standard comprises tools and steps that AI practitioners can execute in their workflows today to help ensure that building AI responsibly is baked into every stage of development. These standards will evolve as the tools and resources to responsibly build AI evolve in response to the rapid pace of AI advancement, particularly pertaining to the growing size of AI models and the new challenges they bring.
The responsible development of AI also means deploying technologies that operate the way they were designed to—and the way people expect them to. In a pair of blog posts, researchers draw on their respective experiences developing a technology to support social agency in children who are born blind and another to support mental health practitioners in guiding patient treatment to stress the need for multiple measures of performance in determining the readiness of increasingly complex AI systems and the incorporation of domain experts and user research throughout the development process.
Advancing AI for decision making
Building the next generation of AI requires continuous research into fundamental new AI innovations. Two significant areas of study in 2022 were causal ML and reinforcement learning.
Causal ML
Identifying causal effects is an integral part of scientific inquiry. It helps us understand everything from educational outcomes to the effects of social policies to risk factors for diseases. Questions of cause and effect are also critical for the design and data-driven evaluation of many technological systems we build today.
This year, Microsoft Research continued its work on causal ML, which combines traditional machine learning with causal inference methods. To help data scientists better understand and deploy causal inference, Microsoft researchers built the DoWhy library, an end-to-end causal inference tool, in 2018. To broaden access to this critical knowledge base, DoWhy has now migrated to an independent open-source governance model in a new PyWhy GitHub organization. As part of this new collaborative model, Amazon Web Services is contributing new technology based on structural causal models.
At this year’s Conference on Neural Information Processing Systems (NeurIPS), researchers presented a suite of open-source causal tools and libraries that aims to simultaneously provide core causal AI functionality to practitioners and create a platform for research advances to be rapidly deployed. This includes ShowWhy, a no-code user interface suite that empowers domain experts to become decision scientists. We hope that our work accelerates use-inspired basic research for improvement of causal AI.
Reinforcement learning (RL)
Reinforcement learning is a powerful tool for learning which behaviors are likely to produce the best outcomes in a given scenario, typically through trial and error. But this powerful tool faces some challenges. Trial and error can consume enormous resources when applied to large datasets. And for many real-time applications, there’s no room to learn from mistakes.
To address RL’s computational bottleneck, Microsoft researchers developed Path Predictive Elimination, a reinforcement learning method that is robust enough to remove noise from continuously changing environments. Also in 2022, a Microsoft team released MoCapAct, a library of pretrained simulated models to enable advanced research on artificial humanoid control at a fraction of the compute resources currently required.
Researchers also developed a new method for using offline RL to augment human-designed strategies for making critical decisions. This team deployed game theory to design algorithms that can use existing data to learn policies that improve on current strategies.
Readers’ choice: Notable blog posts for 2022
Thank you for reading
2022 was an exciting year for research, and we look forward to the future breakthroughs our global research community will deliver. In the coming year, you can expect to hear more from us about our vision, and the impact we hope to achieve. We appreciate the opportunity to share our work with you, and we hope you will subscribe to the Microsoft Research Newsletter for the latest developments.
Writers and Editors Elise Ballard Kristina Dodge Kate Forster Chris Stetkiewicz Larry West
We’re excited to announce that Microsoft is named a Leader in The Forrester Wave™: Security Analytics Platforms, Q4 2022. Microsoft achieved the highest possible score in 17 different criteria, including partner ecosystem, innovation roadmap, product security, case management, and architecture.
With threats like ransomware increasing in volume and complexity, it’s never been more important for chief information security officers (CISOs) to invest in solutions that will keep their companies safe and running. As the threat landscape continues to proliferate, cloud-native security information and event management (SIEM) solutions like Microsoft Sentinel have become a central part of a SecOps solution and have evolved to meet the new needs of customers to move faster.
We believe this placement validates our continued investment in Microsoft Sentinel, security research, and threat intelligence. We take it as a vote of confidence in our ability to keep our customers safe and working fearlessly. Microsoft Security is named a leader on seven different Forrester Wave™ reports and continues to invest in innovative solutions that work together to keep our customers’ businesses safer.
Microsoft was evaluated on several capabilities that empower customers to move faster to identify, investigate, and remediate threats. Some particularly important features include:
Providing flexibility to customers to create their own rules using Kusto Query Language (KQL) or by bringing their own machine learning. This allows security operations center (SOC) teams to build automations that work for their organization and reduces the amount of time spent on repetitive tasks.
Comprehensive threat intelligence that empowers customers to keep up with the evolving threat landscape.
Scaled search and storage of large volumes of data allow customers to protect their digital ecosystems at scale and monitor all their clouds, platforms, and endpoints in one place.
The Microsoft Sentinel strategy
Microsoft Sentinel is a next-generation SIEM solution that collects security data across multicloud, multi-platform data sources. The comprehensive SOC platform provides user entity and behavior analytics (UEBA), threat intelligence, and security orchestration, automation, and response (SOAR) capabilities, along with deep integrations into Microsoft Defender threat protection products’ comprehensive coverage across SIEM and extended detection and response (XDR). Sentinel empowers companies to leverage cloud-scale, innovative AI and automation to move at machine speed and stay ahead of evolving threats.
What makes the Microsoft suite of security solutions unique is the native integrations of SIEM with XDR to provide quick setup, more comprehensive coverage and context, and faster response time. Customers who leverage Microsoft Defender XDR products may be eligible for discounts on Microsoft Sentinel data ingestion.
Over the past year, Microsoft has invested in many new capabilities, including content for Internet of Things (IoT) devices, business application coverage including SAP, enhanced SOAR capabilities, and improved workflow management. These capabilities help our customers to protect more of their digital ecosystem, automate responses to more types of threats, and build an efficient and collaborative SOC.
What’s next in Microsoft Security
Microsoft is dedicated to continued leadership in security. Continued investments will provide customers with the intelligence, automation, and scalability they need to protect their businesses and work efficiently. Upcoming enhancements include the integration of more threat intelligence, new ways to hunt across large sets of data, and more context and prioritization guidance in alerts. New AI solutions will allow SecOps teams to more easily identify the most urgent issues and give guidance on how similar customers have reacted to similar incidents. The Microsoft vision is to provide a central platform for SOCs to understand the health of their entire business and quickly act on issues.
Microsoft Security is committed to empowering SecOps teams with security tools and platforms that enable the critical protection your users rely on. To experience Microsoft Sentinel at your organization, get started with a free trial today.
To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.
