Posted on Leave a comment

Samsung and Microsoft transform real estate and smart property management

New pilot collaboration offers single, integrated portal for managing devices and smart appliances in buildings

SEOUL, South Korea and REDMOND, Wash. — July 13, 2020 — Samsung Electronics Co. and Microsoft Corp. today announced a global collaboration focused on digitally transforming the real estate development and property management industries. This collaboration, combining smart appliances and digital cloud technologies, aims at helping to drive improved building operations and maintenance, along with creating better experiences for both service technicians and residents.

This new strategic alliance, with pilots currently under development, brings together the power of Microsoft’s Azure IoT platform and productivity cloud services with Samsung’s smart devices and SmartThings platform, to help optimize building operations, equipment maintenance, energy management, asset performance, and new tenant experiences for commercial, hospitality and residential buildings as well as mixed-use developments.

The companies will leverage Samsung’s smart home appliances, HVAC systems and smart TVs integrated with SmartThings, together with Microsoft’s Azure Digital Twins technology and Microsoft Dynamics 365 Field Service, to improve building maintenance and management by aggregating and analyzing IoT data from building systems and connected appliances. For example, with this new capability building managers can not only create an integrated dashboard for handling building issues in real time but potentially before failure, saving time and resources.

Microsoft’s Azure IoT platform is able to process data messaging from millions of building sensors and devices and then use machine learning and AI to help building managers and operators determine what issues should be addressed in what order, and then link to Dynamics 365 Field Service to determine who is the right person, with the right skills, in the right location to resolve the issue. This helps to reduce service calls while also improving the productivity of service technicians who can now troubleshoot multiple issues on a single visit and, if needed, get remote assistance through the Microsoft Teams application.

This collaboration with Samsung extends this capability to include Samsung smart appliances, HVAC systems and TVs, with plans to expand into digital signage equipment. The alliance also covers Samsung mobile devices, including the XCover Pro mobile phone, to create improved experiences for frontline workers involved in handling building issues. Additionally, Samsung plans to offer SmartThings mobile development tools to enable builders to craft custom, tailored connected living experiences for their end users.

The collaboration will leverage data from Samsung’s range of smart refrigerators, washing machines, vacuums, air purifiers, ovens and other devices connected through the intelligent SmartThings platform. Such data integration allows building operators to monitor nearly all devices in real time, identify issues and take appropriate measures before real damage happens, should a problem occur.

“We believe collaboration with a key partner like Microsoft is essential for innovation, as the company shares our vision of inspiring the world to shape the future by innovating in technology and products,” said Chanwoo Park, corporate vice president heading up the IoT Biz Group at Samsung. “Providing building owners and operators with a robust and powerful set of tools to help them optimize their building costs and equipment, including the management of Samsung’s connected appliances and other devices, is paramount to our long-term alliance. Together with Microsoft, we are helping to solve real challenges faced by our customers by creating secure integrated insight and digital solutions that keep properties functioning sustainably and efficiently while providing better experiences for residents.”

“With Azure Digital Twins, we can create comprehensive digital models of entire environments and a living digital replica of real-world things, places, business processes and people to help customers gain insights that drive better products, optimization of operations, cost reduction and breakthrough customer experiences. This collaboration with Samsung opens up new opportunities for further innovation in the real estate development and property management industries,” said Sam George, corporate vice president, Azure IoT, Microsoft. “Together, we’ll bring the best of Microsoft’s trusted, easy-to-use and secure Azure IoT platform, Azure Digital Twins and Dynamics 365 Field Service technology with Samsung’s expertise in connected devices and appliances to streamline building operations and maintenance.”

In addition to bringing new capabilities to the real estate and property management world, the companies have aligned their worldwide marketing, partner and sales programs to deliver these new integrated solutions for their customers, including facilities management companies and real estate developers.

Oxford Properties, one of the largest real estate companies in North America, also with operations in Europe and Australia, says this new alliance has the potential to add high value for customers.

“We are excited about the collaboration of Microsoft’s Digital Twins technology and Samsung’s range of connected devices, and the potential of these instruments to deliver meaningful new insights across the commercial real estate value-chain,” said Dean Hopkins, chief operating officer, Oxford Properties. “Investing in digital twins sets a foundation to unlock future opportunities. We are working with thought leaders around the world to advance the intelligence of our buildings and see enormous potential to positively impact building operations, asset management and customer experiences. Microsoft and Samsung coming together to accelerate the value that digital twins are bringing to the commercial real estate ecosystem is a great step forward.”

The National University of Singapore (NUS) will serve as a pilot for solutions pioneered under this alliance between Samsung and Microsoft as part of the university’s ongoing efforts to create a smart, safe and sustainable campus for students and staff.

“NUS is very excited to work with Microsoft and Samsung in piloting smart building management solutions on our campus,” said Professor Yong Kwet Yew, senior vice president of Campus Infrastructure at NUS. “The experience gained from this trial could help us transform the way we maintain our buildings with predictive maintenance, enable better user experiences and create a smart campus ecosystem, and it has potential to scale up at the national level.”

About Samsung Electronics Co., Ltd.

Samsung inspires the world and shapes the future with transformative ideas and technologies. The company is redefining the worlds of TVs, smartphones, wearable devices, tablets, digital appliances, network systems, and memory, system LSI, foundry and LED solutions. For the latest news, please visit the Samsung Newsroom at http://news.samsung.com.

About Microsoft

Microsoft (Nasdaq “MSFT” @microsoft) enables digital transformation for the era of an intelligent cloud and an intelligent edge. Its mission is to empower every person and every organization on the planet to achieve more.

For more information, press only:

Microsoft Media Relations, WE Communications for Microsoft, (425) 638-7777, rrt@we-worldwide.com

Note to editors: For more information, news and perspectives from Microsoft, please visit the Microsoft News Center at http://news.microsoft.com. Web links, telephone numbers and titles were correct at time of publication, but may have changed. For additional assistance, journalists and analysts may contact Microsoft’s Rapid Response Team or other appropriate contacts listed at https://news.microsoft.com/microsoft-public-relations-contacts.

Posted on Leave a comment

Microsoft Flight Simulator to launch Aug. 18 for PC; pre-install today with Xbox Game Pass for PC (Beta)

Start preparing for take-off. Xbox Game Studios and Asobo Studio are thrilled to announce that Microsoft Flight Simulator launches on August 18, and you can pre-order on Windows 10* or pre-install with Xbox Game Pass for PC (Beta) today.

We have been thrilled and humbled by the support and dedication that has fueled the Flight Sim community for the past 38 years. Since the launch of the very first Microsoft Flight Simulator in 1982, we knew it was finally the right time to develop the next-generation version of Microsoft Flight Simulator. We have the right tools, technology, partners and hardware to release the most realistic and authentic flight simulator to date. Microsoft Flight Simulator features include:

  • Vivid and Detailed Landscapes – Immerse yourself in the vast and beautiful world that is our planet with more than 37 thousand airports, 1.5 billion buildings, 2 trillion trees, mountains, roads, rivers and more.
  • A Living World – Earth is vibrant and ever-changing and so is the world of Microsoft Flight Simulator which includes live traffic, real-time weather and animals.  
  • Highly Detailed Aircraft – Hone your pilot skills in a variety of aircraft from light planes to commercial jets with comprehensive flight models. Every aircraft includes highly detailed and accurate cockpits with realistic instrumentation.
  • New Checklist System – From pro to beginner, scale your level from full manual to full assist with interactive and highlighted instrument guidance and checklist.
  • Dynamic Weather – The new weather engine enables users to switch on the live weather mode to experience real-time weather including accurate wind speed and direction, temperature, humidity, rain and more.
  • New Day & Night Engine – Experience flight at any time of day or year allowing for night VFR, visual flight rules, navigation.
  • Aerodynamic Modeling – A state-of-the-art physics engine with over 1,000 control surfaces per plane allows for a truly realistic experience.

Three Editions of Microsoft Flight Simulator

Explore all of this and more with any of the three editions of Microsoft Flight Simulator.

  • The Microsoft Flight Simulator Standard Edition ($59.99 USD) includes 20 highly detailed planes with unique flight models and 30 hand-crafted airports. The Standard Edition will be available on day one with Xbox Game Pass for PC (Beta).
  • The Deluxe Edition ($89.99 USD) includes everything from Microsoft Flight Simulator’s standard edition plus five additional highly accurate planes with unique flight models and five additional handcrafted international airports.
  • The Premium Deluxe Edition ($119.99 USD) includes everything from Microsoft Flight Simulator Standard Edition plus 10 additional highly accurate planes with unique flight models and 10 additional handcrafted international airports.

In addition to its digital versions, Microsoft Flight Simulator will be available as a retail disc version at European stores via Aerosoft. For more information on these retail disc versions in Europe, please visit the Aerosoft website.

Your fleet of planes and detailed airports from whatever edition you choose are all available on launch day as well as access to the ongoing content updates that will continually evolve and expand the flight simulation platform. 

Enhanced Airports

Airplanes

Whether you are new to flight simulation or you are an aviation pro, the sky is calling you in Microsoft Flight Simulator

*For more information on supported specs, please visit FlightSimulator.com.

Posted on Leave a comment

Video fatigue and a late-night host with no audience inspire a new way to help people feel together, remotely

When the global pandemic hit and everyone turned to video calls for work, school and happy hour, Jeremy Bailenson thought he was prepared.

Video conferencing had been around for years, after all, and the Stanford University professor had spent two decades studying and writing about digital communication and behavior. But video calls had always been more of an option than the rule, and Bailenson – along with the rest of the world – quickly found himself shocked by the impact of a complete shift to remote communication.

“After a week of shelter-in-place, I was just flabbergasted by how intense and exhausting it was,” says Bailenson, who lives in California, the first U.S. state that required residents to stay home to reduce the spread of the COVID-19 virus. “Most video conference studies are about how to improve productivity and collaboration, but the notion of it being draining hasn’t been studied.”

While Bailenson began re-reading “everything there was to read about video conferencing,” his friend at Microsoft, Jaron Lanier, was pondering a different angle to the problem. A late-night talk-show host in New York whose band Lanier occasionally played in was struggling to perform his monologue to a camera in his living room, without a live audience to react to his jokes. Lanier cast a net into Microsoft’s sea of researchers, psychologists and programmers, and within weeks he had pulled together what he calls a “magical” new feature to help the TV host and his viewers feel connected. His idea evolved into a Teams feature, Together mode, that potentially could reduce the fatigue of video calls for everyone.

Portrait of Jeremy Bailenson smiling at camera
Jeremy Bailenson, a Stanford University professor, spent two decades researching digital communication and behavior, but he was still surprised by how fatiguing it was to shift completely to remote work and video calls when the global pandemic hit this year. (Photo provided by Bailenson.)

“It was a fortuitous coincidence of needs” that led to a dramatic leap in improving remote meetings, says Lanier, a computer scientist, musician, artist and author who coined the term “virtual reality” and is considered a pioneer in the field.

Together mode, now rolling out in Microsoft Teams, combines decades of research and product development to place all the participants on a video call together in a virtual space, such as an auditorium, meeting room or coffee bar, so they look like they’re in the same place together. The new feature ditches the traditional grid of boxes, creating an environment that users say has a profound impact on the feel of the video conference and provides more cohesion to the group.

Together mode is built to give people the impression that everyone is looking at the entire group in a big virtual mirror, which Lanier says was the unique yet simple solution that changes the whole experience. People’s brains are used to being aware of others based on their locations, and the mirror effect makes it harder for the brain to notice eye contact irregularities. Those are some of the qualities that make it easier for everyone to tell how they are responding to each other.

“We’re social creatures, and the social and spatial awareness systems in the brain can finally function more naturally” within Together mode, Lanier says.

Scientists began studying problems with eye contact – or gaze misalignment – in earnest in the 1960s, and Lanier has been working to improve that element of video conferencing since the analog days of the 1970s. Yet while the technology has grown more robust and stable over the decades, there had been no real improvements to the human experience that were viable for widespread use.  Together mode uses cloud computing instead of the specialized cameras and screens that used to be needed to make video calls better.

To understand video-call fatigue, Bailenson, the founding director of Stanford’s Virtual Human Interaction Lab, combed through decades of studies on communication and found a few key causes.

For example, he says, if someone’s face looms large in your visual sphere in real life, it generally means you’re either about to fight or mate. So you’re alert and hyper-aware – reactions that are automatic and subconscious – and your heart rate goes up. And in video calls, there’s often a grid with multiple people’s faces filling the boxes. It’s a lot for your body’s nervous system to handle, he says.

In addition, people are constantly interpreting others’ eye movements, posture, how their heads are tilted and more, and attributing meaning to those non-verbal cues. Researchers in the 1960s watched videotapes of groups frame by frame, Bailenson says, and discovered a complex, intricate dance: One person would turn their head and the other would lean back a little, for example.

Posted on Leave a comment

Reimagining virtual collaboration for the future of work and learning

We’ve reached an inflection point. As the global response to COVID-19 evolves, communities around the world have moved from an era of “remote everything” into a more hybrid model of work, learning, and life. And as we all scramble to keep up, the future of work and education is being shaped before our eyes. At Microsoft, we’ve spent the last few months learning from our customers and studying how they use our tools. We’ve also worked with experts across virtual reality, AI, and productivity research to help understand the future of work. These findings, which are published here, guide us as we design technology to help our customers today and in the future.

Today we’re announcing a set of new features in Microsoft Teams that make virtual interactions more natural, more engaging, and ultimately, more human. These features offer three key benefits for people at work and in education. First, they help you feel more connected with your team and reduce meeting fatigue. Second, they make meetings more inclusive and engaging. And third, they help streamline your work and save time. It’s all about enabling people everywhere to collaborate, to stay connected, and to discover new ways to be productive from anywhere. Let’s dig into the details.

Feel more connected and reduce meeting fatigue

Together mode—At a time when people are conducting more virtual meetings than ever, our research has shown that many of us feel less connected since moving to remote work, and experience more fatigue during video meetings than during in-person collaboration. Together mode is a new meeting experience in Teams that uses AI segmentation technology to digitally place participants in a shared background, making it feel like you’re sitting in the same room with everyone else in the meeting or class. Together mode makes meetings more engaging by helping you focus on other people’s faces and body language and making it easier to pick up on the non-verbal cues that are so important to human interaction. It’s great for meetings in which multiple people will speak, such as brainstorms or roundtable discussions, because it makes it easier for participants to understand who is talking. Together mode with auditorium view is rolling out now and will be generally available in August. And we’ll bring more views to Together mode in the future.

Dynamic view—While Together mode offers an extraordinary new meeting experience, it’s not intended for every meeting. We believe that traditional video meetings people use every day can also be more engaging and dynamic. A set of enhancements we call dynamic view gives you more control over how you see shared content and other participants in a meeting. Using AI, meetings dynamically optimize shared content and video participants. New controls—including the ability to show shared content and specific participants side-by-side—let you personalize the view to suit your preferences and needs. Dynamic view builds on the meetings enhancements we announced last month, which include large gallery view (rolling out in August), where you can see video of up to 49 people in a meeting simultaneously, and virtual breakout rooms, which allow meeting organizers to split meeting participants into smaller groups for things like brainstorming sessions or workgroup discussions.

An image of the new Dynamic view.Dynamic viewAn image of the new large gallery view.Large gallery view

Video filters—We’ve all become familiar with video filters used in photography and social media apps, and now we’re bringing them to Teams. Before joining a meeting, you can use the filters to subtly adjust lighting levels and soften the focus of the camera to customize your appearance.

An animated image of background filters.

Reflect messaging extension—Our research shows that employee well-being is more important to productivity than ever. Creating an emotionally supportive environment is key to keeping people healthy, happy, and focused. The new Reflect messaging extension gives managers, leaders and teachers an easy way to check in with how their team or students are feeling — either in general, or about a specific topic like work-life balance, the status of a project, current events, or a change within the organization. IT administrators will be able to install the Reflect extension from GitHub, and then make it available to employees in their organization in the message extension menu. Once installed, the extension provides suggested check-in questions and the ability to add custom questions that team members can respond to in a poll-like experience. Managers or teachers can also choose to make poll results anonymous. The Reflect messaging extension will be available in the coming weeks.

An image of reflect messaging extension.

Make meetings more inclusive, engaging, and effective 

Live reactions—Non-verbal cues like smiles and head nods can be difficult to notice in online meetings, making it challenging for presenters to gauge audience reactions and for participants in large meetings to share a sentiment without interrupting the meeting flow. Soon, you will be able to react during a meeting using emojis that will appear to all participants. Live reactions is a shared feature with PowerPoint Live Presentations, which allows audience members to provide instant feedback to the presenter. We are also bringing PowerPoint Live Presentations to Teams in the future, further enabling audience engagement right from Teams.

An animated image of Live reactions.

Chat bubbles—During meetings, chat has become a lively space for conversation and idea-sharing, and offers an option for people to participate in the discussion without having to jump in verbally. But it can be challenging to pay attention to video feeds, presentations, and chats all at the same time. Currently, Teams users need to manually open a chat window to view the chat screen. Soon, however, chats sent during a Teams meeting will surface on the screens of all meeting participants, making the chat more central to the conversation. 

Speaker attribution for live captions and transcripts—While Teams already provides live captions as a way to follow along with what is being said in a meeting, soon we will add speaker attribution to captions so that everyone knows who is speaking. Live transcripts, coming later this year, provide another way to follow along with what has been said and who said it. After a meeting, the transcript file is automatically saved in a tab as a part of the meeting.

Note: Remarks made by participants joining from a conference room device will be attributed to the room rather than to the individuals in the room.

An animated image of Live Captions with speaker attribution.Live Captions with speaker attribution

An animated image of Live transcription with speaker attribution.Live transcription with speaker attribution

Interactive meetings for 1,000 participants and overflow—There are times when it’s important to bring large groups together for meetings or classes. For more interactive meetings—where attendees can chat, unmute to talk, and turn on their videos for real-time collaboration—Teams meetings are growing to support up to 1,000 participants. When you want to bring more people together to watch a presentation or discussion, Teams can support a view-only meeting experience for up to 20,000 participants.

Microsoft Whiteboard updates—Visual collaboration tools can make meetings and teaching environments more effective and inclusive. Whiteboard in Teams will soon be updated with new features including faster load times, sticky notes, text, and drag and drop capabilities. These features enable team members who don’t have access to a touchscreen or Surface Hub to participate in whiteboarding sessions during Teams meetings.

An image of Microsoft Whiteboard updates.

Streamline your work and save time 

Tasks app–The Tasks app in Teams, rolling out this month, provides a new unified view of tasks from across Microsoft To Do, Planner, and Outlook. Smart lists like “Assigned to me” bring tasks together across different shared plans, whether you’re on desktop, web, or mobile. Add Tasks as a tab in a channel and get your familiar Planner tab experience with the new list view.

Suggested replies—Get your message across with just one tap! Suggested replies in Teams chat uses assistive AI to create short responses based on the context of the previous message. So the next time someone asks you “Do you have time to meet today?” you can respond “I sure do!” without even pulling up your keyboard. This feature will be rolling out this month.

Cortana in Teams—Coming soon to the Teams mobile app, Cortana uses AI and the Microsoft Graph to provide voice assistance in Teams. To stay connected to your team even when you have your hands full, you can ask Cortana to make a call, join a meeting, send chat messages, share files, and more. These voice assistance experiences are delivered using Cortana enterprise-grade services that meet Microsoft 365 privacy, security, and compliance commitments. Cortana will be available in the Teams mobile app on iOS and Android in the coming weeks for Microsoft 365 Enterprise users in the U.S. in English.

Microsoft Teams displays—Organizations need to enable their employees to set up more effective home office spaces while also preparing some to return to the office. To help, we’re introducing new Microsoft Teams display, a new category of all-in-one dedicated Teams devices that feature an ambient touchscreen and a hands-free experience powered by Cortana. With natural language, users can ask Cortana to join and present in meetings, dictate replies to a Teams chat, and more. These devices seamlessly integrate with your PC, providing easy access to Teams chat, meetings, calling, calendar, and files. And with a camera shutter and microphone mute switch, your conversations stay private. The Lenovo ThinkSmart View will be the first Microsoft Teams display to market, and Yealink will deliver one of the first devices in this category too. Microsoft Teams displays with Cortana will be available in the U.S. starting later this year.

An image of PCs with Microsoft Teams desplays.

Touchless meeting experiences—As some people begin to return to their worksites, touchless meeting experiences in shared spaces are more relevant than ever. Today, Teams enables people to join meetings and share content to meeting room devices from their own mobile device or PC. Later this year, we’ll enable these capabilities on Surface Hub as well. We’re building on these capabilities with a new room remote in the Teams mobile app, which will provide additional meeting controls such as the ability to leave the meeting, mute and unmute the room, adjust audio volume, and turn cameras on and off. Beginning later this year, voice assistance will be enabled for Microsoft Teams Room devices, allowing in-room participants to ask Cortana to join and leave a meeting, add a participant from the address book to a meeting using their name or phone number, and more. We’re also introducing the ability to wirelessly cast to any Teams Room, collaboration bar, or Surface Hub device, enabling seamless ad-hoc in-person collaboration for people in a shared space.

Note: Voice commands will launch first for Microsoft 365 Enterprise users in the U.S., in English. Not all Teams Room audio devices will support Cortana voice assistance.

An image of Room remote for Microsoft Teams devices.
Room remote for Microsoft Teams devices

Unless otherwise specified, all of these features will roll out later this year.

And they all reflect our vision for the future of work: where everyone is able to contribute and do their best work; where they can move fluidly between experiences, apps, and devices; where AI lends a helping hand to streamline tasks, provide short cuts, and save you time; and where technology contributes to wellbeing and doesn’t detract from it.

From the kickoff call to the project’s launch—and all points in-between—Teams is the place where people come together to get work done. Working alongside our customers, we’ll continue reimagining the future of work and delivering technologies that put people at the center of every experience.

Posted on Leave a comment

Introducing Kernel Data Protection — platform security technology for preventing data corruption

Attackers, confronted by security technologies that prevent memory corruption, like Code Integrity (CI) and Control Flow Guard (CFG), are expectedly shifting their techniques towards data corruption. Attackers use data corruption techniques to target system security policy, escalate privileges, tamper with security attestation, modify “initialize once” data structures, among others.

Kernel Data Protection (KDP) is a new technology that prevents data corruption attacks by protecting parts of the Windows kernel and drivers through virtualization-based security (VBS). KDP is a set of APIs that provide the ability to mark some kernel memory as read-only, preventing attackers from ever modifying protected memory. For example, we’ve seen attackers use signed but vulnerable drivers to attack policy data structures and install a malicious, unsigned driver. KDP mitigates such attacks by ensuring that policy data structures cannot be tampered with.

The concept of protecting kernel memory as read-only has valuable applications for the Windows kernel, inbox components, security products, and even third-party drivers like anti-cheat and digital rights management (DRM) software. On top of the important security and tamper protection applications of this technology, other benefits include:

  • Performance improvements – KDP lessens the burden on attestation components, which would no longer need to periodically verify data variables that have been write-protected
  • Reliability improvements – KDP makes it easier to diagnose memory corruption bugs that don’t necessarily represent security vulnerabilities
  • Providing an incentive for driver developers and vendors to improve compatibility with virtualization-based security, improving adoption of these technologies in the ecosystem

KDP uses technologies that are supported by default on Secured-core PCs, which implement a specific set of device requirements that apply the security best practices of isolation and minimal trust to the technologies that underpin the Windows operating system. KDP enhances the security provided by the features that make up Secured-core PCs by adding another layer of protection for sensitive system configuration data.

In this blog we’ll share technical details about how Kernel Data Protection works and how it’s implemented on Windows 10, with the goal of inspiring and empowering driver developers and vendors to take full advantage of this technology designed to tackle data corruption attacks.

Kernel Data Protection: An overview

In VBS environments, the normal NT kernel runs in a virtualized environment called VTL0, while the secure kernel runs in a more secure and isolated environment called VTL1. More details on VBS and the secure kernel are available on Channel 9 here and here. KDP is intended to protect drivers and software running in the Windows kernel (i.e., the OS code itself) against data-driven attacks. It is implemented in two parts:

  • Static KDP enables software running in kernel mode to statically protect a section of its own image from being tampered with from any other entity in VTL0.
  • Dynamic KDP helps kernel-mode software to allocate and release read-only memory from a “secure pool”. The memory returned from the pool can be initialized only once.

The memory managed by KDP is always verified by the secure kernel (VTL1) and protected using second level address translation (SLAT) tables by the hypervisor. As a result, no software running in the NT kernel (VTL0) will ever be able to modify the content of the protected memory.

Both dynamic and static KDP, which are already available in the latest Windows 10 Insider Build and work with any kind of memory, except for executable pages. Protection for executable pages is already provided by hypervisor-protected code integrity (HVCI), which prevents any non-signed memory from being ever executable, granting the W^X (a page that is either writable or executable, but never both) condition. HVCI and the W^X conditions are not explained in this article (refer to the new upcoming Windows Internals book for further details).

Static KDP

A driver that wants a section of its image protected through static KDP should call the MmProtectDriverSection API, which has the following prototype:

NTSTATUS MmProtectDriverSection (PVOID AddressWithinSection, SIZE_T Size, ULONG Flags)

A driver specifies an address located inside a data section and, optionally, the size of the protected area and some flags. As of this writing, the “size” parameter is reserved for future use: the entire data section where the address resides will always be protected by the API.

In case the function succeeds, the memory backing the static section becomes read-only for VTL0 and protected through the SLAT. Unloading a driver that has a protected section is not allowed; attempting to do so will result in, by design, a blue screen error. However, we know that sometimes a driver should be able to be unloaded. Therefore, we have introduced the MM_PROTECT_DRIVER_SECTION_ALLOW _UNLOAD flag (1). If the caller specifies it, the system will be able to unload the target driver, which means that in this case, the protected section will be first unprotected and then released by NtUnloadDriver.

Dynamic KDP

Dynamic KDP allows a driver to allocate and initialize read-only memory using services provided by a secure pool, which is managed by the secure kernel. The consumer first creates a secure pool context associated with a tag. All of the consumer’s future memory allocations will be associated with the created secure pool context. After the context is created, read-only allocations can be performed through a new extended parameter to the ExAllocatePool3 API:

PVOID ExAllocatePool3 (POOL_FLAGS Flags, SIZE_T NumberOfBytes, ULONG Tag, PCPOOL_EXTENDED_PARAMETER ExtendedParameters, ULONG Count);

The caller can then specify the size of the allocation and the initial buffer from where to copy the memory in a POOL_EXTENDED_PARAMS_SECURE_POOL data structure. The returned memory region can’t be modified by any entity running in VTL0. In addition, at allocation time, the caller supplies a tag and a cookie value, which are encoded and embedded into the allocation. The consumer can, at any time, validate that an address is within the memory range reserved for dynamic KDP allocations and that the expected cookie and tag are in fact encoded into a given allocation. This allows the caller to check that their pointer to a secure pool allocation has not been switched with a different allocation.

Similar to static KDP, by default the memory region can’t be freed or modified. The caller can specify at allocation time that the allocation is freeable or modifiable using the SECURE_POOL_FLAGS_FREEABLE (1) and SECURE_POOL_FLAG_MODIFIABLE(2) flags. Using these flags reduces the security of allocation but allows dynamic KDP memory to be used in scenarios where leaking all allocations would be infeasible, such as allocations which are made per process on the machine.

Implementing KDP on Windows 10

As mentioned, both static KDP and dynamic KDP rely on the physical memory being protected by the SLAT in the hypervisor. When a processor supports the SLAT, it uses another layer for memory address translation. (Note: AMD implements the SLAT through “nested page tables”, while Intel uses the term “extended page tables”.)

The second-level address translation (SLAT)

When the hypervisor enables SLAT support, and a VM is executing in VMX non-root mode, the processor translates an initial virtual address called Guest Virtual Address (GVA, or stage 1 virtual address in ARM64) in an intermediate physical address called Guest Physical Address (GPA, or IPA in ARM64). This translation is still managed by the page tables, addressed by the CR3 control register managed by the Guest OS. The final result of the translation yields back to the processor a GPA, with the access protection specified in the guest page tables. Note that only software operating in kernel mode can interact with page tables. A rootkit usually operates in kernel mode and can indeed modify the protection of the intermediate physical pages.

The hypervisor helps the processor to translate the GPA using the extended (or nested) page tables. On non-SLAT systems, when a virtual address is not present in the TLB, the processor needs to consult all the page tables in the hierarchy to reconstruct the final physical address. As shown in Figure 1, the virtual address is split into four parts (on LA48 systems). Each part represents the index into a page table of the hierarchy. The physical address of the initial PML4 table is specified by the CR3 register. This explains why the processor is always able to translate the address and get the next physical address of the next table in the hierarchy. It’s important to note that in each page table entry of the hierarchy, the NT kernel specifies a page protection through a set of attributes. The final physical address is accessible only if the sum of the protections specified in each page table entry allows it.

Diagram showing X64 stage 1 address translation from virtual address to guest physical address

Figure 1. The X64 Stage 1 address translation (Virtual address to guest physical address)

When the SLAT is on, the intermediate physical address specified in the guest’s CR3 register needs to be translated to a real system physical address (SPA). The mechanism is similar: the hypervisor configures the nCR3 field of the active virtual machine control block (VMCB) representing the currently executing VM to the physical address of the nested (or extended) page tables (note that the field is called “EPT pointer” in the Intel architecture). The nested page tables are built in a similar way to standard page tables, so the processor needs to scan the entire hierarchy to find the correct physical address, as illustrated in figure 2. In the figure, “n” indicates nested page tables in the hierarchy, which is managed by the hypervisor, while “g” indicates guest page tables, which is managed by the NT Kernel.

Diagram showing X64 stage 2 physical address translation from GPA to SPA

Figure 2. The X64 Stage 2 physical address translation (GPA to SPA)

As shown, the final translation of a guest virtual address to a system physical address goes through two translation types: GVA to GPA, configured by the guest VM’s kernel, and GPA to SPA, configured by the hypervisor. Note that in the worst case, the translation involves all four page hierarchy levels, which results in 20 table lookups. The mechanism could be slow and is mitigated by processor support for an enhanced TLB. In the TLB entries, another ID that identifies the currently executing VM is included (called virtual processor identifier or VPID in Intel systems, address space ID or ASID in AMD systems), so the processor can cache the translation result of a virtual address belonging to two different VMs without any collision.

Diagram showing nested entry of an NPT page table in the hierarchy

Figure 3. Nested entry of an NPT page table in the hierarchy

As highlighted in Figure 3, an NPT entry specifies multiple access protection attributes. This allows the hypervisor to further protect the system physical address (the NPT cannot be accessed by any other entity except for the hypervisor itself). When the processor attempts to read, write, or run an address to which the NPTs disallow access, an NPT violation (EPT violation in Intel architecture) is raised, and a VM exit is generated. A VM exit generated by NTP violation does not happen frequently. In general, it is produced in nested configurations or when Software MBEC is in use for HVCI. If the NPT violation happens for other reasons, the Microsoft Hypervisor injects an access violation exception to the current virtual processor (VP), which is managed by the guest OS in different ways but typically through a bug check if no exception handler elects to handle the exception.

Static KDP implementation

The SLAT protection is the main principle that allows KDP to exist. In Windows, dynamic and static KDP implementations are similar and are managed by the secure kernel. The secure kernel is the only entity that is able to emit the ModifyVtlProtectionMask hypercall to the hypervisor with the goal of modifying the SLAT access protection for a physical page mapped in the lower VTL0.

For static KDP, the NT kernel verifies that the driver is not a session driver or mapped with large pages. If one of these conditions exists, or if the section is a discardable section, static KDP can’t be applied. If the entity that called the MmProtectDriverSection API did not request the target image to be unloadable, the NT kernel performs the first call into the secure kernel, which pins the normal address range (NAR) associated with the driver. The “pinning” operation prevents the address space of the driver from being reused, making the driver not unloadable. The NT kernel then brings all the pages that belong to the section in memory and makes them private (i.e., not addressed by the prototype PTEs). The pages are then marked as read-only in the leaf PTE structures (highlighted as “gPTE” in figure 2). At this stage, the NT kernel can finally call the secure kernel for protecting the underlying physical pages through the SLAT. The secure kernel applies the protection in two phases:

  1. Register all the physical pages that belong to the section and mark them as “owned by VTL0” by adding the proper NTEs (normal table addresses) in the database and updating the underlying secure PFNs, which belong to VTL1. This allows the secure kernel to track the physical pages, which still belong to the NT kernel.
  2. Apply the read-only protection to the VTL0 SLAT table. The hypervisor uses one SLAT table and VMCB per VTL.

The target image’s section is now protected. No entity in VTL0 will be able to write to any of the pages belonging to the section. As highlighted, the secure kernel in this scenario has protected some memory pages that were initially allocated by the NT kernel in VTL0.

Dynamic KDP implementation

Dynamic KDP uses services provided by the new segment heap for allocating memory from a secure pool, which is almost entirely managed by the secure kernel.

In early phases of its boot process, the NT memory manager calculates the randomized virtual base address of a 512GB region used for the secure pool, which spans exactly one of the 256 kernel PML4 entries. Later in phase 1, the NT memory manager emits a secure call, internally named  INITIALIZE_SECURE_POOL, which includes the calculated memory region and allows the secure kernel to initialize the secure pool.

The secure kernel creates a NAR representing the entire 512GB virtual region belonging to the unsecure NT kernel, and initializes all the relative NTEs belonging to the NAR. The secure pool virtual address space in the secure kernel is 256GB wide, which means that the PML4 mapping it is shared with some other content and is not at the same base address compared to the NT one. So, while initializing the secure pool descriptor, the secure kernel also calculates a delta value, which is the difference between the secure pool base address in the secure kernel and the one reserved in the NT kernel (as shown in figure 4). This is important, because it allows the secure kernel to specify to the NT kernel where to map a physical page belonging to the secure pool.

Diagram showing the secure pool from VTL1 to VT0 delta

Figure 4. The Secure Pool VTL1 to VTL 0 DELTA value.

When software running in VTL0 kernel requests some memory to be allocated from the secure pool, a secure call is made to the secure kernel, which invokes the internal RtlpHpAllocateHeap heap function, which is exposed in both VTLs. If the segment heap calculates that there are no more free memory segments left in the secure pool, it calls the SkmmAllocatePoolMemory routine, which allocates new memory pages for the pool. The heap always tries to avoid committing new memory pages if it doesn’t really need to.

Like the NtAllocateVirtualMemory API, which is exposed by the NT kernel, the SkmmAllocatePoolMemory API supports two kinds of operations: reserve and commit. A reserve operation allows the secure kernel’s memory manager to reserve some PTEs needed for the pool allocation. A commit operation actually allocates free physical pages.

Physical pages are allocated from a bundle of free pages that belong to the secure kernel, whose secure PFNs are in the secure state, and mapped in the VTL 1’s page table, which means that all the VTL 1 paging table hierarchy are allocated. Like static KDP, the secure kernel sends the “ModifyVtlProtectionMask” hypercall to the hypervisor, with the goal of mapping the physical pages as read-only in the VTL0 SLAT table. After the pages become accessible to VTL0, the secure kernel copies the data specified by the caller and calls back NT.

The NT kernel uses services provided by the memory manager to map the guest physical pages in VTL0. Remember that the entire root partition physical address space of both VTL0 and VTL1 is mapped with the identity mapping, meaning that a guest physical page number valid in VTL0 is also valid in VTL1. The secure kernel asks the NT memory manager to map a page belonging to the secure pool by knowing exactly which virtual address the page should be mapped to. This is thanks to the delta value calculated previously in phase 1 (figure 4).

The allocation is returned to the caller in VTL0. The underlaying pages, as with static KDP, are no more writable from any entity in VTL0.

Astute readers will note that the above description of KDP deals only with establishing SLAT protections for the guest physical address(es) backing a given protected memory region. KDP does not enforce how the virtual address range mapping a protected region is translated. Today, the secure kernel verifies only on a periodic basis that protected memory regions translate to the appropriate, SLAT-protected GPA. The design of KDP permits the possibility of future extensions to assert more direct control over the address translation hierarchy of protected memory regions.

Applications of KDP on inbox components

To demonstrate how KDP can provide value two inbox components, we’re highlighting how it’s implemented in CI.dll, the code integrity engine in Windows, and the Windows Defender System Guard runtime attestation engine.

First, CI.dll. The goal of using KDP is to protect internal policy state after it has been initialized (i.e., read from the registry or generated at boot time). These data structures are critical to protect as if they are tampered with—a driver that is properly signed but vulnerable could attack the policy data structures and then install an unsigned driver on the system. With KDP, this attack is mitigated by ensuring the policy data structures cannot be tampered with.

Second, Windows Defender System Guard. To provide runtime attestation, the attestation broker is only allowed to connect to the attestation driver one time. This is because the state is stored in VTL1 memory. The driver stores the connection state in its memory and this needs to be protected to prevent an attack from trying to reset the connection with a potentially tampered with broker agent. KDP can lock these variables and ensure that only a single connection between the broker and driver can be established.

Code integrity and Windows Defender System Guard are two of the critical features of Secured-core PCs. KDP enhances protection for these vital security systems and raise the bar that attackers need to overcome to compromise Secured-core PCs.

These are just a few examples of how useful protecting kernel and driver memory as read-only can be for the security and integrity of the system. As KDP is adopted more broadly, we expect to be able to expand the scope of protection as we look to protect against data corruption attacks more broadly.

Getting started with KDP

Both dynamic and static KDP do not have any further requirements other than the ones needed for running virtualization-based security. In ideal conditions, VBS can be started on any computer that supports:

  • Intel, AMD or ARM virtualization extensions
  • Second-level address translation: NPT for AMD, EPT for Intel, Stage 2 address translation for ARM
  • Optionally, hardware MBEC, which reduces the performance cost associated with HVCI

More info on the requirements for VBS can be found here. On Secured-core PCs, virtualization-based security is supported and hardware-backed security features are enabled by default. Customers can find Secured-core PCs from a variety of partner vendors that feature the comprehensive Secured-core security features that are now enhanced by KDP.

Andrea Allievi

Security Kernel Core Team

Posted on Leave a comment

How companies are making buildings smarter with Azure IoT

Commercial real estate developers, building owners, facilities management companies, and tenants have a huge opportunity to address, and solve for, the unique business challenges faced by their industry, by applying the Internet of Things (IoT) to buildings. For example, by leveraging data from IoT sensors and building management systems, companies can gain insights that enable them to save energy, reduce operational expenses, increase occupant comfort, and optimize space.

However, the COVID-19 crisis has presented a new set of challenges for developers, owners, and management companies. New forecasts show the smart building market size growing between 7.3 percent and 11.6 percent annually to overall market revenues of between $65.2 billion and $82.7 billion USD in 2025.1

View from an internal balcony inside a high-rise, smart building using Azure IoT looking down on a light-filled lobby filled with tables, chairs, and plants.

Smart buildings also help companies meet regulations for tracking and reducing greenhouse gas emissions.

Let’s look at how Bosch Building Technologies, Bentley Systems, Schneider Electric, and ICONICS use Azure IoT to deliver the benefits of smart buildings.

Decreasing energy requirements

The American Council for an Energy-Efficient Economy estimates that implementing smart building technology in an existing building can result in energy savings of 30–50 percent.2 For example, companies can combine data from occupancy sensors with data from HVAC and lighting systems to lower room temperatures and turn lights off in unoccupied rooms.

Bosch Building Technologies developed an in-house Energy Platform to analyze energy consumption and pursue ongoing energy efficiency. Based on Microsoft Azure, the Energy Platform monitors and analyzes energy consumption in real-time. Bosch customers use the Energy Platform to connect to IoT enabled devices and then link to existing meters, sensors, and machines. Customers can make informed decisions to improve energy and resource efficiency.

Bosch offers the solution to customers and uses it internally at more than 100 manufacturing plants worldwide. At one of their larger plants, Bosch saves up to €1.2 million (approximately $1.3 million USD) a year.

Bosch also created a Building Intelligence as a Service program to provide new IoT-based services for customers. Bosch adopted Azure Digital Twins as part of their Connected Building Services offering. By leveraging Azure Digital Twins, the company can query data from entire rooms or spaces, rather than from disparate sensors, to build complete digital models of the physical building environment.

By using Azure Digital Twins, Bosch gains more precise data for a wide range of building technology systems. With this level of precision, it’s easier for customers to fully understand data points, consumption results, context, and how they relate to the physical environment to quickly gain insights on energy usage to inform their business decisions.

Librarian checks her smartphone while standing outside an elevator near a balcony overlooking a floor below that shows a desk and books.

Human factor design of new buildings can help decrease energy requirements.

Creating a connected workplace

At Microsoft’s Frasers Tower in Singapore, Bentley Systems and Schneider Electric implemented sensors and telemetry to create a connected workplace. They used a mix of 179 Bluetooth beacons in meeting rooms and 900 sensors for lighting, air quality, and temperature. The platform generates nearly 2,100 data points that are stored and analyzed in Azure. Using the data, Microsoft optimizes various aspects of the spaces, making them more comfortable for employees, while reducing energy consumption in a sustainable and economical manner.

Additionally, Bentley Systems built a digital twin of the Fraser Towers on its Bentley iTwin platform—using Azure Digital Twins, Azure IoT Hub, and Azure Time Series Insights. The iTwin platform uses both historical and real-time data from IoT sensors to create an exact digital replica of the physical building. The building management team uses the information to dynamically allocate space, increase utilization, reduce costs, improve competitiveness, and enhance collaboration and productivity.

Maintenance worker on a ladder in conference room changing an Azure IoT sensor in a lighting fixture.

Sensors generate data that is stored and analyzed to decrease energy use.

Monitoring occupancy and reducing costs

ICONICS smart building software has run on Microsoft Azure since 2015. The software is an integration hub for building management systems that control heating, ventilation, and lighting and collect and centralize each system’s sensor data. ICONICS relies on Azure Digital Twins to boost solution scalability and rapidly deliver innovative capabilities to customers, such as viewing space occupancy and spatial analytics.

Microsoft uses the ICONICS smart building software to collect sensor data in office buildings in the Puget Sound area of Washington State. The ICONICS solution aggregates the data over multiple buildings to give facility managers visibility into building health and applies big data analytics to provide insights that drive decisions in order to deliver energy savings. In fact, the Microsoft Energy Smart Buildings program, leveraging ICONICS software, has saved Microsoft 20 percent off its energy bills.

Next steps

Smart buildings provide insights that enable real estate developers, commercial building owners, facilities managers, and tenants to save energy, reduce operational expenses, increase occupant comfort, and meet regulatory and sustainability goals.

To learn more about best practices for planning smart building projects, download the white paper, Smart buildings: From design to reality, co-written by Microsoft and L&T Technology Services.

Also visit, Azure IoT to find the right IoT approach for your solutions.

1Impact of COVID-19 on the Global IoT in Smart Commercial Buildings Market to 2025 – ResearchAndMarkets.com.

2 Smart Buildings: Using Smart Technology to Save Energy in Existing Buildings.

Posted on Leave a comment

The New Yorker: Can our ballots be both secret and secure?

Near the end of last year, I met Josh Benaloh, a senior cryptographer at Microsoft, in a conference room in Building 99 on the company’s sprawling campus, in Redmond, Washington, to talk about a fundamental problem with American elections. When we vote, we take it on faith that our ballots have been recorded—and recorded correctly. This is not always the case. In 2015, in Shelby County, Tennessee, hundreds of votes that were cast in predominantly African-American precincts disappeared somewhere between the polling place and the final tally. Where they had gone, and why, remains a mystery, because the ballots were cast on a touch-screen voting machine that did not provide a paper record. In 2018, three thousand votes went missing during a Florida recount. The next year, eight hundred uncounted ballots were found in a storage closet in Midland, Texas, after a hotly contested school-bond vote. To prevent these types of errors, Benaloh said, “You could, in theory, sign your name on your ballot and watch it go through the system.” In actual elections, however, that is precisely what is not supposed to happen. Our ballots are secret; after we drop them in the ballot box, they are, literally, out of our hands.

We don’t publish everyone’s name next to their candidate selections because, Benaloh said, “if we do that, we’ll also be opening up everyone to coercion and vote selling.” Both were features of American democracy well into the late nineteenth century, as voters revealed their choices in public—polling often took place during carnivals and festivals—either by voice or by dropping color-coded tickets, printed by each party, into a ballot box. By 1888, corruption had become so widespread that states began to abandon the spectacle. Voters in Massachusetts, following the examples of Australia and Britain, were the first in the U.S. to register their choices in a private space, on uniform ballots printed at public expense.

Since 2018, as part of a program called Defending Democracy, Benaloh has been working on voting software that attempts to solve the problem of trust in secret-ballot elections. At Microsoft, he is both a researcher and an internal consultant, using what he learns in his theoretical investigations to help the company develop secure products. His election software is based on a mathematical process that he invented called homomorphic encryption. Standard encryption obscures information behind unintelligible strings of letters and numbers; homomorphic encryption enables those unintelligible strings to be added together while still remaining behind the veil. Applied to elections, this technology could allow ballots to be aggregated, tallied, and verified without the individual votes having to be decrypted. If it worked, voters could check that their choices had been accurately counted, without anyone else ever seeing them.

At sixty years old, Benaloh is still boyish, with a stubbly beard and curly hair that is just beginning to gray. When he began thinking about how encryption might improve voting, as an undergraduate at the Massachusetts Institute of Technology, he had no sense that anything was wrong with the electoral system. “I didn’t really know a lot about elections,” Benaloh said. “I was a geeky kid growing up in New York who loved numbers, and elections were the time when everyone else was looking at numbers all day.” This was back when his surname was Cohen, before he married his wife, Laurie Blake, who was then a math teacher, and they scrambled the letters of their last names together. (“ ‘Ben’ sort of from the Latin prefix ‘benefactor,’ ” he told me, “and ‘aloh’ for the Hawaiian greeting ‘aloha.’ ”) While taking a class on cryptography, he started to see voting as a powerful way to show that the mathematical tools he was developing could be used to create a ballot that was transparent and private, and that the accuracy of elections could be verified from start to finish.

In 1987, after successfully defending his doctoral dissertation, titled “Verifiable Secret-Ballot Elections,” at Yale, Benaloh moved to Toronto, for a three-year postdoc appointment, and then to upstate New York, to teach computer science at Clarkson University. He continued to refine the math for end-to-end verifiable elections. This included an effort to figure out how to apply his research to voting by mail, which he is still attempting to do, but with more urgency, in the face of the COVID-19 pandemic. (“I’m getting close,” he told me recently.) He also settled on a method that would give voters a simple way to test the integrity of the process: they could “spoil” ballots. Unlike cast ballots, spoiled ballots would be decrypted, and anyone could check whether the choices they had made on those ballots were the ones revealed by the decryption. In 2012, Benaloh put his ideas into practice, as one of seven researchers tapped by the clerk of Travis County, Texas, to create an actual voting system from the ground up. “We were trying to design something that achieved the mathematical needs of end-to-end verifiability in a way that their voters could interact with,” he said. But STAR-Vote, as the system was called, never made it off the page and into the polling place.

In 2016, after it became clear that Russian intelligence was probing state election systems, Benaloh took part in an extensive investigation conducted by the National Academies of Sciences, Engineering, and Medicine to determine the best ways to enhance the integrity of American elections. Its September, 2018, report, “Securing the Vote: Protecting American Democracy,” offered forty-one suggestions for making voting more secure, including adding end-to-end verifiability. By then, Microsoft had witnessed attacks on the electoral system firsthand. The company had provided cybersecurity services for both parties’ conventions in the previous election cycle; in July, 2016, during the Democratic National Convention, Microsoft’s threat-intelligence team noticed that a nation-state actor, later traced to Russian intelligence, was registering fake Microsoft domain names. Not long afterward, the team saw the same thing happening during the French and European Union elections. Fake domains are often the bait for phishing expeditions, and Russian hackers were initially targeting academics and consultants likely to be involved in key issues of a campaign. “If you’ve infiltrated an academic who is going to be an adviser to the Presidential campaign, now it’s easier to hack into the Presidential campaign,” Tom Burt, the company’s vice-president for customer security and trust, told me. “That person sends an e-mail saying ‘look at this really cool document,’ and they click on it and they’re infected.”

In 2018, Microsoft created the Defending Democracy program, which offered political campaigns a service called AccountGuard. The company trained campaign staff on basic cyber hygiene and monitored their accounts for malicious activity. (AccountGuard is now offered to nonprofits, academics, and political consultants in twenty-nine countries.) The program reached out to Benaloh to ask about the possibility of using the kinds of mathematical tools he’d been developing to create a verifiable voting system. “Josh had been thinking about this for a long time, but nobody had made the investment to do it,” Burt told me. “It was going to be expensive, but it was something we could invest in, and I was willing to take a risk.” (Burt, a rugged, silver-haired veteran of corporate law, would only tell me that the cost was “in the seven-figure range.”)

Benaloh began to conceive what an end-to-end encrypted ballot-system toolkit would look like. It would be a piece of software—an add-on to voting machines or scanners, not the hardware itself. It would also be system-agnostic, able to work alongside most kinds of voting apparatuses, whether digital or analog. As Benaloh told Congress last June, with an end-to-end verifiable election system, “voters will have the ability to use their unique tracking codes to look up their encrypted votes and confirm that they are unaltered and correctly counted.” Election officials, meanwhile, he said, “will be able to publish C.V.R.S.”—cast-vote records—“without releasing sensitive raw election data that can be abused by malicious actors.”

Posted on Leave a comment

How T-Mobile used Power Apps to adapt to the COVID-19 pandemic — and define a new path forward

Business analysis managers Greg Soto and Matt McDermott – both of whom sit in T-Mobile’s Northeast Regional Business Planning organization – quickly produced a solution using Power Apps, a “low-code” method for building professional-grade applications using simple, drag-and-drop functionality and pre-built templates.

The result was the COVID-19 Employee Roster Mobile App, an easily updateable app that uses Power BI to unify detailed information from numerous sources, yielding dashboards and reports that provided T-Mobile team members with constant staffing insights via their handheld devices.

“From the first piece of code to a rough draft, it all took about 24 hours,” explains Soto. “From there, it took one more day to iron out requests and notes, and then we took it live.”

T-Mobile announced early on it would take steps to maintain income for hourly employees, and their paychecks were dependent on constant, accurate updates regarding who was working in a store, who was “on the bench” (aka willing to work, but without a physical location) and who had been assigned to a virtual retail location.

“The first screen you see asks what region you’re in,” explains McDermott. “If I were a district manager, I’d select my district and then hit the ‘new district details’ button. The next page takes you to a snapshot of every single store in your district, with all that detailed information summarized for you. So on one page, you can see how your entire market is laid out, and you can pull employees that are on the bench to stores that need them. Then you can click on a store, and edit that store specifically using detailed, automated, live, real-time information in a simple, easy-to-use interface.”

T-mobile retail employees wearing masks and gloves
Each day, retail employees can opt into or out of the opportunity to work, giving store managers a real-time view into their availability.

According to Douglas Allbright, a T-Mobile retail store manager in Syracuse, New York, the app’s arrival was a godsend. “Right around March 15, T-Mobile made the call to close my location in a mall,” he recalls. “We began operating out of a corporate store in Syracuse, combining three teams into one store — it was a very chaotic time. Suddenly, the challenge was to manage scheduling while simultaneously prioritizing everyone’s health and safety. To do that, people needed to opt in or opt out, often on a daily basis.”

The app protected employees’ privacy because it asked only whether they were available and able to work, not details about their health. And only managers like Allbright could see how many employees were available for each store and virtual retail center.

“We could suddenly see where everyone was at a glance, and share a single resource tool — I can’t say enough about how useful that was,” Allbright explains, adding that T-Mobile’s deployment of similar low-code apps in past months had laid the groundwork for its adoption. “Since we had used Power Apps previously, overnight we were able to go to the app.”

Another important benefit of the app was that when the public needed T-Mobile, they were there.

A T-Mobile employee maintains a distance of 6 feet while helping a customer
Retail customers are required to wear masks or face coverings and maintain physical distance.

“The first week, keeping that first wave of stores open, customers were coming in saying, ‘Thank you so much, thank you for being here,” Dave Holt, a district manager overseeing the Jersey Shore, recalls with pride. “We were assisting nurses, doctors, people who needed phones to connect to loved ones.”

One customer story touched him the most, Holt says.

“We had been open for two days when a nurse came in,” he remembers. “She had pre-paid service with another provider, but her phone wasn’t functioning, and she could not find one of their open stores. We did some troubleshooting on her existing phone and made sure she had communication at such an important time. We didn’t sell her anything, didn’t activate anything — she was just thankful we were there and able to help her in a safe, healthy way.”

As March rolled into April and then May, the app continued to keep track of the self-reported availability of each employee and their assigned hours. Holt also noticed, as the weeks of public isolation passed by, that keeping stores open had become important to some employees’ well-being.

“I have somebody on my team who suffers from depression,” he explains. “They came to me and said, ‘Dave, I’m really thankful that I’m able to get into a store and get working. Because if I had spent this time sitting at home, it would have made my situation much worse.’ It’s good to hear that not only did we help customers, but staying open also helped people get through a very difficult situation.”

Posted on Leave a comment

How to protect your remote workforce from application-based attacks like consent phishing

The global pandemic has dramatically shifted how people work. As a result, organizations around the world have scaled up cloud services to support collaboration and productivity from home. We’re also seeing more apps leverage Microsoft’s identity platform to ensure seamless access and integrated security as cloud app usage explodes, particularly in collaboration apps such as Zoom, Webex Teams, Box and Microsoft Teams. With increased cloud app usage and the shift to working from home, security and how employees access company resources are even more top of mind for companies.

While application use has accelerated and enabled employees to be productive remotely, attackers are looking at leveraging application-based attacks to gain unwarranted access to valuable data in cloud services. While you may be familiar with attacks focused on users, such as email phishing or credential compromise, application-based attacks, such as consent phishing, is another threat vector you must be aware of.  Today we wanted to share one of the ways application-based attacks can target the valuable data your organization cares about, and what you can do today to stay safe.

Consent phishing: An application-based threat to keep an eye on

Today developers are building apps by integrating user and organizational data from cloud platforms to enhance and personalize their experiences. These cloud platforms are rich in data but in turn have attracted malicious actors seeking to gain unwarranted access to this data. One such attack is consent phishing, where attackers trick users into granting a malicious app access to sensitive data or other resources. Instead of trying to steal the user’s password, an attacker is seeking permission for an attacker-controlled app to access valuable data.

While each attack tends to vary, the core steps usually look something like this:

  1. An attacker registers an app with an OAuth 2.0 provider, such as Azure Active Directory.
  2. The app is configured in a way that makes it seem trustworthy, like using the name of a popular product used in the same ecosystem.
  3. The attacker gets a link in front of users, which may be done through conventional email-based phishing, by compromising a non-malicious website, or other techniques.
  4. The user clicks the link and is shown an authentic consent prompt asking them to grant the malicious app permissions to data.
  5. If a user clicks accept, they will grant the app permissions to access sensitive data.
  6. The app gets an authorization code which it redeems for an access token, and potentially a refresh token.
  7. The access token is used to make API calls on behalf of the user.

If the user accepts, the attacker can gain access to their mail, forwarding rules, files, contacts, notes, profile and other sensitive data and resources.

An image of a Consent screen from a sample malicious app named “Risky App."

Consent screen from a sample malicious app named “Risky App”

How to protect your organization

At Microsoft, our integrated security solutions from identity and access management, device management, threat protection and cloud security enable us to evaluate and monitor trillions of signals to help identify malicious apps. From our signals, we’ve been able to identify and take measures to remediate malicious apps by disabling them and preventing users from accessing them. In some instances, we’ve also taken legal action to further protect our customers.

We’re also continuing to invest in ways to ensure our application ecosystem is secure by enabling customers to set policies on the types of apps users can consent to as well as highlighting apps that come from trusted publishers. While attackers will always persist, there are steps you can take to further protect your organization. Some best practices to follow include:

  • Educate your organization on consent phishing tactics:
    • Check for poor spelling and grammar. If an email message or the application’s consent screen has spelling and grammatical errors, it’s likely to be a suspicious application.
    • Keep a watchful eye on app names and domain URLs. Attackers like to spoof app names that make it appear to come from legitimate applications or companies but drive you to consent to a malicious app. Make sure you recognize the app name and domain URL before consenting to an application.
  • Promote and allow access to apps you trust:
    • Promote the use of applications that have been publisher verified. Publisher verification helps admins and end-users understand the authenticity of application developers. Over 660 applications by 390 publishers have been verified thus far.
    • Configure application consent policies by allowing users to only consent to specific applications you trust, such as application developed by your organization or from verified publishers.
  • Educate your organization on how our permissions and consent framework works:

The increased use of cloud applications has demonstrated the need to improve application security. At Microsoft, we’re committed to building capabilities that proactively protect you from malicious apps while giving you the tools to set policies that balance security and productivity. For additional best practices and safeguards review the Detect and Remediate Illicit Consent Grants in Office 365 and Five steps to securing your identity infrastructure.

Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.

Posted on Leave a comment

Azure AI: Build mission-critical AI apps with new Cognitive Services capabilities

As the world adjusts to new ways of working and staying connected, we remain committed to providing Azure AI solutions to help organizations invent with purpose.

Building on our vision to empower all developers to use AI to achieve more, today we’re excited to announce expanded capabilities within Azure Cognitive Services, including:.

  • Text Analytics for health preview.
  • Form Recognizer general availability.
  • Custom Commands general availability.
  • New Neural Text to Speech voices.

Companies in healthcare, insurance, sustainable farming, and other fields continue to choose Azure AI to build and deploy AI applications to transform their businesses. According to IDC1, by 2022, 75 percent of enterprises will deploy AI-based solutions to improve operational efficiencies and deliver enhanced customer experiences.

To meet this growing demand, today’s product updates expand on existing language, vision, and speech capabilities in Azure Cognitive Services to help developers build mission-critical AI apps that enable richer insights, save time and reduce costs, and improve customer engagement.

Get rich insights with powerful natural language processing

One of the ways organizations are adapting is scaling the ability to rapidly process data and generate new insights from data. COVID-19 has accelerated the urgency, particularly for the healthcare industry. With the overwhelming amount of healthcare data generated every year2, it is increasingly critical for providers to quickly unlock access to this information to find new solutions that improve patient outcomes.

Uncover insights in medical data with Text Analytics for health.

We are excited to introduce Text Analytics for health, a new feature of Text Analytics that enables health care providers, researchers, and companies to extract rich insights and relationships from unstructured medical data. Trained on a diverse range of medical data—covering various formats of clinical notes, clinical trials protocols, and more—the health feature is capable of processing a broad range of data types and tasks, without the need for time-intensive, manual development of custom models to extract insights from the data.

In response to the COVID-19 pandemic, Microsoft partnered with the Allen Institute of AI and leading research groups to prepare the COVID-19 Open Research Dataset. Based on the resource of over 47,000 scholarly articles, we developed a COVID-19 search engine using Text Analytics for health and Cognitive Search, enabling researchers to generate new insights in support of the fight against the disease.

Additionally, we continue to make advancements in natural language processing (NLP) so developers can more quickly build apps that generate insights about sentiment in text. The opinion mining feature in Text Analytics assigns sentiment to specific features or topics so that users can better understand customer feedback from social media data, review sites, and more.

Save time and reduce costs by turning forms into usable data

A lot of the unstructured data is contained in forms that have tables, objects, and other elements. These types of documents typically take manual labeling by document type or intensive coding to extract insights.

We’re making Form Recognizer generally available to help developers extract information from millions of documents efficiently and accurately—no data science expertise needed.

Extract text and structure from documents with Form Recognizer.

Customers like Sogeti, part of the Capgemini Group, are using Form Recognizer to help their clients more quickly process large volumes of digital documents.

“Sogeti constantly looks for new ways to help clients in their digital transformation journey by providing cutting-edge solutions in AI and machine learning. Our Cognitive Document Processing (CDP) offer enables clients to process and classify unstructured documents and extract data with high accuracy resulting in reduced operating costs and processing time. CDP leverages the powerful cognitive and tagging capabilities of the Form Recognizer to extract effortlessly, keyless paired data and other relevant information from scanned/digital unstructured documents, further reducing the overall process time.” – Mark Oost – Chief Technology Officer at Sogeti, Artificial Intelligence and Machine Learning

Wilson Allen, a leading provider of consulting and analytics solutions, is using Form Recognizer to help law and other professional services firms process and evaluate documents (PDFs and images, including financial forms, loan applications, and more), and train custom models to accurately extract values from complex forms.

“The addition of Form Recognizer to our toolkit is helping us turn large amounts of unstructured data into valuable information, saving more than 400 hours of manual data entry and freeing up time for employees to work on more strategic tasks.” – Norm Mullock – VP of Strategy at Wilson Allen

Improve customer engagement with voice-enabled apps

People and organizations continue to look for ways to enrich customer experiences while balancing the transition to digital-led, touch-free operations2. Advancements in voice technology are empowering developers to create more seamless, natural, voice-enabled experiences for customers to interact with brands.

One of those advancements, Custom Commands, a capability of Speech in Cognitive Services, is now generally available. Custom Commands allows developers to create task-oriented voice applications more easily for command-and-control scenarios that have a well-defined set of variables, like voice-controlled smart home thermostats. It brings together Speech to Text for speech recognition, Language Understanding for capturing spoken entities, and voice response with Text to Speech, to accelerate the addition of voice capabilities to your apps with a low-code authoring experience.

In addition, Neural Text to Speech is expanding language support with 15 new natural-sounding voices based on state-of-the-art neural speech synthesis models: Salma in Arabic (Egypt), Zariyah in Arabic (Saudi Arabia), Alba in Catalan (Spain), Christel in Danish (Denmark), Neerja in English (India), Noora in Finnish (Finland), Swara in Hindi (India), Colette in Dutch (Netherland), Zofia in Polish (Poland), Fernanda in Portuguese (Portugal), Dariya in Russian (Russia), Hillevi in Swedish (Sweden), Achara in Thai (Thailand), HiuGaai in Chinese (Cantonese, Traditional) and HsiaoYu in Chinese (Taiwanese Mandarin).

Customers are already adding speech capabilities to their apps to improve customer engagement. With Cognitive Services and Bot Service, the BBC created an AI-enabled voice assistant, Beeb, that delivers a more engaging, tailored experience for its diverse audiences.

We are excited to introduce these new product innovations that empower all developers to build mission-critical AI apps. To learn more, check out our resources below.

Get started today

Learn more with the resources below and get started with Azure Cognitive Services and an Azure free account.


1 Worldwide Artificial Intelligence Predictions (IDC FutureScape 2020).

2 Adapting customer experience in the time of coronavirus (McKinsey 2020).