Posted on Leave a comment

Video fatigue and a late-night host with no audience inspire a new way to help people feel together, remotely

When the global pandemic hit and everyone turned to video calls for work, school and happy hour, Jeremy Bailenson thought he was prepared.

Video conferencing had been around for years, after all, and the Stanford University professor had spent two decades studying and writing about digital communication and behavior. But video calls had always been more of an option than the rule, and Bailenson – along with the rest of the world – quickly found himself shocked by the impact of a complete shift to remote communication.

“After a week of shelter-in-place, I was just flabbergasted by how intense and exhausting it was,” says Bailenson, who lives in California, the first U.S. state that required residents to stay home to reduce the spread of the COVID-19 virus. “Most video conference studies are about how to improve productivity and collaboration, but the notion of it being draining hasn’t been studied.”

While Bailenson began re-reading “everything there was to read about video conferencing,” his friend at Microsoft, Jaron Lanier, was pondering a different angle to the problem. A late-night talk-show host in New York whose band Lanier occasionally played in was struggling to perform his monologue to a camera in his living room, without a live audience to react to his jokes. Lanier cast a net into Microsoft’s sea of researchers, psychologists and programmers, and within weeks he had pulled together what he calls a “magical” new feature to help the TV host and his viewers feel connected. His idea evolved into a Teams feature, Together mode, that potentially could reduce the fatigue of video calls for everyone.

Portrait of Jeremy Bailenson smiling at camera
Jeremy Bailenson, a Stanford University professor, spent two decades researching digital communication and behavior, but he was still surprised by how fatiguing it was to shift completely to remote work and video calls when the global pandemic hit this year. (Photo provided by Bailenson.)

“It was a fortuitous coincidence of needs” that led to a dramatic leap in improving remote meetings, says Lanier, a computer scientist, musician, artist and author who coined the term “virtual reality” and is considered a pioneer in the field.

Together mode, now rolling out in Microsoft Teams, combines decades of research and product development to place all the participants on a video call together in a virtual space, such as an auditorium, meeting room or coffee bar, so they look like they’re in the same place together. The new feature ditches the traditional grid of boxes, creating an environment that users say has a profound impact on the feel of the video conference and provides more cohesion to the group.

Together mode is built to give people the impression that everyone is looking at the entire group in a big virtual mirror, which Lanier says was the unique yet simple solution that changes the whole experience. People’s brains are used to being aware of others based on their locations, and the mirror effect makes it harder for the brain to notice eye contact irregularities. Those are some of the qualities that make it easier for everyone to tell how they are responding to each other.

“We’re social creatures, and the social and spatial awareness systems in the brain can finally function more naturally” within Together mode, Lanier says.

Scientists began studying problems with eye contact – or gaze misalignment – in earnest in the 1960s, and Lanier has been working to improve that element of video conferencing since the analog days of the 1970s. Yet while the technology has grown more robust and stable over the decades, there had been no real improvements to the human experience that were viable for widespread use.  Together mode uses cloud computing instead of the specialized cameras and screens that used to be needed to make video calls better.

To understand video-call fatigue, Bailenson, the founding director of Stanford’s Virtual Human Interaction Lab, combed through decades of studies on communication and found a few key causes.

For example, he says, if someone’s face looms large in your visual sphere in real life, it generally means you’re either about to fight or mate. So you’re alert and hyper-aware – reactions that are automatic and subconscious – and your heart rate goes up. And in video calls, there’s often a grid with multiple people’s faces filling the boxes. It’s a lot for your body’s nervous system to handle, he says.

In addition, people are constantly interpreting others’ eye movements, posture, how their heads are tilted and more, and attributing meaning to those non-verbal cues. Researchers in the 1960s watched videotapes of groups frame by frame, Bailenson says, and discovered a complex, intricate dance: One person would turn their head and the other would lean back a little, for example.

Posted on Leave a comment

Reimagining virtual collaboration for the future of work and learning

We’ve reached an inflection point. As the global response to COVID-19 evolves, communities around the world have moved from an era of “remote everything” into a more hybrid model of work, learning, and life. And as we all scramble to keep up, the future of work and education is being shaped before our eyes. At Microsoft, we’ve spent the last few months learning from our customers and studying how they use our tools. We’ve also worked with experts across virtual reality, AI, and productivity research to help understand the future of work. These findings, which are published here, guide us as we design technology to help our customers today and in the future.

Today we’re announcing a set of new features in Microsoft Teams that make virtual interactions more natural, more engaging, and ultimately, more human. These features offer three key benefits for people at work and in education. First, they help you feel more connected with your team and reduce meeting fatigue. Second, they make meetings more inclusive and engaging. And third, they help streamline your work and save time. It’s all about enabling people everywhere to collaborate, to stay connected, and to discover new ways to be productive from anywhere. Let’s dig into the details.

Feel more connected and reduce meeting fatigue

Together mode—At a time when people are conducting more virtual meetings than ever, our research has shown that many of us feel less connected since moving to remote work, and experience more fatigue during video meetings than during in-person collaboration. Together mode is a new meeting experience in Teams that uses AI segmentation technology to digitally place participants in a shared background, making it feel like you’re sitting in the same room with everyone else in the meeting or class. Together mode makes meetings more engaging by helping you focus on other people’s faces and body language and making it easier to pick up on the non-verbal cues that are so important to human interaction. It’s great for meetings in which multiple people will speak, such as brainstorms or roundtable discussions, because it makes it easier for participants to understand who is talking. Together mode with auditorium view is rolling out now and will be generally available in August. And we’ll bring more views to Together mode in the future.

Dynamic view—While Together mode offers an extraordinary new meeting experience, it’s not intended for every meeting. We believe that traditional video meetings people use every day can also be more engaging and dynamic. A set of enhancements we call dynamic view gives you more control over how you see shared content and other participants in a meeting. Using AI, meetings dynamically optimize shared content and video participants. New controls—including the ability to show shared content and specific participants side-by-side—let you personalize the view to suit your preferences and needs. Dynamic view builds on the meetings enhancements we announced last month, which include large gallery view (rolling out in August), where you can see video of up to 49 people in a meeting simultaneously, and virtual breakout rooms, which allow meeting organizers to split meeting participants into smaller groups for things like brainstorming sessions or workgroup discussions.

An image of the new Dynamic view.Dynamic viewAn image of the new large gallery view.Large gallery view

Video filters—We’ve all become familiar with video filters used in photography and social media apps, and now we’re bringing them to Teams. Before joining a meeting, you can use the filters to subtly adjust lighting levels and soften the focus of the camera to customize your appearance.

An animated image of background filters.

Reflect messaging extension—Our research shows that employee well-being is more important to productivity than ever. Creating an emotionally supportive environment is key to keeping people healthy, happy, and focused. The new Reflect messaging extension gives managers, leaders and teachers an easy way to check in with how their team or students are feeling — either in general, or about a specific topic like work-life balance, the status of a project, current events, or a change within the organization. IT administrators will be able to install the Reflect extension from GitHub, and then make it available to employees in their organization in the message extension menu. Once installed, the extension provides suggested check-in questions and the ability to add custom questions that team members can respond to in a poll-like experience. Managers or teachers can also choose to make poll results anonymous. The Reflect messaging extension will be available in the coming weeks.

An image of reflect messaging extension.

Make meetings more inclusive, engaging, and effective 

Live reactions—Non-verbal cues like smiles and head nods can be difficult to notice in online meetings, making it challenging for presenters to gauge audience reactions and for participants in large meetings to share a sentiment without interrupting the meeting flow. Soon, you will be able to react during a meeting using emojis that will appear to all participants. Live reactions is a shared feature with PowerPoint Live Presentations, which allows audience members to provide instant feedback to the presenter. We are also bringing PowerPoint Live Presentations to Teams in the future, further enabling audience engagement right from Teams.

An animated image of Live reactions.

Chat bubbles—During meetings, chat has become a lively space for conversation and idea-sharing, and offers an option for people to participate in the discussion without having to jump in verbally. But it can be challenging to pay attention to video feeds, presentations, and chats all at the same time. Currently, Teams users need to manually open a chat window to view the chat screen. Soon, however, chats sent during a Teams meeting will surface on the screens of all meeting participants, making the chat more central to the conversation. 

Speaker attribution for live captions and transcripts—While Teams already provides live captions as a way to follow along with what is being said in a meeting, soon we will add speaker attribution to captions so that everyone knows who is speaking. Live transcripts, coming later this year, provide another way to follow along with what has been said and who said it. After a meeting, the transcript file is automatically saved in a tab as a part of the meeting.

Note: Remarks made by participants joining from a conference room device will be attributed to the room rather than to the individuals in the room.

An animated image of Live Captions with speaker attribution.Live Captions with speaker attribution

An animated image of Live transcription with speaker attribution.Live transcription with speaker attribution

Interactive meetings for 1,000 participants and overflow—There are times when it’s important to bring large groups together for meetings or classes. For more interactive meetings—where attendees can chat, unmute to talk, and turn on their videos for real-time collaboration—Teams meetings are growing to support up to 1,000 participants. When you want to bring more people together to watch a presentation or discussion, Teams can support a view-only meeting experience for up to 20,000 participants.

Microsoft Whiteboard updates—Visual collaboration tools can make meetings and teaching environments more effective and inclusive. Whiteboard in Teams will soon be updated with new features including faster load times, sticky notes, text, and drag and drop capabilities. These features enable team members who don’t have access to a touchscreen or Surface Hub to participate in whiteboarding sessions during Teams meetings.

An image of Microsoft Whiteboard updates.

Streamline your work and save time 

Tasks app–The Tasks app in Teams, rolling out this month, provides a new unified view of tasks from across Microsoft To Do, Planner, and Outlook. Smart lists like “Assigned to me” bring tasks together across different shared plans, whether you’re on desktop, web, or mobile. Add Tasks as a tab in a channel and get your familiar Planner tab experience with the new list view.

Suggested replies—Get your message across with just one tap! Suggested replies in Teams chat uses assistive AI to create short responses based on the context of the previous message. So the next time someone asks you “Do you have time to meet today?” you can respond “I sure do!” without even pulling up your keyboard. This feature will be rolling out this month.

Cortana in Teams—Coming soon to the Teams mobile app, Cortana uses AI and the Microsoft Graph to provide voice assistance in Teams. To stay connected to your team even when you have your hands full, you can ask Cortana to make a call, join a meeting, send chat messages, share files, and more. These voice assistance experiences are delivered using Cortana enterprise-grade services that meet Microsoft 365 privacy, security, and compliance commitments. Cortana will be available in the Teams mobile app on iOS and Android in the coming weeks for Microsoft 365 Enterprise users in the U.S. in English.

Microsoft Teams displays—Organizations need to enable their employees to set up more effective home office spaces while also preparing some to return to the office. To help, we’re introducing new Microsoft Teams display, a new category of all-in-one dedicated Teams devices that feature an ambient touchscreen and a hands-free experience powered by Cortana. With natural language, users can ask Cortana to join and present in meetings, dictate replies to a Teams chat, and more. These devices seamlessly integrate with your PC, providing easy access to Teams chat, meetings, calling, calendar, and files. And with a camera shutter and microphone mute switch, your conversations stay private. The Lenovo ThinkSmart View will be the first Microsoft Teams display to market, and Yealink will deliver one of the first devices in this category too. Microsoft Teams displays with Cortana will be available in the U.S. starting later this year.

An image of PCs with Microsoft Teams desplays.

Touchless meeting experiences—As some people begin to return to their worksites, touchless meeting experiences in shared spaces are more relevant than ever. Today, Teams enables people to join meetings and share content to meeting room devices from their own mobile device or PC. Later this year, we’ll enable these capabilities on Surface Hub as well. We’re building on these capabilities with a new room remote in the Teams mobile app, which will provide additional meeting controls such as the ability to leave the meeting, mute and unmute the room, adjust audio volume, and turn cameras on and off. Beginning later this year, voice assistance will be enabled for Microsoft Teams Room devices, allowing in-room participants to ask Cortana to join and leave a meeting, add a participant from the address book to a meeting using their name or phone number, and more. We’re also introducing the ability to wirelessly cast to any Teams Room, collaboration bar, or Surface Hub device, enabling seamless ad-hoc in-person collaboration for people in a shared space.

Note: Voice commands will launch first for Microsoft 365 Enterprise users in the U.S., in English. Not all Teams Room audio devices will support Cortana voice assistance.

An image of Room remote for Microsoft Teams devices.
Room remote for Microsoft Teams devices

Unless otherwise specified, all of these features will roll out later this year.

And they all reflect our vision for the future of work: where everyone is able to contribute and do their best work; where they can move fluidly between experiences, apps, and devices; where AI lends a helping hand to streamline tasks, provide short cuts, and save you time; and where technology contributes to wellbeing and doesn’t detract from it.

From the kickoff call to the project’s launch—and all points in-between—Teams is the place where people come together to get work done. Working alongside our customers, we’ll continue reimagining the future of work and delivering technologies that put people at the center of every experience.

Posted on Leave a comment

Introducing Kernel Data Protection — platform security technology for preventing data corruption

Attackers, confronted by security technologies that prevent memory corruption, like Code Integrity (CI) and Control Flow Guard (CFG), are expectedly shifting their techniques towards data corruption. Attackers use data corruption techniques to target system security policy, escalate privileges, tamper with security attestation, modify “initialize once” data structures, among others.

Kernel Data Protection (KDP) is a new technology that prevents data corruption attacks by protecting parts of the Windows kernel and drivers through virtualization-based security (VBS). KDP is a set of APIs that provide the ability to mark some kernel memory as read-only, preventing attackers from ever modifying protected memory. For example, we’ve seen attackers use signed but vulnerable drivers to attack policy data structures and install a malicious, unsigned driver. KDP mitigates such attacks by ensuring that policy data structures cannot be tampered with.

The concept of protecting kernel memory as read-only has valuable applications for the Windows kernel, inbox components, security products, and even third-party drivers like anti-cheat and digital rights management (DRM) software. On top of the important security and tamper protection applications of this technology, other benefits include:

  • Performance improvements – KDP lessens the burden on attestation components, which would no longer need to periodically verify data variables that have been write-protected
  • Reliability improvements – KDP makes it easier to diagnose memory corruption bugs that don’t necessarily represent security vulnerabilities
  • Providing an incentive for driver developers and vendors to improve compatibility with virtualization-based security, improving adoption of these technologies in the ecosystem

KDP uses technologies that are supported by default on Secured-core PCs, which implement a specific set of device requirements that apply the security best practices of isolation and minimal trust to the technologies that underpin the Windows operating system. KDP enhances the security provided by the features that make up Secured-core PCs by adding another layer of protection for sensitive system configuration data.

In this blog we’ll share technical details about how Kernel Data Protection works and how it’s implemented on Windows 10, with the goal of inspiring and empowering driver developers and vendors to take full advantage of this technology designed to tackle data corruption attacks.

Kernel Data Protection: An overview

In VBS environments, the normal NT kernel runs in a virtualized environment called VTL0, while the secure kernel runs in a more secure and isolated environment called VTL1. More details on VBS and the secure kernel are available on Channel 9 here and here. KDP is intended to protect drivers and software running in the Windows kernel (i.e., the OS code itself) against data-driven attacks. It is implemented in two parts:

  • Static KDP enables software running in kernel mode to statically protect a section of its own image from being tampered with from any other entity in VTL0.
  • Dynamic KDP helps kernel-mode software to allocate and release read-only memory from a “secure pool”. The memory returned from the pool can be initialized only once.

The memory managed by KDP is always verified by the secure kernel (VTL1) and protected using second level address translation (SLAT) tables by the hypervisor. As a result, no software running in the NT kernel (VTL0) will ever be able to modify the content of the protected memory.

Both dynamic and static KDP, which are already available in the latest Windows 10 Insider Build and work with any kind of memory, except for executable pages. Protection for executable pages is already provided by hypervisor-protected code integrity (HVCI), which prevents any non-signed memory from being ever executable, granting the W^X (a page that is either writable or executable, but never both) condition. HVCI and the W^X conditions are not explained in this article (refer to the new upcoming Windows Internals book for further details).

Static KDP

A driver that wants a section of its image protected through static KDP should call the MmProtectDriverSection API, which has the following prototype:

NTSTATUS MmProtectDriverSection (PVOID AddressWithinSection, SIZE_T Size, ULONG Flags)

A driver specifies an address located inside a data section and, optionally, the size of the protected area and some flags. As of this writing, the “size” parameter is reserved for future use: the entire data section where the address resides will always be protected by the API.

In case the function succeeds, the memory backing the static section becomes read-only for VTL0 and protected through the SLAT. Unloading a driver that has a protected section is not allowed; attempting to do so will result in, by design, a blue screen error. However, we know that sometimes a driver should be able to be unloaded. Therefore, we have introduced the MM_PROTECT_DRIVER_SECTION_ALLOW _UNLOAD flag (1). If the caller specifies it, the system will be able to unload the target driver, which means that in this case, the protected section will be first unprotected and then released by NtUnloadDriver.

Dynamic KDP

Dynamic KDP allows a driver to allocate and initialize read-only memory using services provided by a secure pool, which is managed by the secure kernel. The consumer first creates a secure pool context associated with a tag. All of the consumer’s future memory allocations will be associated with the created secure pool context. After the context is created, read-only allocations can be performed through a new extended parameter to the ExAllocatePool3 API:

PVOID ExAllocatePool3 (POOL_FLAGS Flags, SIZE_T NumberOfBytes, ULONG Tag, PCPOOL_EXTENDED_PARAMETER ExtendedParameters, ULONG Count);

The caller can then specify the size of the allocation and the initial buffer from where to copy the memory in a POOL_EXTENDED_PARAMS_SECURE_POOL data structure. The returned memory region can’t be modified by any entity running in VTL0. In addition, at allocation time, the caller supplies a tag and a cookie value, which are encoded and embedded into the allocation. The consumer can, at any time, validate that an address is within the memory range reserved for dynamic KDP allocations and that the expected cookie and tag are in fact encoded into a given allocation. This allows the caller to check that their pointer to a secure pool allocation has not been switched with a different allocation.

Similar to static KDP, by default the memory region can’t be freed or modified. The caller can specify at allocation time that the allocation is freeable or modifiable using the SECURE_POOL_FLAGS_FREEABLE (1) and SECURE_POOL_FLAG_MODIFIABLE(2) flags. Using these flags reduces the security of allocation but allows dynamic KDP memory to be used in scenarios where leaking all allocations would be infeasible, such as allocations which are made per process on the machine.

Implementing KDP on Windows 10

As mentioned, both static KDP and dynamic KDP rely on the physical memory being protected by the SLAT in the hypervisor. When a processor supports the SLAT, it uses another layer for memory address translation. (Note: AMD implements the SLAT through “nested page tables”, while Intel uses the term “extended page tables”.)

The second-level address translation (SLAT)

When the hypervisor enables SLAT support, and a VM is executing in VMX non-root mode, the processor translates an initial virtual address called Guest Virtual Address (GVA, or stage 1 virtual address in ARM64) in an intermediate physical address called Guest Physical Address (GPA, or IPA in ARM64). This translation is still managed by the page tables, addressed by the CR3 control register managed by the Guest OS. The final result of the translation yields back to the processor a GPA, with the access protection specified in the guest page tables. Note that only software operating in kernel mode can interact with page tables. A rootkit usually operates in kernel mode and can indeed modify the protection of the intermediate physical pages.

The hypervisor helps the processor to translate the GPA using the extended (or nested) page tables. On non-SLAT systems, when a virtual address is not present in the TLB, the processor needs to consult all the page tables in the hierarchy to reconstruct the final physical address. As shown in Figure 1, the virtual address is split into four parts (on LA48 systems). Each part represents the index into a page table of the hierarchy. The physical address of the initial PML4 table is specified by the CR3 register. This explains why the processor is always able to translate the address and get the next physical address of the next table in the hierarchy. It’s important to note that in each page table entry of the hierarchy, the NT kernel specifies a page protection through a set of attributes. The final physical address is accessible only if the sum of the protections specified in each page table entry allows it.

Diagram showing X64 stage 1 address translation from virtual address to guest physical address

Figure 1. The X64 Stage 1 address translation (Virtual address to guest physical address)

When the SLAT is on, the intermediate physical address specified in the guest’s CR3 register needs to be translated to a real system physical address (SPA). The mechanism is similar: the hypervisor configures the nCR3 field of the active virtual machine control block (VMCB) representing the currently executing VM to the physical address of the nested (or extended) page tables (note that the field is called “EPT pointer” in the Intel architecture). The nested page tables are built in a similar way to standard page tables, so the processor needs to scan the entire hierarchy to find the correct physical address, as illustrated in figure 2. In the figure, “n” indicates nested page tables in the hierarchy, which is managed by the hypervisor, while “g” indicates guest page tables, which is managed by the NT Kernel.

Diagram showing X64 stage 2 physical address translation from GPA to SPA

Figure 2. The X64 Stage 2 physical address translation (GPA to SPA)

As shown, the final translation of a guest virtual address to a system physical address goes through two translation types: GVA to GPA, configured by the guest VM’s kernel, and GPA to SPA, configured by the hypervisor. Note that in the worst case, the translation involves all four page hierarchy levels, which results in 20 table lookups. The mechanism could be slow and is mitigated by processor support for an enhanced TLB. In the TLB entries, another ID that identifies the currently executing VM is included (called virtual processor identifier or VPID in Intel systems, address space ID or ASID in AMD systems), so the processor can cache the translation result of a virtual address belonging to two different VMs without any collision.

Diagram showing nested entry of an NPT page table in the hierarchy

Figure 3. Nested entry of an NPT page table in the hierarchy

As highlighted in Figure 3, an NPT entry specifies multiple access protection attributes. This allows the hypervisor to further protect the system physical address (the NPT cannot be accessed by any other entity except for the hypervisor itself). When the processor attempts to read, write, or run an address to which the NPTs disallow access, an NPT violation (EPT violation in Intel architecture) is raised, and a VM exit is generated. A VM exit generated by NTP violation does not happen frequently. In general, it is produced in nested configurations or when Software MBEC is in use for HVCI. If the NPT violation happens for other reasons, the Microsoft Hypervisor injects an access violation exception to the current virtual processor (VP), which is managed by the guest OS in different ways but typically through a bug check if no exception handler elects to handle the exception.

Static KDP implementation

The SLAT protection is the main principle that allows KDP to exist. In Windows, dynamic and static KDP implementations are similar and are managed by the secure kernel. The secure kernel is the only entity that is able to emit the ModifyVtlProtectionMask hypercall to the hypervisor with the goal of modifying the SLAT access protection for a physical page mapped in the lower VTL0.

For static KDP, the NT kernel verifies that the driver is not a session driver or mapped with large pages. If one of these conditions exists, or if the section is a discardable section, static KDP can’t be applied. If the entity that called the MmProtectDriverSection API did not request the target image to be unloadable, the NT kernel performs the first call into the secure kernel, which pins the normal address range (NAR) associated with the driver. The “pinning” operation prevents the address space of the driver from being reused, making the driver not unloadable. The NT kernel then brings all the pages that belong to the section in memory and makes them private (i.e., not addressed by the prototype PTEs). The pages are then marked as read-only in the leaf PTE structures (highlighted as “gPTE” in figure 2). At this stage, the NT kernel can finally call the secure kernel for protecting the underlying physical pages through the SLAT. The secure kernel applies the protection in two phases:

  1. Register all the physical pages that belong to the section and mark them as “owned by VTL0” by adding the proper NTEs (normal table addresses) in the database and updating the underlying secure PFNs, which belong to VTL1. This allows the secure kernel to track the physical pages, which still belong to the NT kernel.
  2. Apply the read-only protection to the VTL0 SLAT table. The hypervisor uses one SLAT table and VMCB per VTL.

The target image’s section is now protected. No entity in VTL0 will be able to write to any of the pages belonging to the section. As highlighted, the secure kernel in this scenario has protected some memory pages that were initially allocated by the NT kernel in VTL0.

Dynamic KDP implementation

Dynamic KDP uses services provided by the new segment heap for allocating memory from a secure pool, which is almost entirely managed by the secure kernel.

In early phases of its boot process, the NT memory manager calculates the randomized virtual base address of a 512GB region used for the secure pool, which spans exactly one of the 256 kernel PML4 entries. Later in phase 1, the NT memory manager emits a secure call, internally named  INITIALIZE_SECURE_POOL, which includes the calculated memory region and allows the secure kernel to initialize the secure pool.

The secure kernel creates a NAR representing the entire 512GB virtual region belonging to the unsecure NT kernel, and initializes all the relative NTEs belonging to the NAR. The secure pool virtual address space in the secure kernel is 256GB wide, which means that the PML4 mapping it is shared with some other content and is not at the same base address compared to the NT one. So, while initializing the secure pool descriptor, the secure kernel also calculates a delta value, which is the difference between the secure pool base address in the secure kernel and the one reserved in the NT kernel (as shown in figure 4). This is important, because it allows the secure kernel to specify to the NT kernel where to map a physical page belonging to the secure pool.

Diagram showing the secure pool from VTL1 to VT0 delta

Figure 4. The Secure Pool VTL1 to VTL 0 DELTA value.

When software running in VTL0 kernel requests some memory to be allocated from the secure pool, a secure call is made to the secure kernel, which invokes the internal RtlpHpAllocateHeap heap function, which is exposed in both VTLs. If the segment heap calculates that there are no more free memory segments left in the secure pool, it calls the SkmmAllocatePoolMemory routine, which allocates new memory pages for the pool. The heap always tries to avoid committing new memory pages if it doesn’t really need to.

Like the NtAllocateVirtualMemory API, which is exposed by the NT kernel, the SkmmAllocatePoolMemory API supports two kinds of operations: reserve and commit. A reserve operation allows the secure kernel’s memory manager to reserve some PTEs needed for the pool allocation. A commit operation actually allocates free physical pages.

Physical pages are allocated from a bundle of free pages that belong to the secure kernel, whose secure PFNs are in the secure state, and mapped in the VTL 1’s page table, which means that all the VTL 1 paging table hierarchy are allocated. Like static KDP, the secure kernel sends the “ModifyVtlProtectionMask” hypercall to the hypervisor, with the goal of mapping the physical pages as read-only in the VTL0 SLAT table. After the pages become accessible to VTL0, the secure kernel copies the data specified by the caller and calls back NT.

The NT kernel uses services provided by the memory manager to map the guest physical pages in VTL0. Remember that the entire root partition physical address space of both VTL0 and VTL1 is mapped with the identity mapping, meaning that a guest physical page number valid in VTL0 is also valid in VTL1. The secure kernel asks the NT memory manager to map a page belonging to the secure pool by knowing exactly which virtual address the page should be mapped to. This is thanks to the delta value calculated previously in phase 1 (figure 4).

The allocation is returned to the caller in VTL0. The underlaying pages, as with static KDP, are no more writable from any entity in VTL0.

Astute readers will note that the above description of KDP deals only with establishing SLAT protections for the guest physical address(es) backing a given protected memory region. KDP does not enforce how the virtual address range mapping a protected region is translated. Today, the secure kernel verifies only on a periodic basis that protected memory regions translate to the appropriate, SLAT-protected GPA. The design of KDP permits the possibility of future extensions to assert more direct control over the address translation hierarchy of protected memory regions.

Applications of KDP on inbox components

To demonstrate how KDP can provide value two inbox components, we’re highlighting how it’s implemented in CI.dll, the code integrity engine in Windows, and the Windows Defender System Guard runtime attestation engine.

First, CI.dll. The goal of using KDP is to protect internal policy state after it has been initialized (i.e., read from the registry or generated at boot time). These data structures are critical to protect as if they are tampered with—a driver that is properly signed but vulnerable could attack the policy data structures and then install an unsigned driver on the system. With KDP, this attack is mitigated by ensuring the policy data structures cannot be tampered with.

Second, Windows Defender System Guard. To provide runtime attestation, the attestation broker is only allowed to connect to the attestation driver one time. This is because the state is stored in VTL1 memory. The driver stores the connection state in its memory and this needs to be protected to prevent an attack from trying to reset the connection with a potentially tampered with broker agent. KDP can lock these variables and ensure that only a single connection between the broker and driver can be established.

Code integrity and Windows Defender System Guard are two of the critical features of Secured-core PCs. KDP enhances protection for these vital security systems and raise the bar that attackers need to overcome to compromise Secured-core PCs.

These are just a few examples of how useful protecting kernel and driver memory as read-only can be for the security and integrity of the system. As KDP is adopted more broadly, we expect to be able to expand the scope of protection as we look to protect against data corruption attacks more broadly.

Getting started with KDP

Both dynamic and static KDP do not have any further requirements other than the ones needed for running virtualization-based security. In ideal conditions, VBS can be started on any computer that supports:

  • Intel, AMD or ARM virtualization extensions
  • Second-level address translation: NPT for AMD, EPT for Intel, Stage 2 address translation for ARM
  • Optionally, hardware MBEC, which reduces the performance cost associated with HVCI

More info on the requirements for VBS can be found here. On Secured-core PCs, virtualization-based security is supported and hardware-backed security features are enabled by default. Customers can find Secured-core PCs from a variety of partner vendors that feature the comprehensive Secured-core security features that are now enhanced by KDP.

Andrea Allievi

Security Kernel Core Team

Posted on Leave a comment

How companies are making buildings smarter with Azure IoT

Commercial real estate developers, building owners, facilities management companies, and tenants have a huge opportunity to address, and solve for, the unique business challenges faced by their industry, by applying the Internet of Things (IoT) to buildings. For example, by leveraging data from IoT sensors and building management systems, companies can gain insights that enable them to save energy, reduce operational expenses, increase occupant comfort, and optimize space.

However, the COVID-19 crisis has presented a new set of challenges for developers, owners, and management companies. New forecasts show the smart building market size growing between 7.3 percent and 11.6 percent annually to overall market revenues of between $65.2 billion and $82.7 billion USD in 2025.1

View from an internal balcony inside a high-rise, smart building using Azure IoT looking down on a light-filled lobby filled with tables, chairs, and plants.

Smart buildings also help companies meet regulations for tracking and reducing greenhouse gas emissions.

Let’s look at how Bosch Building Technologies, Bentley Systems, Schneider Electric, and ICONICS use Azure IoT to deliver the benefits of smart buildings.

Decreasing energy requirements

The American Council for an Energy-Efficient Economy estimates that implementing smart building technology in an existing building can result in energy savings of 30–50 percent.2 For example, companies can combine data from occupancy sensors with data from HVAC and lighting systems to lower room temperatures and turn lights off in unoccupied rooms.

Bosch Building Technologies developed an in-house Energy Platform to analyze energy consumption and pursue ongoing energy efficiency. Based on Microsoft Azure, the Energy Platform monitors and analyzes energy consumption in real-time. Bosch customers use the Energy Platform to connect to IoT enabled devices and then link to existing meters, sensors, and machines. Customers can make informed decisions to improve energy and resource efficiency.

Bosch offers the solution to customers and uses it internally at more than 100 manufacturing plants worldwide. At one of their larger plants, Bosch saves up to €1.2 million (approximately $1.3 million USD) a year.

Bosch also created a Building Intelligence as a Service program to provide new IoT-based services for customers. Bosch adopted Azure Digital Twins as part of their Connected Building Services offering. By leveraging Azure Digital Twins, the company can query data from entire rooms or spaces, rather than from disparate sensors, to build complete digital models of the physical building environment.

By using Azure Digital Twins, Bosch gains more precise data for a wide range of building technology systems. With this level of precision, it’s easier for customers to fully understand data points, consumption results, context, and how they relate to the physical environment to quickly gain insights on energy usage to inform their business decisions.

Librarian checks her smartphone while standing outside an elevator near a balcony overlooking a floor below that shows a desk and books.

Human factor design of new buildings can help decrease energy requirements.

Creating a connected workplace

At Microsoft’s Frasers Tower in Singapore, Bentley Systems and Schneider Electric implemented sensors and telemetry to create a connected workplace. They used a mix of 179 Bluetooth beacons in meeting rooms and 900 sensors for lighting, air quality, and temperature. The platform generates nearly 2,100 data points that are stored and analyzed in Azure. Using the data, Microsoft optimizes various aspects of the spaces, making them more comfortable for employees, while reducing energy consumption in a sustainable and economical manner.

Additionally, Bentley Systems built a digital twin of the Fraser Towers on its Bentley iTwin platform—using Azure Digital Twins, Azure IoT Hub, and Azure Time Series Insights. The iTwin platform uses both historical and real-time data from IoT sensors to create an exact digital replica of the physical building. The building management team uses the information to dynamically allocate space, increase utilization, reduce costs, improve competitiveness, and enhance collaboration and productivity.

Maintenance worker on a ladder in conference room changing an Azure IoT sensor in a lighting fixture.

Sensors generate data that is stored and analyzed to decrease energy use.

Monitoring occupancy and reducing costs

ICONICS smart building software has run on Microsoft Azure since 2015. The software is an integration hub for building management systems that control heating, ventilation, and lighting and collect and centralize each system’s sensor data. ICONICS relies on Azure Digital Twins to boost solution scalability and rapidly deliver innovative capabilities to customers, such as viewing space occupancy and spatial analytics.

Microsoft uses the ICONICS smart building software to collect sensor data in office buildings in the Puget Sound area of Washington State. The ICONICS solution aggregates the data over multiple buildings to give facility managers visibility into building health and applies big data analytics to provide insights that drive decisions in order to deliver energy savings. In fact, the Microsoft Energy Smart Buildings program, leveraging ICONICS software, has saved Microsoft 20 percent off its energy bills.

Next steps

Smart buildings provide insights that enable real estate developers, commercial building owners, facilities managers, and tenants to save energy, reduce operational expenses, increase occupant comfort, and meet regulatory and sustainability goals.

To learn more about best practices for planning smart building projects, download the white paper, Smart buildings: From design to reality, co-written by Microsoft and L&T Technology Services.

Also visit, Azure IoT to find the right IoT approach for your solutions.

1Impact of COVID-19 on the Global IoT in Smart Commercial Buildings Market to 2025 – ResearchAndMarkets.com.

2 Smart Buildings: Using Smart Technology to Save Energy in Existing Buildings.

Posted on Leave a comment

The New Yorker: Can our ballots be both secret and secure?

Near the end of last year, I met Josh Benaloh, a senior cryptographer at Microsoft, in a conference room in Building 99 on the company’s sprawling campus, in Redmond, Washington, to talk about a fundamental problem with American elections. When we vote, we take it on faith that our ballots have been recorded—and recorded correctly. This is not always the case. In 2015, in Shelby County, Tennessee, hundreds of votes that were cast in predominantly African-American precincts disappeared somewhere between the polling place and the final tally. Where they had gone, and why, remains a mystery, because the ballots were cast on a touch-screen voting machine that did not provide a paper record. In 2018, three thousand votes went missing during a Florida recount. The next year, eight hundred uncounted ballots were found in a storage closet in Midland, Texas, after a hotly contested school-bond vote. To prevent these types of errors, Benaloh said, “You could, in theory, sign your name on your ballot and watch it go through the system.” In actual elections, however, that is precisely what is not supposed to happen. Our ballots are secret; after we drop them in the ballot box, they are, literally, out of our hands.

We don’t publish everyone’s name next to their candidate selections because, Benaloh said, “if we do that, we’ll also be opening up everyone to coercion and vote selling.” Both were features of American democracy well into the late nineteenth century, as voters revealed their choices in public—polling often took place during carnivals and festivals—either by voice or by dropping color-coded tickets, printed by each party, into a ballot box. By 1888, corruption had become so widespread that states began to abandon the spectacle. Voters in Massachusetts, following the examples of Australia and Britain, were the first in the U.S. to register their choices in a private space, on uniform ballots printed at public expense.

Since 2018, as part of a program called Defending Democracy, Benaloh has been working on voting software that attempts to solve the problem of trust in secret-ballot elections. At Microsoft, he is both a researcher and an internal consultant, using what he learns in his theoretical investigations to help the company develop secure products. His election software is based on a mathematical process that he invented called homomorphic encryption. Standard encryption obscures information behind unintelligible strings of letters and numbers; homomorphic encryption enables those unintelligible strings to be added together while still remaining behind the veil. Applied to elections, this technology could allow ballots to be aggregated, tallied, and verified without the individual votes having to be decrypted. If it worked, voters could check that their choices had been accurately counted, without anyone else ever seeing them.

At sixty years old, Benaloh is still boyish, with a stubbly beard and curly hair that is just beginning to gray. When he began thinking about how encryption might improve voting, as an undergraduate at the Massachusetts Institute of Technology, he had no sense that anything was wrong with the electoral system. “I didn’t really know a lot about elections,” Benaloh said. “I was a geeky kid growing up in New York who loved numbers, and elections were the time when everyone else was looking at numbers all day.” This was back when his surname was Cohen, before he married his wife, Laurie Blake, who was then a math teacher, and they scrambled the letters of their last names together. (“ ‘Ben’ sort of from the Latin prefix ‘benefactor,’ ” he told me, “and ‘aloh’ for the Hawaiian greeting ‘aloha.’ ”) While taking a class on cryptography, he started to see voting as a powerful way to show that the mathematical tools he was developing could be used to create a ballot that was transparent and private, and that the accuracy of elections could be verified from start to finish.

In 1987, after successfully defending his doctoral dissertation, titled “Verifiable Secret-Ballot Elections,” at Yale, Benaloh moved to Toronto, for a three-year postdoc appointment, and then to upstate New York, to teach computer science at Clarkson University. He continued to refine the math for end-to-end verifiable elections. This included an effort to figure out how to apply his research to voting by mail, which he is still attempting to do, but with more urgency, in the face of the COVID-19 pandemic. (“I’m getting close,” he told me recently.) He also settled on a method that would give voters a simple way to test the integrity of the process: they could “spoil” ballots. Unlike cast ballots, spoiled ballots would be decrypted, and anyone could check whether the choices they had made on those ballots were the ones revealed by the decryption. In 2012, Benaloh put his ideas into practice, as one of seven researchers tapped by the clerk of Travis County, Texas, to create an actual voting system from the ground up. “We were trying to design something that achieved the mathematical needs of end-to-end verifiability in a way that their voters could interact with,” he said. But STAR-Vote, as the system was called, never made it off the page and into the polling place.

In 2016, after it became clear that Russian intelligence was probing state election systems, Benaloh took part in an extensive investigation conducted by the National Academies of Sciences, Engineering, and Medicine to determine the best ways to enhance the integrity of American elections. Its September, 2018, report, “Securing the Vote: Protecting American Democracy,” offered forty-one suggestions for making voting more secure, including adding end-to-end verifiability. By then, Microsoft had witnessed attacks on the electoral system firsthand. The company had provided cybersecurity services for both parties’ conventions in the previous election cycle; in July, 2016, during the Democratic National Convention, Microsoft’s threat-intelligence team noticed that a nation-state actor, later traced to Russian intelligence, was registering fake Microsoft domain names. Not long afterward, the team saw the same thing happening during the French and European Union elections. Fake domains are often the bait for phishing expeditions, and Russian hackers were initially targeting academics and consultants likely to be involved in key issues of a campaign. “If you’ve infiltrated an academic who is going to be an adviser to the Presidential campaign, now it’s easier to hack into the Presidential campaign,” Tom Burt, the company’s vice-president for customer security and trust, told me. “That person sends an e-mail saying ‘look at this really cool document,’ and they click on it and they’re infected.”

In 2018, Microsoft created the Defending Democracy program, which offered political campaigns a service called AccountGuard. The company trained campaign staff on basic cyber hygiene and monitored their accounts for malicious activity. (AccountGuard is now offered to nonprofits, academics, and political consultants in twenty-nine countries.) The program reached out to Benaloh to ask about the possibility of using the kinds of mathematical tools he’d been developing to create a verifiable voting system. “Josh had been thinking about this for a long time, but nobody had made the investment to do it,” Burt told me. “It was going to be expensive, but it was something we could invest in, and I was willing to take a risk.” (Burt, a rugged, silver-haired veteran of corporate law, would only tell me that the cost was “in the seven-figure range.”)

Benaloh began to conceive what an end-to-end encrypted ballot-system toolkit would look like. It would be a piece of software—an add-on to voting machines or scanners, not the hardware itself. It would also be system-agnostic, able to work alongside most kinds of voting apparatuses, whether digital or analog. As Benaloh told Congress last June, with an end-to-end verifiable election system, “voters will have the ability to use their unique tracking codes to look up their encrypted votes and confirm that they are unaltered and correctly counted.” Election officials, meanwhile, he said, “will be able to publish C.V.R.S.”—cast-vote records—“without releasing sensitive raw election data that can be abused by malicious actors.”

Posted on Leave a comment

How to protect your remote workforce from application-based attacks like consent phishing

The global pandemic has dramatically shifted how people work. As a result, organizations around the world have scaled up cloud services to support collaboration and productivity from home. We’re also seeing more apps leverage Microsoft’s identity platform to ensure seamless access and integrated security as cloud app usage explodes, particularly in collaboration apps such as Zoom, Webex Teams, Box and Microsoft Teams. With increased cloud app usage and the shift to working from home, security and how employees access company resources are even more top of mind for companies.

While application use has accelerated and enabled employees to be productive remotely, attackers are looking at leveraging application-based attacks to gain unwarranted access to valuable data in cloud services. While you may be familiar with attacks focused on users, such as email phishing or credential compromise, application-based attacks, such as consent phishing, is another threat vector you must be aware of.  Today we wanted to share one of the ways application-based attacks can target the valuable data your organization cares about, and what you can do today to stay safe.

Consent phishing: An application-based threat to keep an eye on

Today developers are building apps by integrating user and organizational data from cloud platforms to enhance and personalize their experiences. These cloud platforms are rich in data but in turn have attracted malicious actors seeking to gain unwarranted access to this data. One such attack is consent phishing, where attackers trick users into granting a malicious app access to sensitive data or other resources. Instead of trying to steal the user’s password, an attacker is seeking permission for an attacker-controlled app to access valuable data.

While each attack tends to vary, the core steps usually look something like this:

  1. An attacker registers an app with an OAuth 2.0 provider, such as Azure Active Directory.
  2. The app is configured in a way that makes it seem trustworthy, like using the name of a popular product used in the same ecosystem.
  3. The attacker gets a link in front of users, which may be done through conventional email-based phishing, by compromising a non-malicious website, or other techniques.
  4. The user clicks the link and is shown an authentic consent prompt asking them to grant the malicious app permissions to data.
  5. If a user clicks accept, they will grant the app permissions to access sensitive data.
  6. The app gets an authorization code which it redeems for an access token, and potentially a refresh token.
  7. The access token is used to make API calls on behalf of the user.

If the user accepts, the attacker can gain access to their mail, forwarding rules, files, contacts, notes, profile and other sensitive data and resources.

An image of a Consent screen from a sample malicious app named “Risky App."

Consent screen from a sample malicious app named “Risky App”

How to protect your organization

At Microsoft, our integrated security solutions from identity and access management, device management, threat protection and cloud security enable us to evaluate and monitor trillions of signals to help identify malicious apps. From our signals, we’ve been able to identify and take measures to remediate malicious apps by disabling them and preventing users from accessing them. In some instances, we’ve also taken legal action to further protect our customers.

We’re also continuing to invest in ways to ensure our application ecosystem is secure by enabling customers to set policies on the types of apps users can consent to as well as highlighting apps that come from trusted publishers. While attackers will always persist, there are steps you can take to further protect your organization. Some best practices to follow include:

  • Educate your organization on consent phishing tactics:
    • Check for poor spelling and grammar. If an email message or the application’s consent screen has spelling and grammatical errors, it’s likely to be a suspicious application.
    • Keep a watchful eye on app names and domain URLs. Attackers like to spoof app names that make it appear to come from legitimate applications or companies but drive you to consent to a malicious app. Make sure you recognize the app name and domain URL before consenting to an application.
  • Promote and allow access to apps you trust:
    • Promote the use of applications that have been publisher verified. Publisher verification helps admins and end-users understand the authenticity of application developers. Over 660 applications by 390 publishers have been verified thus far.
    • Configure application consent policies by allowing users to only consent to specific applications you trust, such as application developed by your organization or from verified publishers.
  • Educate your organization on how our permissions and consent framework works:

The increased use of cloud applications has demonstrated the need to improve application security. At Microsoft, we’re committed to building capabilities that proactively protect you from malicious apps while giving you the tools to set policies that balance security and productivity. For additional best practices and safeguards review the Detect and Remediate Illicit Consent Grants in Office 365 and Five steps to securing your identity infrastructure.

Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.

Posted on Leave a comment

Azure AI: Build mission-critical AI apps with new Cognitive Services capabilities

As the world adjusts to new ways of working and staying connected, we remain committed to providing Azure AI solutions to help organizations invent with purpose.

Building on our vision to empower all developers to use AI to achieve more, today we’re excited to announce expanded capabilities within Azure Cognitive Services, including:.

  • Text Analytics for health preview.
  • Form Recognizer general availability.
  • Custom Commands general availability.
  • New Neural Text to Speech voices.

Companies in healthcare, insurance, sustainable farming, and other fields continue to choose Azure AI to build and deploy AI applications to transform their businesses. According to IDC1, by 2022, 75 percent of enterprises will deploy AI-based solutions to improve operational efficiencies and deliver enhanced customer experiences.

To meet this growing demand, today’s product updates expand on existing language, vision, and speech capabilities in Azure Cognitive Services to help developers build mission-critical AI apps that enable richer insights, save time and reduce costs, and improve customer engagement.

Get rich insights with powerful natural language processing

One of the ways organizations are adapting is scaling the ability to rapidly process data and generate new insights from data. COVID-19 has accelerated the urgency, particularly for the healthcare industry. With the overwhelming amount of healthcare data generated every year2, it is increasingly critical for providers to quickly unlock access to this information to find new solutions that improve patient outcomes.

Uncover insights in medical data with Text Analytics for health.

We are excited to introduce Text Analytics for health, a new feature of Text Analytics that enables health care providers, researchers, and companies to extract rich insights and relationships from unstructured medical data. Trained on a diverse range of medical data—covering various formats of clinical notes, clinical trials protocols, and more—the health feature is capable of processing a broad range of data types and tasks, without the need for time-intensive, manual development of custom models to extract insights from the data.

In response to the COVID-19 pandemic, Microsoft partnered with the Allen Institute of AI and leading research groups to prepare the COVID-19 Open Research Dataset. Based on the resource of over 47,000 scholarly articles, we developed a COVID-19 search engine using Text Analytics for health and Cognitive Search, enabling researchers to generate new insights in support of the fight against the disease.

Additionally, we continue to make advancements in natural language processing (NLP) so developers can more quickly build apps that generate insights about sentiment in text. The opinion mining feature in Text Analytics assigns sentiment to specific features or topics so that users can better understand customer feedback from social media data, review sites, and more.

Save time and reduce costs by turning forms into usable data

A lot of the unstructured data is contained in forms that have tables, objects, and other elements. These types of documents typically take manual labeling by document type or intensive coding to extract insights.

We’re making Form Recognizer generally available to help developers extract information from millions of documents efficiently and accurately—no data science expertise needed.

Extract text and structure from documents with Form Recognizer.

Customers like Sogeti, part of the Capgemini Group, are using Form Recognizer to help their clients more quickly process large volumes of digital documents.

“Sogeti constantly looks for new ways to help clients in their digital transformation journey by providing cutting-edge solutions in AI and machine learning. Our Cognitive Document Processing (CDP) offer enables clients to process and classify unstructured documents and extract data with high accuracy resulting in reduced operating costs and processing time. CDP leverages the powerful cognitive and tagging capabilities of the Form Recognizer to extract effortlessly, keyless paired data and other relevant information from scanned/digital unstructured documents, further reducing the overall process time.” – Mark Oost – Chief Technology Officer at Sogeti, Artificial Intelligence and Machine Learning

Wilson Allen, a leading provider of consulting and analytics solutions, is using Form Recognizer to help law and other professional services firms process and evaluate documents (PDFs and images, including financial forms, loan applications, and more), and train custom models to accurately extract values from complex forms.

“The addition of Form Recognizer to our toolkit is helping us turn large amounts of unstructured data into valuable information, saving more than 400 hours of manual data entry and freeing up time for employees to work on more strategic tasks.” – Norm Mullock – VP of Strategy at Wilson Allen

Improve customer engagement with voice-enabled apps

People and organizations continue to look for ways to enrich customer experiences while balancing the transition to digital-led, touch-free operations2. Advancements in voice technology are empowering developers to create more seamless, natural, voice-enabled experiences for customers to interact with brands.

One of those advancements, Custom Commands, a capability of Speech in Cognitive Services, is now generally available. Custom Commands allows developers to create task-oriented voice applications more easily for command-and-control scenarios that have a well-defined set of variables, like voice-controlled smart home thermostats. It brings together Speech to Text for speech recognition, Language Understanding for capturing spoken entities, and voice response with Text to Speech, to accelerate the addition of voice capabilities to your apps with a low-code authoring experience.

In addition, Neural Text to Speech is expanding language support with 15 new natural-sounding voices based on state-of-the-art neural speech synthesis models: Salma in Arabic (Egypt), Zariyah in Arabic (Saudi Arabia), Alba in Catalan (Spain), Christel in Danish (Denmark), Neerja in English (India), Noora in Finnish (Finland), Swara in Hindi (India), Colette in Dutch (Netherland), Zofia in Polish (Poland), Fernanda in Portuguese (Portugal), Dariya in Russian (Russia), Hillevi in Swedish (Sweden), Achara in Thai (Thailand), HiuGaai in Chinese (Cantonese, Traditional) and HsiaoYu in Chinese (Taiwanese Mandarin).

Customers are already adding speech capabilities to their apps to improve customer engagement. With Cognitive Services and Bot Service, the BBC created an AI-enabled voice assistant, Beeb, that delivers a more engaging, tailored experience for its diverse audiences.

We are excited to introduce these new product innovations that empower all developers to build mission-critical AI apps. To learn more, check out our resources below.

Get started today

Learn more with the resources below and get started with Azure Cognitive Services and an Azure free account.


1 Worldwide Artificial Intelligence Predictions (IDC FutureScape 2020).

2 Adapting customer experience in the time of coronavirus (McKinsey 2020).

Posted on Leave a comment

New AI for Accessibility grantee projects aim to drive mental health research, data insights and innovations

Through our work in the Microsoft AI for Accessibility program, we have learned there are big gaps in mental health services around the globe. In some countries, there may only be one mental health professional per 100,000 people. When paired with the reality that 1 in 5 people have a mental health condition, we are asking how technology can and should be involved. In February, we shared our call for project proposals that aim to accelerate mental health research, data insights, and innovations using AI, and today we want to highlight the projects we’re supporting.

Adaptive text-messaging with Mental Health America

Of the 89% of people who screened positive for major depression through Mental Health America’s online survey last year, 79% do not want to pursue psychotherapy or medications, yet 50% want access to digital tools. There are thousands of mental health apps available, but even the top 30 apps see 97% of people stop using them in the first two weeks. Northwestern University and University of Toronto, in partnership with Mental Health America, are developing an adaptive text-messaging service to help people better manage their mental health. Text messages can reach everyone with bite-sized information and prompts, and to increase people’s engagement, machine learning algorithms will be used to discover how to customize the content and timing of messages. By combining clinical psychology, human computer interaction, and machine learning, they hope to design interventions that are easily delivered, effective, and engaging.

Understanding empathy in text-based peer support

While understanding the time and frequency for when a person wants to engage with mental health prompts is important, the tone and language of messages plays a unique role. The University of Washington, in partnership with TalkLife and Supportiv, are building a natural language model to understand empathy in text-based peer support. By adapting measurements of empathy developed in clinical therapy settings, they intend to create an annotation rubric for the millions of messages in their dataset, train models to recognize aspects of empathy, then offer suggestions to make responses more empathetic. This project will help researchers better understand the language of empathy within a mental health scenario and offer an opportunity to explore how to improve human connection among the peer support volunteer community and those seeking help.

Woman working on her laptop

Deliver more impactful services in India

In addition to online peer support forums, crisis lines may be one of the only ways people can get mental health support in some parts of the world. People calling for support can experience long wait times, dropped calls, or be matched with volunteers who don’t share the same sociocultural background or lived experiences. Georgia Tech, working with Befrienders India, is developing a dashboard and models intended to match crisis line callers with volunteers based on demographic and sociocultural characteristics, the needs and issues of the callers, and the lived experiences of the volunteers. The team aims to explore if AI can not only improve the efficiency and staffing of the crisis line system, but more importantly, help deliver more impactful services for people.

Photo collage of Dr. Munmun De Choudhury, Principal Investigator; Dr. Neha Kumar, Co-Principal Investigator; Sachin Pendse, PhD student at Georgia Tech

More to learn, more work to do

Digital tools and peer services are not replacements for mental health professionals, but our grantees will be investigating how to customize technology support for language, cultural background, help us communicate with each other with more empathy, and get nudges at times and in ways that are the most meaningful. We are grateful for the passion and commitment to inclusion each of our grantees has demonstrated and we look forward to sharing more as this research progresses.

If you are interested in applying for the Microsoft AI for Accessibility program, our next grant application deadline is on July 30, 2020 and is an open call for any project idea related to disability, accessibility, and AI. We will then two have additional focused award rounds, one on Smart Cities and Transportation with a deadline of December 15, 2020, and a second on Education with a deadline of March 12, 2021. Check out our application FAQ’s for more details on the program.

Posted on Leave a comment

Microsoft takes legal action against COVID-19-related cybercrime

Today, the U.S. District Court for the Eastern District of Virginia unsealed documents detailing Microsoft’s work to disrupt cybercriminals that were taking advantage of the COVID-19 pandemic in an attempt to defraud customers in 62 countries around the world. Our civil case has resulted in a court order allowing Microsoft to seize control of key domains in the criminals’ infrastructure so that it can no longer be used to execute cyberattacks.

Microsoft’s Digital Crimes Unit (DCU) first observed these criminals in December 2019, when they deployed a sophisticated, new phishing scheme designed to compromise Microsoft customer accounts. The criminals attempted to gain access to customer email, contact lists, sensitive documents and other valuable information. Based on patterns discovered at that time, Microsoft utilized technical means to block the criminals’ activity and disable the malicious application used in the attack. Recently, Microsoft observed renewed attempts by the same criminals, this time using COVID-19-related lures in the phishing emails to target victims.

This malicious activity is yet another form of business email compromise (BEC) attack, which has increased in complexity, sophistication and frequency in recent years. According to the FBI’s 2019 Internet Crime Report, the most-costly complaints received by their Internet Crime Complaint Center (IC3) involved BEC crimes, with losses of over $1.7 billion, representing nearly half of all financial losses due to cybercrime. While most of the public’s attention in recent years has justifiably focused on the malign acts of nation state actors, the increasing economic harm caused by cybercriminals must also be considered and confronted by the public and private sectors. For our part, Microsoft and our Digital Crimes Unit will continue to investigate and disrupt cybercriminals and will seek to work with law enforcement agencies around the world, whenever possible, to stop these crimes.

These cybercriminals designed the phishing emails to look like they originated from an employer or other trusted source and frequently targeted business leaders across a variety of industries, attempting to compromise accounts, steal information and redirect wire transfers. When the group first began carrying out this scheme, the phishing emails contained deceptive messages associated with generic business activities. For example, the malicious link in the email was titled with business terms such as “Q4 Report – Dec19,” as seen below.  

Business-themed phishing email
Business-themed phishing email

With these recent efforts, however, the phishing emails instead contained messages regarding COVID-19 as a means to exploit pandemic-related financial concerns and induce targeted victims to click on malicious links. For example, using terms such as “COVID-19 Bonus,” as seen here.

COVID-19-themed phishing email
COVID-19-themed phishing email

Once victims clicked on the deceptive links, they were ultimately prompted to grant access permissions to a malicious web application (web app). Web apps are familiar-looking as they are widely used in organizations to drive productivity, create efficiencies and increase security in a distributed network. Unknown to the victim, these malicious web apps were controlled by the criminals, who, with fraudulently obtained permission, could access the victim’s Microsoft Office 365 account. This scheme enabled unauthorized access without explicitly requiring the victims to directly give up their login credentials at a fake website or similar interface, as they would in a more traditional phishing campaign.

After clicking through the consent prompt for the malicious web app (pictured below), the victim unwittingly granted criminals permission to access and control the victims’ Office 365 account contents, including email, contacts, notes and material stored in the victims’ OneDrive for Business cloud storage space and corporate SharePoint document management and storage system.

Consent screen of the Malicious Web App
Consent screen of the malicious web app

Microsoft takes many measures to monitor and block malicious web apps based on telemetry indicating atypical behavior and has continued to enhance our protections based on this activity. In cases where criminals suddenly and massively scale their activity and move quickly to adapt their techniques to evade Microsoft’s built-in defensive mechanisms, additional measures such as the legal action filed in this case are necessary. This unique civil case against COVID-19-themed BEC attacks has allowed us to proactively disable key domains that are part of the criminals’ malicious infrastructure, which is a critical step in protecting our customers.

As we’ve observed, cybercriminals have been adapting their lures to take advantage of current events, using COVID-19-related themes to deceive victims. While the lures may have changed, the underlying threats remain, evolve and grow, and it’s more important than ever to remain vigilant against cyberattacks.

To further protect yourself against phishing campaigns, including BEC, we recommend, first, that you enable two-factor authentication on all business and personal email accounts. Second, learn how to spot phishing schemes and protect yourself from them. Third, enable security alerts about links and files from suspicious websites and carefully check your email forwarding rules for any suspicious activity. Businesses can learn how to recognize and remediate these types of attacks and also take these steps to increase the security of their organizations.

Tags: , , , , , ,

Posted on Leave a comment

New study shows customers save time, resources with Microsoft Cloud App Security

The global pandemic has forever changed our workplaces and reshaped our cybersecurity priorities. While in recent months cloud apps have helped people around the globe stay productive and connected. They also pose an increased cybersecurity risk to businesses large and small, especially when you don’t know which cloud apps your employees may be using.  Now, as many countries and companies are entering a new phase toward hybrid work environments, we must apply digital empathy—the idea that cyber systems need to provide both strong security and a great user experience—to address this critical security and compliance priority.

Even before COVID-19, software-as-a-service (SaaS) was growing rapidly because the cloud makes it easy and cost-effective for people to find the tools they need. At the same time, businesses using conventional security suites to try to address vulnerabilities and protect their estates were finding limited success with low visibility into their data, user behavior and sensitive data moving to the cloud.

According to a May 2020 Forrester Consulting Total Economic Impact™ (TEI) Study commissioned by Microsoft, these limitations have led to the rise of shadow IT, difficulty recognizing and remediating security threats, and the need to rapidly adapt to new compliance requirements for the cloud. The study interviewed four existing customers in four industries, including manufacturing, medical devices, education, and health care. It also provided a closer look at the potential financial impact of using Microsoft’s Cloud App Security solution to gain visibility of an organization’s native and third-party applications. That included easier monitoring of security and risks associated with cloud applications and sensitive data, improving detection and remediation of incidents, and improving compliance.

The Forrester study shows a three-year 151% ROI and less than 3-month payback on Cloud App Security investment

To better understand the benefits, costs and risks associated with a Microsoft Cloud App Security investment, Forrester interviewed four organizations with years of experience using Cloud App Security. It also developed a financial analysis of a composite organization to create a financial model framework. The results show organizations can save time and resources with a three-year ROI of 151% and payback of less than 3 months by more easily discovering potential security and compliance risks, automating threat protection and providing more time for people to focus their attention on higher priorities.

Key findings include:

  • 80% reduction in time to monitor, assess and govern cloud application portfolio risks.
  • 75% elimination of threats automatically due to increased visibility and automated threat protection.
  • 40% reduction in the likelihood of a data breach with the potential savings of more than $1.6 million over three years.
  • 90% reduction in the hours required to audit cloud apps.

When customers deploy Cloud App Security in their environment(s), they are frequently surprised at how many apps it uncovers. For almost any use case, employees can often quickly begin using an app without support from IT. This can result in hundreds or even thousands of unmanaged apps—what we refer to as Shadow IT. Although employees mean well, they don’t always understand the security and compliance risks associated with sharing and storing data in cloud apps. One of the organizations interviewed used MCAS to discover 9,000 apps being used by employees—1,600 of which did not meet the company’s security standards and were immediately shut down.

A pull quote.

Another customer in the study noted the compliance benefits were critical as the health care organization moved sensitive information off-premises to the cloud. “We’ve been somewhat slow to move to the cloud because of protected health information and Health Insurance Portability and Accountability Act (HIPAA) regulations,” said the CIO interviewed. In researching cloud application security brokers, this leader realized the ability to get good governance, compliance and audit support was key as the organization moved to the cloud.

From using AI to crunch massive data sets, to analyzing threats in a fraction of a second, given the global scale of the pandemic, integrated security and diversity of data are two key advantages organizations reap as a result of leveraging the cloud. These are also two advantages among the five significant longer-term cybersecurity paradigm shifts, including digital empathy, zero trust, and cyber resilience strategies, that we anticipate as a result of organizations needing to respond quickly to the challenges of the pandemic.

Our Microsoft Cloud App Security Journey

At Microsoft, we’ve been on a journey with our customers gathering feedback and enhancing Microsoft Cloud App Security to meet their needs. The software has matured significantly, with new capabilities released every two weeks such as integrations with our 1st party security and compliance products as well as many 3rd party vendors that continue to represent a large portion of the market.  These product improvements have led to the benefits and value described in this independent study with MCAS customers, and these benefits also ring true for Microsoft’s own Security Operations Center.

In an organization as large as ours with 156,000 worldwide employees, 160+ physical data centers in 60 countries and countless endpoints to monitor, it’s a significant task to track all the cloud services that our employees use. When the company team first deployed Cloud App Security in 2017, it created visibility they didn’t have before across all the non-Microsoft apps used. Once discovered, the team leveraged more than 80 risk factors built into Cloud App Security to evaluate them for compliance with corporate policies. If an app doesn’t meet the company standards, the team can block it from the network. Conversely, the team also uses Cloud App Security to sanction approved apps and if an app is really popular, onboard it onto Azure Active Directory (Azure AD) for single sign-on (SSO), further improving security for employees. Being able to weed out vulnerable apps and apply Azure AD security controls to non-Microsoft apps gives a lot more control over the app portfolio.

The Microsoft SOC receives tens of thousands of security signals a day. With integrated user and entity behavioral analytics (UEBA) and machine learning (ML) algorithms in MCAS our team can weed out false positives, detect behavioral anomalies across all our cloud apps and better respond to threats. This helps us uncover ransomware, compromised users, or rogue applications. This past June we released new documentation to help customers get familiar with our UEBA alerts.

Microsoft’s SOC team echoed the report’s findings on the usefulness of Cloud App Security in investigation and remediation.  Allowing SOC analysts to see the data that is truly necessary helps them to ask the right kinds of questions, pivot with agility in pursuit of data that sparks curiosity and leads to better response patterns.

They also pointed out that Cloud App Security’s ability to assist in further refining the detections that already exist, emailing or texting analysts in custom policy designs and leveraging the powerful API integrations, including SIEM integrations, all led to better response and deeper, more correlated incidents across multiple data sets.  The ability to customize queries remove alerts on “normal behavior” allows teams to zero in on abnormalities and even create a detection rule.

The remediation tools natively available in Cloud App Security which allow immediate revoking of user tokens (therefore prompting an immediate request to re-sign in) drastically simplifies the time to respond and the ability to increase agility when answering an attack.  One of the most challenging things in this environment is how much the speed of attack has increased in recent years.  With Cloud App Security, the team is better postured to identify a compromised user account, enforce revocation of user tokens to mitigate the threat, as well as analyze the touchpoints along the way that provides a deeper understanding of the “BDA” or – before, during, and after – phases of the attack. These findings can ultimately lead to stronger preventative (and detective) controls that address the root cause of the attack.

In a post-pandemic world where our cybersecurity priorities have forever shifted, all companies big and small must think differently about how to keep their data and people safe. By applying digital empathy to their approach, trusting nothing and no one in their Zero Trust journey, and leveraging the power of the cloud and threat intelligence from their tools and people, we all will be stronger and safer no matter what global event, security risks or cyberattacks come next.

Learn more

Read Forrester’s Total Economic Impact ™ of Microsoft Cloud App Security for details on how Cloud App Security can save time and money.

Find out how Cloud App Security can help you manage and secure your cloud environment.

Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.