Tag: Recent News

Over 270 million active users rely on Microsoft Teams to connect, share, learn, and collaborate, which affirms our commitment to continually improving call quality. In a previous Teams Blog post, we introduced machine-learning-based noise suppression, which automatically removes unwanted background sounds from calls and meetings. Since then, we have been working to expand the reach of this feature to more platforms and scenarios. After its initial release on Windows, background noise suppression is now available on Mac and iOS as well.

After the release of Teams background noise suppression for Windows users as an optional feature, we went through an iterative development and evaluation cycle to optimize our model and advance broader research in this field. With this goal in mind, we launched various competitions including our latest at the International Conference on Acoustics, Speech and Signal Processing 2022, and open-sourced both our dataset and the perceptual quality crowdsourcing framework.

These events helped improve the quality and complexity of our model. Our iterative experimentation process showed that improved call quality led to an increase in call duration for one-on-one calls. We also saw a 32% reduction in complaints of background noise reported in our end-of-call survey. Considering the significance of these improvements, we enabled machine-learning-based noise suppression as default for Teams customers using Windows. The same experiments were also conducted for Mac and iOS which showed similar improvements, leading us to release this feature as default on these platforms as well. This change of enabling noise suppression by default for most calls makes this feature the most widely used AI feature in Microsoft Teams, but more importantly improving experiences for millions of users who confidently take Teams calls and meetings from anywhere.

In addition to this extended platform coverage, we have also been working to make machine-learning-based noise suppression available on an expanded range of device types, including support for ARM-based devices and Microsoft Teams Rooms.

While the user benefits from machine-learning-based noise suppression appear obvious and ubiquitous, we were mindful of scenarios where it’s important to not suppress non-speech content, such as during music lessons. While it may be desirable that your kid’s music practice doesn’t disturb your work meeting, we also want to avoid negatively impacting the teacher’s ability to conduct the music lesson over Teams.

To differentiate those use-cases, we have implemented music detection into the noise suppression capability. This feature alerts users when AI identifies music so they can choose to disable noise suppression and enable transmitting music via our “High-fidelity Music Mode”. Music detection has been released for our desktop Windows client and has allowed us to turn on machine-learning-based noise suppression by default for our education customers as well. A future release of this feature is planned for education users on Mac and iOS devices as well as across all Teams Android and web clients.

While we’re excited about the advancements in Teams call quality we’ve achieved through machine-learning and AI, we’re just beginning to realize the opportunities ahead. Keep watching this blog to learn how innovation, testing, and optimization enables us to continually improve meeting and call quality in Teams.

In 2021, nearly three-quarters of employees experienced a 70% increase in meetings after organizations shifted to remote work due to the onset of the COVID-19 pandemic.¹ When 43% of remote workers state they do not feel included in meetings², the natural question is, how can we make meetings more effective and inclusive?

We were curious too, so we decided to survey a large number of Microsoft customers who wanted to improve meeting experiences in their organizations. As it happens, this was the first large-scale study that we’re aware of conducted by a technology company to determine what makes meetings effective and inclusive.

We used statistical, mathematical, and machine learning techniques to analyze meeting dynamics through survey data and anonymous telemetry. This enabled us to build a model to understand and predict which meetings would be effective, and why. Criteria such as using an agenda, active participation, having video turned on, keeping meeting sizes small, and sharing pre-meeting material all ranked high for driving inclusiveness and effectiveness. Additionally, data analysis showed strong connections between meeting participation and attendees’ subjective perceptions of inclusiveness, sense of comfort, and meeting effectiveness.

Since meetings are a permanent fixture of modern work, we encourage you to dive into the findings below and find new ways to promote behaviors that lead to more effective and inclusive meetings.

Identifying Our Audience and Survey Methods

We used two survey methodologies in tandem to measure meeting effectiveness and inclusiveness across Microsoft employees and a select group of Microsoft Teams customers:

1. Post-meeting pop-up – A two-question survey within Teams that popped up at the end of a meeting to collect sentiment in near real-time.
2. Email-based survey – Approximately 20 questions designed to understand the nuances of the most recent meeting the respondent attended.

The in-Teams survey methodology was designed with goals to:

• Reach a random sample of meetings throughout the organization
• Minimize intrusion in the workday
• Lower the burden to respond
• Keep responses anonymous to ensure a psychologically safe place to provide feedback on meeting effectiveness and inclusivity

Survey types and population groups have different survey response rates, adding another layer of sampling to our data collection. While this additional layer induces a “non-response” bias to the data, our analysis showed a negligible amount of bias that did not degrade the data utility.

In-product survey

Modeling the Survey Data

To understand the drivers of meeting effectiveness and inclusiveness, we began examining the strength of dependencies between user ratings on these criteria with expected factors including meeting duration, the number of participants, usage of audio/video/screen sharing, and presence of an agenda among others.

We used a variety of statistical tests and machine learning (ML) algorithms to learn the set of factors and combinations that correspond to fundamental differences in ratings of effectiveness and inclusiveness. While an abundance of articles and small-sample studies provide a long list of potential drivers for meeting effectiveness and inclusivity, many of them are shown to not be significant after large-sample analysis.

A proven method of breaking down the collective pattern of dependencies among the set of potential factors and ratings is through the use of graphical models. We developed a process for selecting top contributors by fitting a web of dependencies and estimating the strength of each connection. A sample of such a model is shown below. The following graphical model from our peer-reviewed publication³ illustrates the multivariate interplay between factors.

Multivariate model of effectiveness (red and green show negative and positive effects, respectively)

The arrows in the above graph show the relationships between the variables and in relative terms. Using green arrows to show positive correlations and red arrows show negative correlations, this model helped us select the truly impactful factors on meeting effectiveness and inclusiveness from a sea of claims and anecdotal hypotheses. For example, the green arrow with “2.5” between “Participation” and “Inclusive” can be interpreted as meetings with high participation are 2.5x more likely to be rated as inclusive.

Participation represents “speaking in the meeting more than once.” It shows a significant positive impact on the chance of the meeting being rated inclusive (4 or 5-star). This pattern consistently shows up regardless of the survey type or audience group. The strength and consistency of this impact encouraged us to develop targeted ML models to further understand details about this correlation. Specifically, to include the interaction effects and explore the ramifications under different segments. Our findings show that:

• Meetings with higher levels of participation were rated more inclusive. Participants who spoke often during the meeting gave a 98% inclusive rating, participants who spoke a few times gave an 89% inclusive rating, participants who spoke only once gave a 67% inclusive rating, and participants who only listened gave a 36% inclusive rating.
• Participants who felt their presence was necessary for the meeting felt more included – giving the meeting a 92% inclusive rating over those whose presence was not necessary, who gave a 53% inclusive rating.
• Meetings with smaller meeting sizes were rated as more inclusive. This is a decreasing trend where 2-person meetings were rated by all participants as inclusive, while meetings with more than 10 people were rated only 60% inclusive.

Our analysis also showed that the impact of participation on inclusiveness was also influenced by other factors including meeting duration and length of the meeting. For example, the most significant gain is observed for thirty-minute or shorter meetings. In this segment, participation can increase meeting inclusiveness by ~6% and effectiveness by ~4%.

Video usage was another area where the large sample of our survey allowed us to learn from nonlinear patterns. While additional research is warranted to understand the breadth of meeting scenarios that benefit most from having video on, and which scenarios benefit from having video off, our survey data did yield interesting findings. We found the meeting scenario with a higher chance of benefiting from video usage is a “small meeting” with fewer than eight participants where some people are in a meeting room, and the rest are joining remotely. The benefit mostly comes from local participants (those in the meeting room) being able to see remote participants via video. Larger meetings may be more prone to “Video Conference” due to prolonged eye gaze, an effect that’s magnified when users are required to stare at numerous one-inch boxes of faces on their screens.⁴

More Resources and Insights

In addition to encouraging behaviors correlated to increased meeting effectiveness and inclusiveness, Microsoft Productivity Score and Viva Insights are valuable resources to help monitor organizational productivity, effectiveness, and engagement.

Productivity Score can be accessed through the Microsoft 365 Admin Center

Better Meetings… Through Science

Bookshelves are full of pages dedicated to the topic of meeting effectiveness, though many of these insights are based on small samples and anecdotes. While there’s still much about meetings yet to explore, such as the influence of meeting purpose on effectiveness and inclusion (brainstorming and problem solving versus status meetings), our research offers valuable constructs for how to make such connections.

What is clear in a hybrid-first world, is that the volume of scheduled and ad-hoc meetings is likely to increase. Organizations need a trusted methodology to identify and measure the factors that lead to more valuable meetings and enable us to form and strengthen connections between colleagues and friends.

1. Atlassian. 2022. You Waste A Lot of Time at Work Infographic | Atlassian. [online] Available at: https://www.atlassian.com/time-wasting-at-work-infographic
2. Work Trend Index: Microsoft’s latest research on the ways we work
3. Cutler, R., Hosseinkashi, Y., Pool, J., Filipi, S., Aichner, R., Tu, Y., & Gehrke, J. (2021, February 19). Meeting effectiveness and inclusiveness in remote collaboration. arXiv.org. Retrieved May 23, 2022, from https://arxiv.org/abs/2102.09803
4. Karl, K., Peluchette, J. and Aghakhani, N., 2021. Virtual Work Meetings During the COVID-19 Pandemic: The Good, Bad, and Ugly. Small Group Research, 53(3), pp.343-365