08-01-2020, 12:30 PM
Real-time noise suppression for video conferencing
<div style="margin: 5px 5% 10px 5%;"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/08/real-time-noise-suppression-for-video-conferencing.png" width="1024" height="379" title="" alt="" /></div><div><p>With people doing video conferencing all day, good audio has recently become much more important. The best option is obviously a proper audio studio. Unfortunately, this is not something you will always have and you might need to make do with a much simpler setup.</p>
<p>In such situations, a noise reduction filter that keeps your voice but filters out ambient noises (street noise, keyboard, …) can be very helpful. In this article, we will take a look at how to integrate such a filter into PulseAudio so that it can easily be used in all applications with no additional requirements on their part.</p>
<figure class="wp-block-audio"><audio controls src="https://fedoramagazine.org/wp-content/uploads/2020/07/test-amp.mp3"></audio><figcaption>Example of switching on noise reduction</figcaption></figure>
<h2>The Idea</h2>
<p>We set up <a href="https://freedesktop.org/wiki/Software/PulseAudio/">PulseAudio</a> for live noise-reduction using <a href="https://github.com/werman/noise-suppression-for-voice">an LADSPA filter</a>.</p>
<p>This creates a new PulseAudio source which can be used as a virtual microphone. Other applications will not even realize that they are not dealing with physical devices and you can select it as if you had an additional microphone connected.</p>
<h2>Terminology</h2>
<p>Before we start, it is good to know the following two PulseAudio terms to better understand what we are doing:</p>
<ul>
<li><em>source</em> – represents a source from which audio can be obtained. Like a microphone</li>
<li><em>sink</em> – represents a consumer of audio like a speaker</li>
</ul>
<p>Each PulseAudio sink also has a source called monitor which can be used to get the audio put into that sink. For example, you could have audio put out by your headphones while using the monitor of your headphone device to record the output.</p>
<h2>Installation</h2>
<p>While PulseAudio is usually pre-installed, we need to get the LADSPA filter for noise reduction. You can <a href="https://github.com/werman/noise-suppression-for-voice">build and install the filter manually</a>, but it is much easier to install the filter via Fedora Copr:</p>
<pre class="wp-block-preformatted">sudo dnf copr enable -y lkiesow/noise-suppression-for-voice
sudo dnf install -y ladspa-realtime-noise-suppression-plugin</pre>
<p>Note that the Copr projects are not maintained and quality-controlled by Fedora directly.</p>
<h2>Enable Noise Reduction Filter</h2>
<p>First, you need to identify the name of the device you want to apply the noise reduction to. In this example, we’ll use the RODE NT-USB microphone as input.</p>
<pre class="wp-block-preformatted">$ pactl list sources short
0 alsa_input.usb-RODE_Microphones_RODE_NT-USB-00.iec958-stereo …
1 alsa_output.usb-0c76_USB_Headphone_Set-00.analog-stereo.monitor …</pre>
<p>Next, we create a new PulseAudio sink, the filter and a loopback between microphone and filter. That way, the output from the microphone is used as input for the noise reduction filter. The output from this filter will then be available via the null sink monitor.</p>
<p>To visualize this, here is the path the audio will travel from the microphone to, for example, a browser:</p>
<pre class="wp-block-preformatted">mic → loopback → ladspa filter → null sink [monitor] → browser</pre>
<p>While this sounds complicated, it is set up with just a few simple commands:</p>
<pre class="wp-block-preformatted">pacmd load-module module-null-sink \ sink_name=mic_denoised_out
pacmd load-module module-ladspa-sink \ sink_name=mic_raw_in \ sink_master=mic_denoised_out \ label=noise_suppressor_stereo \ plugin=librnnoise_ladspa \ control=50
pacmd load-module module-loopback \ source=alsa_input.usb-RODE_Microphones_RODE_NT-USB-00.iec958-stereo \ sink=mic_raw_in \ channels=2</pre>
<p>That’s it. You should now be able to select the new device.</p>
<figure class="wp-block-image size-large"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/08/real-time-noise-suppression-for-video-conferencing.png" alt="" class="wp-image-31413" /><figcaption>New recording devices in pavucontrol</figcaption></figure>
<h2>Chromium</h2>
<p>Unfortunately, browsers based on Chromium will hide monitor devices by default. This means, that we cannot select the newly created noise-reduction device in the browser. One workaround is to select another device first, then use pavucontrol to assign the noise-reduction device afterward.</p>
<p>But if you do this on a regular basis, you can work around the issue by using the <em>remap-source</em> module to convert the null sink monitor to a regular PulseAudio source. The module is actually meant for remapping audio channels – e.g. swapping left and right channel on stereo audio – but we can just ignore these additional capabilities and create a new source similar to the monitor:</p>
<pre class="wp-block-preformatted">pacmd load-module module-remap-source \ source_name=denoised \ master=mic_denoised_out.monitor \ channels=2</pre>
<p>The remapped device delivers audio identical to the original one so that assigning this with PulseAudio will yield no difference. But this device does now show up in Chromium:</p>
<figure class="wp-block-image size-large"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/08/real-time-noise-suppression-for-video-conferencing-1.png" alt="" class="wp-image-31412" /><figcaption>Remapped monitor device in Chrome</figcaption></figure>
<h2>Improvements</h2>
<p>While the guide above should help you with all the basics and will get you a working setup, there are a few things you can improve.</p>
<p>But while the commands above should generally work, you might need to experiment with the following suggestions.</p>
<h3>Latency</h3>
<p>By default, the loopback module will introduce a slight audio latency. You can hear this by running an echo test:</p>
<pre class="wp-block-preformatted">gst-launch-1.0 pulsesrc ! pulsesink</pre>
<p>You might be able to reduce this latency by using the <em>latency_msec</em> option when loading the <em>loopback</em> module:</p>
<pre class="wp-block-preformatted">pacmd load-module module-loopback \ latency_msec=1 \ source=alsa_input.usb-RODE_Microphones_RODE_NT- USB-00.iec958-stereo \ sink=mic_raw_in \ channels=2</pre>
<h3>Voice Threshold</h3>
<p>The noise reduction library provides controls for a voice threshold. The filter will return silence if the probability for sound being voice is lower than this threshold. In other words, the higher you set this value, the more aggressive the filter becomes.</p>
<p>You can pass different thresholds to the filter by supplying them as control argument when the <em>ladspa-sink</em> module is being loaded.</p>
<pre class="wp-block-preformatted">pacmd load-module module-ladspa-sink \ sink_name=mic_raw_in \ sink_master=mic_denoised_out \ label=noise_suppressor_stereo \ plugin=librnnoise_ladspa \ control=95</pre>
<h3>Mono vs Stereo</h3>
<p>The example above will work with stereo audio. When working with a simple microphone, you may want to use a mono signal instead.</p>
<p>For switching to mono, use the following values instead when loading the different modules:</p>
<ul>
<li><em>label=noise_suppressor_mono</em> – when loading the <em>ladspa-sink</em> module</li>
<li><em>channels=1</em> – when loading the <em>loopback</em> and remap-source modules</li>
</ul>
<h3>Persistence</h3>
<p>Using the <em>pacmd</em> command for the setup, settings are not persistent and will disappear if PulseAudio is restarted. You can add these commands to your PulseAudio configuration file if you want them to be persistent. For that, edit <em>~/.config/pulse/default.pa</em> and add your commands like this:</p>
<pre class="wp-block-preformatted">.include /etc/pulse/default.pa load-module module-null-sink sink_name=mic_denoised_out
load-module module-ladspa-sink …
…</pre>
<h2>Limitations</h2>
<p>If you listen to the example above, you will notice that the filter reliably reduces background noise. But unfortunately, depending on the situation, it can also cause a loss in voice quality.</p>
<p>The following example shows the results with some street noise. Activating the filter reliably removes the noise, but in this example, the voice quality noticeably drops as well:</p>
<figure class="wp-block-audio"><audio controls src="https://fedoramagazine.org/wp-content/uploads/2020/07/street-amp.mp3"></audio><figcaption>Noise reduction of constant street noise</figcaption></figure>
<p>As a conclusion, we can say that this can help if you find yourself in less than ideal audio scenarios. It is also very effective if you are not the main speaker in a video conference and you do not want to constantly mute yourself.</p>
<p>Still, good audio equipment and a quiet environment will always be better.</p>
<p>Have fun.</p>
</div>
https://www.sickgaming.net/blog/2020/07/...ferencing/
<div style="margin: 5px 5% 10px 5%;"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/08/real-time-noise-suppression-for-video-conferencing.png" width="1024" height="379" title="" alt="" /></div><div><p>With people doing video conferencing all day, good audio has recently become much more important. The best option is obviously a proper audio studio. Unfortunately, this is not something you will always have and you might need to make do with a much simpler setup.</p>
<p>In such situations, a noise reduction filter that keeps your voice but filters out ambient noises (street noise, keyboard, …) can be very helpful. In this article, we will take a look at how to integrate such a filter into PulseAudio so that it can easily be used in all applications with no additional requirements on their part.</p>
<figure class="wp-block-audio"><audio controls src="https://fedoramagazine.org/wp-content/uploads/2020/07/test-amp.mp3"></audio><figcaption>Example of switching on noise reduction</figcaption></figure>
<h2>The Idea</h2>
<p>We set up <a href="https://freedesktop.org/wiki/Software/PulseAudio/">PulseAudio</a> for live noise-reduction using <a href="https://github.com/werman/noise-suppression-for-voice">an LADSPA filter</a>.</p>
<p>This creates a new PulseAudio source which can be used as a virtual microphone. Other applications will not even realize that they are not dealing with physical devices and you can select it as if you had an additional microphone connected.</p>
<h2>Terminology</h2>
<p>Before we start, it is good to know the following two PulseAudio terms to better understand what we are doing:</p>
<ul>
<li><em>source</em> – represents a source from which audio can be obtained. Like a microphone</li>
<li><em>sink</em> – represents a consumer of audio like a speaker</li>
</ul>
<p>Each PulseAudio sink also has a source called monitor which can be used to get the audio put into that sink. For example, you could have audio put out by your headphones while using the monitor of your headphone device to record the output.</p>
<h2>Installation</h2>
<p>While PulseAudio is usually pre-installed, we need to get the LADSPA filter for noise reduction. You can <a href="https://github.com/werman/noise-suppression-for-voice">build and install the filter manually</a>, but it is much easier to install the filter via Fedora Copr:</p>
<pre class="wp-block-preformatted">sudo dnf copr enable -y lkiesow/noise-suppression-for-voice
sudo dnf install -y ladspa-realtime-noise-suppression-plugin</pre>
<p>Note that the Copr projects are not maintained and quality-controlled by Fedora directly.</p>
<h2>Enable Noise Reduction Filter</h2>
<p>First, you need to identify the name of the device you want to apply the noise reduction to. In this example, we’ll use the RODE NT-USB microphone as input.</p>
<pre class="wp-block-preformatted">$ pactl list sources short
0 alsa_input.usb-RODE_Microphones_RODE_NT-USB-00.iec958-stereo …
1 alsa_output.usb-0c76_USB_Headphone_Set-00.analog-stereo.monitor …</pre>
<p>Next, we create a new PulseAudio sink, the filter and a loopback between microphone and filter. That way, the output from the microphone is used as input for the noise reduction filter. The output from this filter will then be available via the null sink monitor.</p>
<p>To visualize this, here is the path the audio will travel from the microphone to, for example, a browser:</p>
<pre class="wp-block-preformatted">mic → loopback → ladspa filter → null sink [monitor] → browser</pre>
<p>While this sounds complicated, it is set up with just a few simple commands:</p>
<pre class="wp-block-preformatted">pacmd load-module module-null-sink \ sink_name=mic_denoised_out
pacmd load-module module-ladspa-sink \ sink_name=mic_raw_in \ sink_master=mic_denoised_out \ label=noise_suppressor_stereo \ plugin=librnnoise_ladspa \ control=50
pacmd load-module module-loopback \ source=alsa_input.usb-RODE_Microphones_RODE_NT-USB-00.iec958-stereo \ sink=mic_raw_in \ channels=2</pre>
<p>That’s it. You should now be able to select the new device.</p>
<figure class="wp-block-image size-large"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/08/real-time-noise-suppression-for-video-conferencing.png" alt="" class="wp-image-31413" /><figcaption>New recording devices in pavucontrol</figcaption></figure>
<h2>Chromium</h2>
<p>Unfortunately, browsers based on Chromium will hide monitor devices by default. This means, that we cannot select the newly created noise-reduction device in the browser. One workaround is to select another device first, then use pavucontrol to assign the noise-reduction device afterward.</p>
<p>But if you do this on a regular basis, you can work around the issue by using the <em>remap-source</em> module to convert the null sink monitor to a regular PulseAudio source. The module is actually meant for remapping audio channels – e.g. swapping left and right channel on stereo audio – but we can just ignore these additional capabilities and create a new source similar to the monitor:</p>
<pre class="wp-block-preformatted">pacmd load-module module-remap-source \ source_name=denoised \ master=mic_denoised_out.monitor \ channels=2</pre>
<p>The remapped device delivers audio identical to the original one so that assigning this with PulseAudio will yield no difference. But this device does now show up in Chromium:</p>
<figure class="wp-block-image size-large"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/08/real-time-noise-suppression-for-video-conferencing-1.png" alt="" class="wp-image-31412" /><figcaption>Remapped monitor device in Chrome</figcaption></figure>
<h2>Improvements</h2>
<p>While the guide above should help you with all the basics and will get you a working setup, there are a few things you can improve.</p>
<p>But while the commands above should generally work, you might need to experiment with the following suggestions.</p>
<h3>Latency</h3>
<p>By default, the loopback module will introduce a slight audio latency. You can hear this by running an echo test:</p>
<pre class="wp-block-preformatted">gst-launch-1.0 pulsesrc ! pulsesink</pre>
<p>You might be able to reduce this latency by using the <em>latency_msec</em> option when loading the <em>loopback</em> module:</p>
<pre class="wp-block-preformatted">pacmd load-module module-loopback \ latency_msec=1 \ source=alsa_input.usb-RODE_Microphones_RODE_NT- USB-00.iec958-stereo \ sink=mic_raw_in \ channels=2</pre>
<h3>Voice Threshold</h3>
<p>The noise reduction library provides controls for a voice threshold. The filter will return silence if the probability for sound being voice is lower than this threshold. In other words, the higher you set this value, the more aggressive the filter becomes.</p>
<p>You can pass different thresholds to the filter by supplying them as control argument when the <em>ladspa-sink</em> module is being loaded.</p>
<pre class="wp-block-preformatted">pacmd load-module module-ladspa-sink \ sink_name=mic_raw_in \ sink_master=mic_denoised_out \ label=noise_suppressor_stereo \ plugin=librnnoise_ladspa \ control=95</pre>
<h3>Mono vs Stereo</h3>
<p>The example above will work with stereo audio. When working with a simple microphone, you may want to use a mono signal instead.</p>
<p>For switching to mono, use the following values instead when loading the different modules:</p>
<ul>
<li><em>label=noise_suppressor_mono</em> – when loading the <em>ladspa-sink</em> module</li>
<li><em>channels=1</em> – when loading the <em>loopback</em> and remap-source modules</li>
</ul>
<h3>Persistence</h3>
<p>Using the <em>pacmd</em> command for the setup, settings are not persistent and will disappear if PulseAudio is restarted. You can add these commands to your PulseAudio configuration file if you want them to be persistent. For that, edit <em>~/.config/pulse/default.pa</em> and add your commands like this:</p>
<pre class="wp-block-preformatted">.include /etc/pulse/default.pa load-module module-null-sink sink_name=mic_denoised_out
load-module module-ladspa-sink …
…</pre>
<h2>Limitations</h2>
<p>If you listen to the example above, you will notice that the filter reliably reduces background noise. But unfortunately, depending on the situation, it can also cause a loss in voice quality.</p>
<p>The following example shows the results with some street noise. Activating the filter reliably removes the noise, but in this example, the voice quality noticeably drops as well:</p>
<figure class="wp-block-audio"><audio controls src="https://fedoramagazine.org/wp-content/uploads/2020/07/street-amp.mp3"></audio><figcaption>Noise reduction of constant street noise</figcaption></figure>
<p>As a conclusion, we can say that this can help if you find yourself in less than ideal audio scenarios. It is also very effective if you are not the main speaker in a video conference and you do not want to constantly mute yourself.</p>
<p>Still, good audio equipment and a quiet environment will always be better.</p>
<p>Have fun.</p>
</div>
https://www.sickgaming.net/blog/2020/07/...ferencing/