Voice isolation on macOS

[wolruf]

On macOS, when using apps like Google Meet with audio/video conferences, Safari is able to trigger the macOS mic panel to use "voice isolation" which is super handy to decrease background noise. I know there are special apps, headsets to minimize such noise I prefer to minimize the number of applications.

I'm not sure if this is dependent on the browser, on the app. I didn't find anything in bugzilla. There's a Reddit thread: https://www.reddit.com/r/MacOS/comments/ye9f6q/ventura_finally_added_universal_voice_isolation/

How to test:

- load https://webrtc.github.io/test-pages/src/single-audio/

- check macOS toolbar if mic becomes "Orange" and has the ability to use "voice isolation" as per screenshot.

[Jon]

Thanks for submitting an idea to the Mozilla Connect community! Your idea is now open to votes (aka kudos) and comments.

[pehrsons]

Hi, Firefox media engineer here.

The mic control panel on macOS should pop up automatically when an app accesses the mic. We have also noted it sometimes doesn't, but we believe it has improved in recent versions. We have been working to hook up the built-in voice processing in macOS for microphone access (currently enabled in 128 Nightly). Plausibly some patch along the way there improved the situation.

[wolruf]

Thanks, indeed, I just tested a nightly build 128.0a1 (2024-06-09) and it triggers the mic "voice isolation" (and even wide spectrum) using macOS 14.5.1.

[wolruf]

Hi @pehrsons with voice isolation coming to release 128.0 do you believe that can also address the power usage when using Google Meet (even without video) ? https://bugzilla.mozilla.org/show_bug.cgi?id=1670193#c12

[pehrsons]

I wouldn't expect a significant difference by enabling vpio. Mainly the audio processing moves from Firefox content processes to the OS. Assuming by without video you mean both send and receive. We are aware of performance issues with video.

[wolruf]

Thanks for taking the time to respond, that's appreciated. I'm frustrated by Google Meet on Firefox/macOS/x86 with the fan kicking in (while a desktop app such as Zoom, MS Teams don't exhibit such issue). I'm guessing switching to M1/2/3 will fix this. I'm still hoping audio-only GMeet won't be fan intensive anymore in 128.0.

[pehrsons]

I have an old MacbookPro10,1 (2012 model) which on Firefox 127 in a 1:1 Google Meet without video (both ends disabled camera) is at ~90% idle CPU overall. Chrome is very similar in the same setup.

If you give more details about the (minimized) conditions under which you see perf issues I could give a more directed answer. But like I said, if video is involved, we are aware of issues.

[wolruf]

so with release 128.0 (from FTP), Voice Isolation kicks in with https://webrtc.github.io/test-pages/src/single-audio/

Voice isolation didn't work seems to work with Google Meet: the "camera" voice isolation icon becomes green. Though there's also the standard camera next to it.

[caturbgs11]

Yeah same here, any updates for this? Really frustrating, the Google meet can't detect voice isolation.

For now, I should open https://webrtc.github.io/test-pages/src/single-audio/ for making the Firefox showing the settings, then open the Google meet.

[pehrsons]

@caturbgs11which macOS are you on?

Note Google Meet does two gUM requests for audio, first one with audio processing then one without. We let multiple uses of a device on a site share the same stream, so this leads us to first create a VoiceProcessingIO audio unit, then stop that and create a regular HAL audio unit instead. I suspect this triggers a macOS bug making the microphone control panel not show up.

https://bugzilla.mozilla.org/show_bug.cgi?id=1914046 might help avoid the audio unit churn, but might put off Google Meet's echo detection feature.

I see the control panel not showing up on macOS 15. Interestingly closing the Meet tab brings the microphone control panel back for a bit. That's when we close the HAL audio unit.

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1921051 for the control panel issue.

[caturbgs11]

@pehrsonsmacOS Sequoia 15.0

this already happen when I'm used the Sonoma version on 14.XX for somehow reason.

[caturbgs11]

@pehrsons I’m on Sequioa 15.0, but the issue was persist since the Sonoma 14.XX version