-
Notifications
You must be signed in to change notification settings - Fork 16.2k
docs: improve desktop-capturer loopback docs #47493
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
docs: improve desktop-capturer loopback docs #47493
Conversation
💖 Thanks for opening this pull request! 💖 Semantic PR titlesWe use semantic commit messages to streamline the release process. Before your pull request can be merged, you should update your pull request title to start with a semantic prefix. Examples of commit messages with semantic prefixes:
Commit signingThis repo enforces commit signatures for all incoming PRs. PR tipsThings that will help get your PR across the finish line:
We get a lot of pull requests on this repo, so please be patient and we will get back to you as soon as we can. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much for the docs change! Appreciate that a lot!
I'll take a closer look when I'm at a computer.
Could you please wrap lines between 80 and 100 characters as per our docs style guide?
@@ -62,7 +62,7 @@ stopButton.addEventListener('click', () => { | |||
<body> | |||
<button id="startButton" class="button">Start</button> | |||
<button id="stopButton" class="button">Stop</button> | |||
<video width="320" height="240" autoplay></video> | |||
<video width="320" height="240" autoplay controls></video> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DX improvement: controls
being visible is important for the dev to be able to easily determine if there's an audio track available or not.
Thanks so much @nikwen! I took your feedback and ran with it, rewrote a bunch of stuff and added even more clarity. Let me know what you think! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! This is great.
Also requesting a review from @codebytere as she probably has the best understanding of that code.
Thanks so much, @nikwen. I've applied your feedback. Last thing I'm debating is adding a "macOS Caveats" header to lead the "System Audio Capture" sections at the bottom. I kinda feel like the macOS notes at the bottom come out of nowhere. Thoughts? |
I think that's a great idea. Thanks! |
Done—Looks much better now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much!
I found some more comments on other ways to make loopback audio capture work on macOS 14.2+:
I wonder if that would work around the macOS 15 bugs. (Or does it introduce new ones?)
Also added some minor things inline. This PR is in pretty good shape now in my opinion.
|
||
### System Audio Capture (before macOS 12.3) | ||
|
||
On macOS 12.3 or lower, system audio capture is not supported due to a fundamental limitation whereby |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On macOS 12.3 or lower, system audio capture is not supported due to a fundamental limitation whereby | |
Before macOS 12.3, system audio capture is not supported due to a fundamental limitation whereby |
Sounds like it works on 12.3.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can indeed try opting in to it by enabling the MacSckSystemAudioLoopbackOverride
flag, but one downstream codepath regards ScreenCaptureKit as too buggy to use until 13.2, so I'm not sure it'll work in practice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I was playing it safe here. We can and probably should get more specific.
### System Audio Capture (before macOS 12.3) | ||
|
||
On macOS 12.3 or lower, system audio capture is not supported due to a fundamental limitation whereby | ||
apps that want to access the system's audio require a [signed kernel extension](https://developer.apple.com/library/archive/documentation/Security/ConceptualSystem_Integrity_Protection_Guide/KernelExtensions/KernelExtensions.html). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
apps that want to access the system's audio require a [signed kernel extension](https://developer.apple.com/library/archive/documentation/Security/ConceptualSystem_Integrity_Protection_Guide/KernelExtensions/KernelExtensions.html). | |
apps that want to access the system's audio require a | |
[signed kernel extension](https://developer.apple.com/library/archive/documentation/Security/ConceptualSystem_Integrity_Protection_Guide/KernelExtensions/KernelExtensions.html). |
Wrapping between 80 and 100 chars.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I'm a total markdown noob but how do I wrap a long URL link without breaking it? Whenever I try to split that big string, it breaks the link in my Markdown Preview.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, links can't be wrapped. It's okay if links are longer if they are the only thing in a line.
> ```js | ||
> // main.js | ||
> app.commandLine.appendSwitch('enable-features', 'MacSckSystemAudioLoopbackOverride'); | ||
> ``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment sounds like it is also needed on macOS 12: https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/flag_descriptions.cc;l=6247-6253;drc=19d14ae4a957ef0f158e3b37cc7f3c76e839ce5d
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can indeed try opting in to it by enabling the MacSckSystemAudioLoopbackOverride
flag, but one downstream codepath regards ScreenCaptureKit as too buggy to use until 13.2, so I'm not sure it'll work in practice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I honestly just need to test this on macOS 12.x to get to the bottom of it. I was erring on the side of caution here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do think the desktopCapturer docs need some clarifying for macOS. They have led me (and, judging by weekly threads on Reddit, many others) astray. These changes help a lot, thanks!
|
||
It is possible to circumvent this limitation by capturing system audio with another macOS app like Soundflower and passing it through a virtual audio input device. This virtual device can then be queried with `navigator.mediaDevices.getUserMedia`. | ||
On macOS 12.3+, `navigator.mediaDevices.getDisplayMedia` can be used to capture system audio |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good to be explicit about which macOS versions, if any, navigator.mediaDevices.getUserMedia()
can be used on.
I seem to recall from recent testing on macOS 14.6.1 that it can't be used at all, but worth double-checking before reporting it in the docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm unaware of this. 😅
> [!NOTE] | ||
> Due to ongoing changes in Chromium, capturing system audio is experimental on macOS 15+ and requires the | ||
> following [command line switch](./command-line-switches.md) to be appended. Please note that you may experience bugs. | ||
> | ||
> ```js | ||
> // main.js | ||
> app.commandLine.appendSwitch('enable-features', 'MacSckSystemAudioLoopbackOverride'); | ||
> ``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is inaccurate.
Looking at how IsSystemLoopbackCaptureSupported() is implemented in Chromium, loopback capture depends on both the macOS version and the state of the MacSckSystemAudioLoopbackOverride
and MacCatapSystemAudioLoopbackCapture
flags (which are both disabled by default). The audio system used may be based on ScreenCaptureKit or Core Audio taps.
- 🔴
version < 12.3
:- Loopback capture is not possible.
- 🔴
12.3 ≤ version < 13.0
:- Loopback capture is probably not possible. Although ScreenCaptureKit is technically available from macOS 12.3, the Chromium codebase notes that it is too buggy to activate until 13.2. You can try enabling the
MacSckSystemAudioLoopbackOverride
flag to opt in to it, but I think it won't work in practice.
- Loopback capture is probably not possible. Although ScreenCaptureKit is technically available from macOS 12.3, the Chromium codebase notes that it is too buggy to activate until 13.2. You can try enabling the
- 🔴
13.0 ≤ version < 13.2
:- Loopback capture is probably not possible. Although the top-level IsMacSckSystemAudioLoopbackCaptureEnabled() check defaults to
true
for this version range (meaning that the ScreenCaptureKit implementation is enabled even without enabling theMacSckSystemAudioLoopbackOverride
flag), in practice, certain downstream codepaths treat it as too buggy to activate.
- Loopback capture is probably not possible. Although the top-level IsMacSckSystemAudioLoopbackCaptureEnabled() check defaults to
- 🟢
13.2 ≤ version < 14.2
:- Loopback capture is possible, via ScreenCaptureKit only, and is enabled by default; no flag needed.
- 🟢
14.2 ≤ version < 15.0
:- Loopback capture is possible, via either ScreenCaptureKit or Core Audio taps. By default, it is enabled, and done via ScreenCaptureKit. However, you can opt-in to Core Audio taps instead by enabling the
MacCatapSystemAudioLoopbackCapture
flag (which takes precedence).
- Loopback capture is possible, via either ScreenCaptureKit or Core Audio taps. By default, it is enabled, and done via ScreenCaptureKit. However, you can opt-in to Core Audio taps instead by enabling the
- 🟡
15.0 ≤ version
:- Loopback capture is possible, via either ScreenCaptureKit or Core Audio taps. By default, it is disabled altogether.
- To enable it via Core Audio taps (the preferred approach from macOS 15.0 and above), enable the
MacCatapSystemAudioLoopbackCapture
flag. As this flag takes precedence, it doesn't matter whetherMacSckSystemAudioLoopbackOverride
is enabled or disabled. - To enable it via ScreenCaptureKit (not recommended, due to permissions prompt issues from macOS 15.0), you must enable the
MacSckSystemAudioLoopbackOverride
flag and disable/omit theMacCatapSystemAudioLoopbackCapture
flag.
- To enable it via Core Audio taps (the preferred approach from macOS 15.0 and above), enable the
- Loopback capture is possible, via either ScreenCaptureKit or Core Audio taps. By default, it is disabled altogether.
So, in summary, to support the maximum version range of 13.2 ≤ version
, you should enable the MacCatapSystemAudioLoopbackCapture
flag to extend support for version ≥ 15.0
.
If both the MacSckSystemAudioLoopbackOverride
and MacCatapSystemAudioLoopbackCapture
flags are enabled, the latter will take precedence.
I should clarify that this is just based on my analysis of the latest commit on the main
branch of Chromium. It might be that things were different in previous versions of Chromium, and it might be that Electron builds Chromium with different flags (however, I find no search results for either).
Does this match your experience?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a great write-up, thank you. I see that there's definitely some inaccuracies that'll need to be fixed here.
Per my experience, talking strictly system audio capture (there very well could be horrible bugs related to screen capture but I haven't really tested that) — For macOS versions 13.2 through 15.5, I'm able to verify that system audio capture works great via ScreenCaptureKit, only flags that are needed are MacSckSystemAudioLoopbackOverride
. System permission prompts work fine.
For testing, I've created this little NPM package to make things easy in Electron Fiddle or an example project of your choice: https://github.com/alectrocute/electron-audio-loopback/tree/main
The implementation in my package is all I've tested, so we'll want to definitely reinforce in the docs that a lot of this is still experimental.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In order to provide complete guidance, the biggest questions in my mind are:
- For
12.3 ≤ version < 13.0
:- if you enable
MacSckSystemAudioLoopbackOverride
, does loopback capture work?
- if you enable
- For
13.0 ≤ version < 13.2
:- without passing any flags, does loopback capture work at all?
- For
15.0 ≤ version
:- without passing any flags, does loopback capture work at all?
- if you enable
MacCatapSystemAudioLoopbackCapture
, does loopback capture work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- For
12.3 ≤ version < 13.0
:
I will report back.
- For
13.0 ≤ version < 13.2
:
I will report back.
For
15.0 ≤ version
:
- without passing any flags, does loopback capture work at all?
I can verify that without passing any flags, loopback capture does not work. No errors thrown, the getDisplayMedia
audio media track just arrives in an ended state.
- if you pass
MacCatapSystemAudioLoopbackCapture
, does loopback capture work?
I will report back.
@nikwen I looked into these when doing my analysis in this comment. I think the flag descriptions are unhelpful, as they don't reflect the full behaviour as to how they're treated in the source (e.g. how |
Thanks, @shirakaba and @alectrocute, for diving further into this. Can you ping me (@nikwen) when you've figured out how it all works and have updated the PR? I'll be honest: I've never worked with system audio capture in Electron. |
Description of Change
desktopCapturer
audio loopback on macOSdesktopCapturer
: UseScreenCaptureKit
API to enable capturing loopback audio on macOS #47490.Checklist
npm test
passesRelease Notes
Notes: none