Disclaimers: Examples involving Service Workers are intentionally simplified unless relevant to the parent intercept changes.\u00a0 All code links are made to DXR<\/a> at a file granularity because I expect those links to be long-term stable and less of a hassle than linking to a query page.\u00a0 You may need to control-f find the term I refer to, however.\u00a0 For code exploration and understanding, I recommend searchfox<\/a>.<\/p>\n Service Workers that have registered a fetch (spec<\/a>) event handler during initial script evaluation will receive fetch events.\u00a0 These fetch events fall into 2 groups:<\/p>\n “fetch” event handlers must do one of the following things when they receive their event:<\/p>\n ServiceWorkerManager<\/a> is the brains of Gecko’s Service Worker implementation.\u00a0 It knows if a given URI is covered by a Service Worker scope.\u00a0 It knows if a given document is controlled.\u00a0 It lets you dispatch fetch events, notification events, everything.\u00a0 Currently there’s one ServiceWorkerManager per process.\u00a0 The ServiceWorkerManagers share unified race-proof Cache API<\/a> storage in the parent process, and a race-prone-broadcast understanding of what registrations exist and which chrome-namespaced cache name where the SW script and its dependencies are stored.<\/p>\n This will change in the near future, but the net result is that for the patch under discussion:<\/p>\n Terminology note: When I say “instance” I refer to a distinct version of the ServiceWorker as identified by its id<\/a> issued during the update process<\/a> with its own specific Cache API storage. In other words, each of a ServiceWorkerRegistration<\/a>‘s installing, waiting, and active is a different instance.<\/p>\n Part of why we need the parent intercept patch is for that bright future where there is only one ServiceWorkerManager and it lives in the parent process.\u00a0 And there is only ever one ServiceWorker instance across all content processes in the browser.\u00a0 To start with, all of these instances will live in a single Service Worker process for main thread contention reasons.\u00a0 As various Project Quantum advancements are made that reduce main thread contention, we should be able to spawn ServiceWorkers in normal content processes.<\/p>\n The diagram above attempts to capture a very simplified version of the expected first steps.<\/p>\n In Gecko and its network layer, Necko, channels are the primary abstraction for network requests.\u00a0 If you have a string URL, you ask nsIIOService<\/a> to create an nsIURI<\/a> and using that, create an nsIChannel<\/a> (which is also an nsIRequest<\/a>).\u00a0 nsIChannel’s asyncOpen2<\/a> method begins the asynchronous process of opening the channel, with notifications provided to the passed nsIStreamListener<\/a>\/nsIRequestObserver<\/a> “listener” which exposes the nsIInputStream<\/a> which contains the actual data stream.<\/p>\n Each channel corresponds to exactly one URI, exposed as “URI” on the nsIChannel.\u00a0 In the event of a redirection, a new channel is created with the new URI.\u00a0 The original URI is propagated to the new channel as well, and is exposed as “originalURI”.\u00a0 HTTP channels additionally may store the “documentURI” of the document originating the request which allegedly is for 3rd-party cookie blocking, but the mozIThirdPartyUtil<\/a> implementation<\/a> and most other code now gets their information from the channel’s associated nsILoadInfo<\/a>.\u00a0 nsILoadInfo provides important context about why a network load was started and what it will be used for.<\/p>\n Each network protocol has its own implementation directory beneath mozilla-central<\/a>\/netwerk<\/a>\/protocol<\/a>.\u00a0 Service Workers only support the HTTP protocol<\/a> (in secure contexts<\/a>).<\/p>\n In single process Firefox (or for HTTP connections initiated from the parent process), an instance of the “real” HTTP channel class nsHttpChannel<\/a> is created.\u00a0 In multi-process Firefox when code in a child (AKA content) process wants to open an HTTP(S) URI, an HttpChannelChild<\/a> instance is created instead.\u00a0 Each HttpChannelChild instance *may* cause a corresponding HttpChannelParent<\/a> instance to be created in the parent process.\u00a0 The child and parent communicate with each other via the PHttpChannel<\/a> IPDL protocol<\/a>.\u00a0 I place an emphasis on *may* because that is something this patch changes.<\/p>\n The HttpChannelParent instance creates and owns an nsHttpChannel instance to do the “real” work.\u00a0 nsHttpChannel deals with the HTTP cache<\/a> and perform the actual network communication.<\/p>\n HttpChannelParent also creates an HttpChannelParentListener<\/a> which is the nsIStreamListener<\/a> it passes to the nsHttpChannel.\u00a0 The HttpChannelParentListener provides a layer of indirection so that the stream can be “diverted” from the child back to the parent and the HttpChannelParent can be removed from the picture.\u00a0 In the simplest case, the HttpChannelParentListener simply redirects OnStartRequest, OnDataAvailable, and OnStopRequest callbacks to its mNextListener, the HttpChannelParent.<\/p>\n HttpChannelChild and HttpChannelParent work together to proxy requests from the child to the parent and data from the parent to the child.\u00a0 Data travels as nsCStrings in OnTransportAndData<\/a> IPC messages, not via file descriptors passed to the child process.<\/p>\n Logic in the child process doesn’t notice the difference between HttpChannelChild and nsHttpChannel because it interacts via XPCOM interfaces like nsIHttpChannel<\/a> rather than the concrete implementation types.\u00a0 This is helped by nsHttpChannel and HttpChannelChild sharing HttpBaseChannel<\/a> as a base class that contains common logic related to implementing the XPCOM interfaces.<\/p>\n As noted above, each channel is associated with a single URI.\u00a0 Accordingly, a new URI means a new channel.\u00a0 For HTTP, redirects usually happen because an HTTP request returned a 3xx response<\/a>.\u00a0 However, Service Workers introduce an additional complication because the fetch Response<\/a> instance eventually provided to respondWith<\/a> need not have the same URL as the request.\u00a0 Because of the 1 channel = 1 URI invariant, an internal redirect must be performed in that case.\u00a0 More on that later.<\/p>\n At a high level, redirects are simple:<\/p>\n In practice, they’re a bit more complex.\u00a0 Here’s a high-level sequence diagram of the (in the parent) nsHttpChannel redirection life-cycle using the StartRedirectChannelToURI programmatic redirect API used by nsIHttpChannel::redirectTo and the InterceptedChannel mechanism (more on that later).\u00a0 For 3xx redirect responses, nsHttpChannel::AsyncProcessRedirection uses ContinueProcessRedirectionAfterFallback whose behavior is analogous to StartRedirectChannelToURI and whose ContinueProcessRedirection is analogous to ContinueAsyncRedirectChannelToURI\/OpenRedirectChannel.<\/p>\n Elaborations on the diagram:<\/p>\n When trying to understand code, I always find it useful to know the motivating use-cases.\u00a0 Especially because these use-cases may interact in ways that increase complexity.\u00a0 So, what causes redirects?<\/p>\n The normal event loop assumption is that events will run in the order they were enqueued, each event running to completion before the next event starts running.\u00a0 Nested event loops like those used by synchronous XMLHttpRequest and synchronous\/rpc Inter-Process Communication (IPC) calls<\/a> violate this assumption.<\/p>\n The canonical Necko concern is that a call to an nsIRequestObserver<\/a>::OnStartRequest implementation will be on the stack spinning a nested event loop, and that a call to the same listener’s (nsIRequestObserver subclass) nsIStreamListener<\/a>::OnDataAvailable implementation will be made.\u00a0 This is not a problem for non-e10s channels because Necko event delivery has built-in back-pressure.\u00a0 Core implementations like nsInputStreamPump<\/a> and NS_AsyncCopy’s nsAStreamCopier<\/a> go out of their way to maintain thread-safe state machines that only allow events to be delivered after the previous event has run to completion.\u00a0 Other low-level interfaces encourage idioms like this, for example nsIAsyncInputStream<\/a>‘s asyncWait requests a single callback notification rather than a subscription to all future notifications.<\/p>\n Messages\/events relayed via IPC can’t have this free back-pressure.\u00a0 At least as long as they’re not “sync” IPC calls.\u00a0 But, for reasons of performance and sanity, it is desirable for all IPC messages to be “async”.\u00a0 Additionally, there are distributed computing issues to deal with where both the parent and child may have different opinions of the channel’s state and in-flight multi-step asynchronous operations.\u00a0 Thus we have the ChannelEventQueue<\/a>, a suspendable queue of ChannelEvent instances (basically non-XPCOM nsIRunnables<\/a>).\u00a0 When receiving IPC events, Necko *Parent and *Child classes will construct ChannelEvent subclasses and use ChannelEventQueue::RunOrEnqueue to ensure events are run sequentially.<\/p>\n Logic in the child content process may realize that the data it’s receiving should be consumed by the parent process instead.\u00a0 In that case the channel is diverted back to the parent process using nsIDivertableChannel<\/a>.\u00a0 It’s currently used by:<\/p>\n These are both nsIURIContentListener<\/a> implementations that are invoked by the nsIURILoader<\/a>.\u00a0 When a URL is opened (in a navigation context), the URI loader opens the channel to discover the content type.\u00a0 This is initiated by a docshell in the (child) content process.\u00a0 And since the content type comes from the “Content-Type” HTTP header, it means data has already started flowing to the child process.\u00a0 Normally this is what we’d want because for HTML documents we want to parse and render them in the child.\u00a0 But content processes are sandboxed and cannot directly write to disk, open applications, or manipulate the certificate store.<\/p>\n The above sequence diagram expresses the high level parent-child diversion flow using the external helper app case as an example.\u00a0 In prose:<\/p>\n Redirects get more complicated under e10s.\u00a0 The good news is that “normal” redirects all happen in the parent, based around the same nsHttpChannel control flow discussed above.\u00a0 The e10s logic builds on that.<\/p>\n foo DISCUSS<\/strong><\/span> foo.\u00a0 See http channel redirect crash notes which happily overlap with this. <\/p>\n <\/p>\n nsINetworkInterceptController<\/a> is the high level interaction point between Service Workers and Necko.\u00a0 Each “document” (really, nsDocShell<\/a>) implements the interface.\u00a0 This means it has the context to know whether the document is controlled and by whom in addition to any arguments passed in method calls.\u00a0 The interface is exposed to the channel by being set as the notificationCallbacks<\/a> attribute on the channel<\/a> or the load group<\/a>.<\/p>\n The interface defines 2 methods:<\/p>\n Under e10s, the nsDocShell instances live in the content process.\u00a0 So what happens in the parent process?\u00a0 The answer is that HttpChannelParentListener also implements the interface and handles the calls.\u00a0 nsHttpChannel always attempts to retrieve an nsINetworkInterceptController from its notificationCallbacks attribute and invoke its shouldPrepareForIntercept method to determine whether interception is appropriate.\u00a0 This happens regardless of whether it’s an e10s scenario where the nsHttpChannel was created by an HttpChannelParent or it’s non-e10s and the nsHttpChannel was directly created in the parent.\u00a0 nsHttpChannel doesn’t distinguish between the two, it just consumes the interface.<\/p>\n As mentioned above, nsIInterceptedChannel is the interface that the Service Worker fetch event handler uses to provide its Response object.\u00a0 Currently this entails a number of separate method calls, but in the future it’s likely the Response will be directly provided instead.<\/p>\n There are two implementations of nsIInterceptedChannel<\/a>: InterceptedChannelChrome and InterceptedChannelContent, both of which subclass InterceptedChannelBase.\u00a0 Although the Chrome variant will only ever be created in the parent process and the Content variant in a child content process, they do not have a parent\/child relationship like HttpChannelParent and HttpChannelChild.\u00a0 We have two implementation types because they each hold mChannel references to the concrete channel types, nsHttpChannel in the parent, and HttpChannelChild in the child.\u00a0 They do this because of their differing means of injecting the synthesized content and resetting interception.<\/p>\n In nsHttpChannel, Service Worker interception is performed during the OpenCacheEntry stage of processing.\u00a0 If we intercept and provide a synthesized result, we never hit the HTTP cache and a synthesized cache entry is generated.\u00a0 InterceptedChannelChome has the specific logic to populate the synthetic cache entry.\u00a0 And it handles calls to resetInterception by manually triggering a programmatic redirect; nsHttpChannel itself has no resetIndirection method.<\/p>\n InterceptedChannelContent’s behavior changes with the patch, which we’ll get to in a later section.\u00a0 The key difference to be aware of is that HttpChannelChild is very aware of ServiceWorkers and interception and so InterceptedChannelContent is able to be a thinner layer.\u00a0 HttpChannelChild exposes a ResetInterception method it is able to invoke directly, and post-patch, content synthesis is simpler too.<\/p>\n Differences in behavior between e10s and non-e10s stem from whether the nsINetworkInterceptController is an nsDocShell (non-e10s in parent, e10s in child) or an HttpChannelParentListener (e10s in parent only).<\/p>\n <\/p>\n The current pre-patch implementation optimizes for the ability to process Service Worker interceptions in the child with as little parent involvement as possible.\u00a0 In a world where the decision to intercept and the processing of the intercept both happen entirely in the child process, this makes a lot of sense.\u00a0 But it also comes with a massive amount of complexity because it means the child channel needs to duplicate logic that would normally be handled in the parent, plus the additional permutations when the parent does need to get involved.<\/p>\n The parent needs to be involved when any of the following things happen:<\/p>\n The child can handle things without involving the parent process when:<\/p>\n Redirects happen in the parent.\u00a0 Why?\u00a0 Because that’s where they happened prior to the introduction of Service Workers.\u00a0 Why?\u00a0 Because nsHttpChannel already knew how to perform redirects and there’s only a performance hit to remote the decision-making process from the parent where the network I\/O is happening down to the child and then back up again.\u00a0 (At least that’s my guess on the simplest evolutionary explanation.)Redirect3Complete<\/p>\n As a result, if a ServiceWorker responds to a fetch event with a redirection, an HttpChannelParent will need to be spun up.\u00a0 Things get more complicated because this transfers control to the parent and that redirected channel may also need to be intercepted.<\/p>\n As mentioned previously, HttpChannelParentListener implements nsINetworkInterceptController.\u00a0 This is how the child regains control of the channel.\u00a0 When responding with a synthesized redirect, the child tells the parent that it should intercept the redirected channel.\u00a0 Its shouldPrepareForIntercept will then return true so that its channelIntercepted method will be called.\u00a0 When it’s channelIntercepted method is called it simply saves off the intercepted channel and does nothing.\u00a0 Instead, the next action will be taken by the HttpChannelChild whose CompleteRedirectSetup method will be sending a SendFinishInterceptedRedirect ping-pong.\u00a0 The parent will stop sending IPC messages on receipt and issue its own send.\u00a0 The child will then delete the parent and continue the redirect handling in the child by invoking AsyncOpen2. The synthesized response data and its nsIStreamListener events are actually being generated by HttpChannelChild’s mSynthesizedResponsePump nsInputStreamPump.\u00a0 This means that in the OnDataAvailable and other listener notifications, the nsIRequest* aRequest is that of the nsInputStreamPump rather than the HttpChannelChild.\u00a0 This is potentially confusing to the listener.\u00a0 So the InterceptStreamListener is registered as the pump’s listener and it redirects each event to the child’s listener, passing the child as aRequest instead.<\/p>\n THIS IS AN OUTDATED DRAFT, IF YOU CAN READ THIS, I PUSHED SOME BUTTONS WRONG.\u00a0 OOPS. Meta: Bug 1231222 overhauls how Service Worker (spec, MDN: overview details) network interception happens in Gecko.\u00a0 I’m writing this post in support of the … Continue reading Background (which you may already know)<\/h1>\n
Service Workers<\/h2>\n
Service Workers and Fetch<\/h3>\n
\n
\n
Service Workers and Multiple Processes: Now<\/h3>\n
![\"Diagram](\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2017\/03\/service-worker-multi.svg\")
<\/p>\n\n
Service Workers and Multiple Processes: The Future<\/h3>\n
![\"Simplified](\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2017\/03\/service-worker-remoted.svg\")
<\/p>\nNecko HTTP Channels<\/h2>\n
Basics: URIs, Channels, Stream Listeners<\/h3>\n
HTTP Channels under Multi-process Firefox (AKA Electrolysis AKA e10s)<\/h3>\n
![\"Simplified](\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2017\/03\/class-httpchannel-e10s.svg\")
<\/p>\nComplication: Redirects<\/h3>\n
\n
![\"Sequence](\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2017\/03\/redirect-nshttpchannel-api.svg\")
<\/a><\/p>\n\n
\n
Sources of Redirects<\/h3>\n
\n
\n
\n
\n
\n
ChannelEventQueue, a Nested Event Loop e10s IPC bandage<\/h3>\n
E10s Complication: Diversions<\/h3>\n
\n
![\"Sequence](\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2017\/03\/diversion-external-after.svg\")
<\/a><\/p>\n\n
E10s Complications: Redirects<\/h3>\n
\n<\/strong><\/p>\nService Worker and HTTP Channel Interactions<\/h2>\n
nsINetworkInterceptController<\/h3>\n
\n
\n
Intercepted Channels<\/h3>\n
![\"Confusing](\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2017\/03\/intercepted-channels.svg\")
<\/a><\/p>\nComplications: Redirects and Secure Upgrades<\/h3>\n
The Old Way: Child Intercept<\/h1>\n
Strategy: Don’t Get The Parents Involved<\/h2>\n
\n
\n
Complexity: Redirects<\/h2>\n
\n**need to document<\/strong> relationship of the self-deading channel to the redirected channel**.\u00a0 It looks like the same child.\u00a0 How do replacement channels work for this?<\/p>\nInterceptStreamListener<\/h3>\n
The New Way: Parent Intercept<\/h1>\n","protected":false},"excerpt":{"rendered":"