Introducing Scroll-Graph

For awhile I’ve been warning people that the FPS counter is very misleading. There’s many case where the FPS might be high but the scroll may still not be smooth. To give a better indicator of the scrolling performance I’ve written Scroll-Graph which can be enabled via layers.scroll-graph. Currently it requires OMTC+OGL but if this is useful we can easily port this to OMTC+D3D. Currently it works best on Desktop, it will need a bit more work for mobile to be compatible with APZC.

To understand how Scroll-Graph works lets look at scrolling from the point of view of a¬†physicist. Scrolling and pan should be smooth. The ideal behavior is for scrolling to behave like you have a page on a low friction surface. Imagine that the page is on a low friction ice arena and that every time you fling the page with your finger or the trackpad you’re applying force to the page. The friction of the ice is applying a small force in the opposite direction. If you plot the velocity graph you’d expect to see something like this roughly:

Expected Velocity Graph

Expected Velocity Graph

Now if we scroll perfectly then we expect the velocity of a layer to follow a similar curve. The important part is *not* the position of the velocity curve but it’s the smoothness of the velocity graph. Smooth scrolling means the page has a smooth velocity curve.

Now on to the fun part. Here’s some examples of Scroll-Graph in the browser. Here’s I pan a page and get smooth scrolling with 2 minor inperfections (Excuse the low FPS gif, the scrolling was smooth):

Smooth ScrollGraph velocity = Smooth Scrolling

Smooth ScrollGraph velocity = Smooth Scrolling

Now here’s an example of a jerky page scrolling. Note the Scroll-Graph is not smooth at all but very ‘spiky’ and it was noticeable to the eye:

Jerky scrolling = Jerky Scroll-Graph

Jerky scrolling = Jerky Scroll-Graph

Off Main Thread Compositing (OMTC) and why it matters

Like most desktop applications, Firefox is driven by an event loop. Currently this event loop is servicing a lot of events including page layout, drawing, image decoding and don’t forget JS. We do our best to handle events quickly (in a few milliseconds) and break up the ones that we know will take longer (such as image decoding). Any event, such as poorly written JS, that takes too long to process will cause the application to feel sluggish and will cause updates, animations and videos to pause.

Every web page is broken into a set of layers (backgrounds, canvas, video, web contents, position fixed element, elements that are being animated) by our layers system. When these layers are updated they must be flattened to the screen to show a final frame. This process is called ‘composition’. Currently this composition is driven by the main event loop. While compositing does add more load to the event loop from my experience the load it adds is often negligible. After all with hardware accelerated layers backends the work is mostly just queuing a few simple graphics command to the GPU.

Simplified painting pipeline (Desktop Current)

So why move compositing to a second thread if it’s relatively cheap? Because we need the event loop to be responsive to service composite events and any slow events (>15ms), such as a long JS script, will cause us to drop frames. By compositing on a separate thread we can still service layers update and keep the browser partially responsive even if the event loop is momentarily blocked by a long running script.

Simplified OMTC painting pipeline (Fennec Beta)

There is one catch however. If the main event loop is being hogged by an event then that will block certain updates, like page reflows, from reaching the compositor. However other types of updates can still be made while the main event loop is bogged down such as video which we decode in yet another thread, css/gif animations and plugins to name a few. Another use case is to support smooth panning and zooming on mobile. The goal we’re aiming for is to make Firefox more responsive.

The feature just shipped into the latest Fennec beta and provides a smoother experience while consuming less resources.

Simplified OMTC painting pipeline (Desktop/Fennec Future)

In the future we plan on landing more changes that will leverage this new architecture. We are considering many proposal shown above. Some are uncertain at this point such as having canvas workers. Right now our current focus is passing decoded video frames directly to the compositor and handling simple CSS animation asynchronously. These features will allow video and animations to perform smoothly over short periods of blocking similar to async scrolling on Fennec. We’re hoping to have some of these features landed in Q2 for mobile and desktop where OMTC will likely still be behind a preference until later this year. Stay tuned to these bugs to try out these features when they land.