Firefox 4 implements a new strategy for painting windowless plugins. This should result in improved performance and responsiveness when users are visiting sites such as Hulu and Vimeo which make use of windowless Flash to render their videos.

Background

On Windows and Linux, there are two basic modes in which plugins can render, windowless and windowed. When a windowed plugin instance is requested, Firefox creates a native widget; the operating system delivers native events, including paint requests, directly to the plugin window. This is simple, but it has a significant disadvantage: the plugin doesn’t participate in normal web layout. This means that the plugin cannot be transparent, and CSS effects such as opacity and most transformations cannot be applied to the plugin. Youtube currently mostly uses windowed plugin instances for rendering their videos.

Windowless plugin, on the other hand, do not have a native widget. Instead, events such as mouse and keyboard events, as well as requests to paint the plugin, are received by the browser and forwarded to the plugin using the NPP_HandleEvent API. Hulu and vimeo both make use of windowless plugin instances. Any Flash plugin with the wmode=”opaque” or wmode=”transparent” attribute in their <embed> or <object> tags is using windowless mode.

Asynchronous Painting

In Firefox 3.6 and earlier, every time the operating system asks the browser to paint its window, we synchronously walk the layout frames and ask each frame to paint itself. When a windowless plugin frame is asked to paint, it synthesizes a WM_PAINT event and sends it to the plugin using NPP_HandleEvent. This is straightforward, but it does involve a blocking call and process round-trip for plugins which run in a separate process.

In Firefox 4, we don’t paint the plugin directly to the screen. Instead, as soon as the plugin is visible we ask it to paint to a retained buffer (an X surface on Linux, and a shared-memory DIBSection on Windows). We retain the pixel data for the next time Firefox is asked to paint. When using D3D rendering, we can eagerly upload the plugin data to a texture, and the plugin texture is composited by the graphics card and GPU.

A Behavior Change: Opacity on Windows

On Windows, the new asynchronous painting API has one significant side effect: plugins responding to a WM_PAINT message must be aware of opacity. The device context which is passed to the plugin is backed by a DIBSection with an opacity channel. Certain Windows Drawing functions, such as the DrawText function, are not aware of opacity and will incorrectly overwrite the opacity data, leaving black splotches where transparent text was intended. Windows drawing functions such as AlphaBlend are the correct way to draw while preserving transparency information.

Most Flash and Silverlight sites work correctly with this new function, but there are a few Flash features which continue to use the old Windows APIs. This bug shows itself in current Firefox nightly builds as black splotches where text should be painted, and is being tracked by Mozilla bug 611698; we are working with Adobe to resolve this issue before Firefox 4 is released.

Testing Wanted

Although our implementation of asynchronous painting passes all of our internal tests and appears to work well in general, the web is a big place and we can’t test every page or plugin available on the web. We would really like people who develop with plugins or use plugin-intensive sites to test Firefox nightly builds and report any bugs which you see! These builds are updated to our most recent Firefox each night, so you will always have the latest and greatest features (and sometimes bugs) to experiment with.

Note to Flash Authors

If your site uses wmode=”transparent” but your Flash application is not actually transparent or translucent, you can get better performance in both Firefox and Chrome by switching to wmode=”opaque”. Please use wmode=”opaque” for content which does not need actual transparent behavior.