Archive for the 'untagged' Category

Graph of the Day: Crash Report Metadata

Friday, April 12th, 2013

When Firefox crashes, it submits a minidump of the crash; it also submits some key/value metadata. The metadata includes basic information about the product/version/buildid, but it also includes a bunch of other information that we use to group, correlate, and diagnose crashes. Using data collected with jydoop, I created a graph of how frequently various metadata keys were submitted on various platforms:

Frequency of Crash Report Metadata on 2-April-2013

The jydoop script to collect this data is simple:

import crashstatsutils
import json
import jydoop

setupjob = crashstatsutils.dosetupjob([('meta_data', 'json'), ('processed_data', 'json')])

def map(k, meta_data, processed_data, context):
    if processed_data is None:
        return

    try:
        meta = json.loads(meta_data)
        processed = json.loads(processed_data)
    except:
        context.write('jsonerror', 1)
        return

    product = meta['ProductName']

    os = processed.get('os_name', None)
    if os is None:
        return

    context.write((product, os, '$total'), 1)
    for key in meta:
        context.write((product, os, key), 1)

combine = jydoop.sumreducer
reduce = jydoop.sumreducer

View the raw data (CSV).

If you are interested in learning what each piece of metadata means and is used for, I’ve started to document them on the Mozilla docs site.

This graph was generated using the flot JS library and the code is available on github.

Zazzle Being Stupid

Tuesday, August 28th, 2012

My wife teaches high school physics. She loves putting quotes from famous physicists, mathematicians, and inventors on the wall of her classroom. After the death of Neil Armstrong, there was a lot of media coverage and I discovered a great quote from a speech that he made to the National Press Club in 2000:

“I am, and ever will be, a white-socks, pocket-protector, nerdy engineer, born under the second law of thermodynamics, steeped in steam tables, in love with free-body diagrams, transformed by Laplace and propelled by compressible flow.”

Because I thought my wife would really like this, I decided to make a poster of this quote on top of a picture of Mr. Armstrong walking on the moon. I created the design on zazzle.com and purchased it. Today I got the following response from Zazzle:

ORDER CANCELLATION NOTICE – Please modify your design(s) and re-order.

Hello Benjamin Smedberg,

Thank you for your recent order: XXXX

Unfortunately, we are unable to process your order due to a conflict with one or more of our acceptable content guidelines.

As a result, the following item(s) cannot be produced:

Title: Apollo 11 Neil Armstrong.jpg, I am, and ever wi…
Product Link: XXXX
Result: Not Approved
Content Notes:

  • Design contains text or image that is in violation of an individual’s rights of celebrity/publicity. If you are interested in purchasing Official Licensed Merchandise from Zazzle please visit: www.zazzle.com/brands
  • Design contains an image or text that may be subject to copyright. If you are interested in purchasing Official Licensed Merchandise from Zazzle please visit: www.zazzle.com/brands
  • Your design contains an image or text that may be trademarked. This may be due to the actual design of the product, description or search tags that are associated to your product. Please feel free to submit a new design to our Marketplace from original elements

I suppose that the speech itself could be subject to copyright, but given the size of the excerpt and the occasion I don’t see how it is really relevant. The photo, being a product of NASA, is not subject to copyright. And I don’t see how right of celebrity or trademark could possibly apply to this at all.

Come on Zazzle, stop being stupid.

Wall-Mount Computing

Monday, October 10th, 2011

For the past year or so, my wife and I have successfully kept all of our calendars (home, work, and kids) in Google calendar. We purchased matching phone computers, and calendering is one of the primary uses we have for the phones apart from actual calling. At work I keep an app tab open in Firefox with my calendar and it all works out pretty well.

I would like to extend our calendar system out to the kids. I want a touchscreen computer which I can mount for the older kids to see upcoming events and also see school information such as library/gym day, packing lunch which we keep in their calendar. I’d also like to be able to write a little app for them to check off their chores in the afternoon, or even leave virtual post-it notes for the family.

I have not yet found a suitable computing device which meets my needs and budget. It seems like somebody could make this kind of device pretty inexpensively because it is always plugged in: you don’t need the expensive battery setup of tablet computers. I can’t seem to find a tablet with a wall mount system, and I fear how quickly they could break something which wasn’t securely mounted.

My first choice would probably be an Android system, but any system that runs webapps and a decent browser would meet my needs. I guess I’m looking for something in the 7-12″ screen size range, and less than $500.

Do my readers have any suggestions?

Gregory Albert Michael Smedberg

Wednesday, February 23rd, 2011

Suzanne and I are pleased to (belatedly) announce the birth of Gregory Albert Michael Smedberg, born 23 January A.D. 2011.

Somehow the blog announcement got lost in the shuffle this time!

Privacy

Monday, January 3rd, 2011

I just published a privacy policy for smedbergs.us. Since I suspect my policy may be a bit unorthodox, I thought I’d post about it.

I, Benjamin Smedberg, will try to do what I think is appropriate with any information collected by this website. I believe that privacy is a tool which should be used to limit the powerful, not a fundamental right. To the extent that I don’t have much power, I don’t feel the need to provide any guarantee of privacy. Any information, including names, email addresses, and other identifying information, may be shared with others or even made public if I believe that is the correct thing to do.

I may make changes to my beliefs or this policy at any time. Please feel free to contact me if you want to ask questions, or change by beliefs.

I get the feeling that I value privacy differently than most of my Mozilla colleagues. I do not think that privacy is a fundamental right of mankind. Privacy is a tool to keep the powerful in check, by limiting how they may use information they have. To the extent that the government has vast information about and power over its citizens, it should be forced to respect their privacy as a counter to its power. This is the true purpose of the fourth amendment to the U.S. constitution.

I do believe that corporations which have information and power should also be limited, especially as they grow in size and scope. But I think that any proposals which treat privacy as a fundamental right instead of a system of checks and balances are ultimately doomed to failure.

Help Me With Starcraft Maps

Thursday, November 18th, 2010

I love Starcraft 2. In general, I think Blizzard has done a great job with the game, especially the multi-player mode and competitive matchmaking. But one part of the game which could use a lot of work is the system for playing and improving custom maps.

After being bitten by the Starcraft bug, I kept thinking of ways to modify or perhaps improve on the standard melee maps that are used in competive “ladder” matches. I watched a Day9 video commentary where he analyzed a nonstandard map. So I tried my hand at creating a custom map. I really enjoyed the process, and I think I came up with a pretty good first pass at a map.

It has been almost impossible to get people to play my map, or give me feedback. The UI for trying new maps in Starcraft itself is primitive, and there isn’t a place I can browse through maps and find ones which look interesting. It feels like Starcraft needs something like the Firefox Addons site, where people could browse and rate the maps.. The only remaining way to get feedback is to ask individual friends to play the map, either against me or against somebody else. This is difficult, because my friends usually have different schedules than I, and also have very different skill levels: playing against them would not be an even match.

I am therefore going to use my blog as an alternative way of soliciting feedback. If you play Starcraft (and I know many of my readers do!), please try out my maps and give me feedback on them. I now have three maps available:

Deserted Battleground (1v1)

Choose between three “natural” expansions from your starting position. One is more vulnerable but advances toward your opponent, and it’s easier to defend against airborne harassment. One in the back corner is far from your opponent. One is protected by cliffs, but very vulnerable to harassment. Xel’naga towers provide vision of the middle of the field, but side-paths can make for surprise attacks if not carefully scouted.

City Scout (1v1 or 4-player melee)

Your starting base has two ramps, and one is very close to an opponents starting position. Attacking the backdoor entrance involves very long distances, but good scouting is essential. Control of the center of the map makes for expansion

Butterfly Island (1v1 or 4-player melee)

A backdoor ramp on your main base is again vulnerable to harassment by speedy units (hellions/speedlings). Resources are scattered around the center of the map and there are several possible locations to plant a base and do mining.

I’d like feedback on the following areas at least:

  • Balance: Although the maps are generally mirror-images for balance, it is possible that certain features of the map may favor one race or another. Are there specific features which you think are unbalanced.
  • Play: are there any features of the map which you love or hate? I use backdoor entrances more than the standard Blizzard maps, and occasionally some unusual mineral/geyser placement.
  • Visual appeal: Do you like the general appearance of the map? Are there things you would improve?

Feel free to leave me comments here at my blog, in email, or by sending messages in-game. My Starcraft handle is Odysseus/749.

In order to play the maps,you must create a game in the Multi-Player section of Starcraft. This is true even if you just want to play the maps against the AI, because you cannot search for maps in the “Play Against AI” panel. You can then use the search function to find the maps by name. If you’re a bronze/silver-level player, I’d love to play games against you: please invite me/add me as a Starcraft buddy.

The Firefox Plugin Hang Detector

Wednesday, June 9th, 2010

If all goes well, Firefox 3.6.4 will be released with support for out-of-process plugins on Windows and Linux next week. As part of this project, Firefox now features a “hang detector” for plugins. The hang detector helps protect users from plugins and plugin scripts which stop responding for 10 seconds.

When Firefox makes an NPAPI call to a plugin which is being run in a separate plugin process, that call is translated into a message which is posted to the process. Firefox then waits for the response. If the plugin takes longer than 10 seconds to respond to a call, Firefox performs the following actions:

  • Stop the plugin process and collect a plugin-process “crash” minidump.
  • Collect a “crash” minidump from the browser process.
  • Terminate the plugin process.
  • Display the plugin-crashed UI, giving the user the opportunity to refresh the page and try again.

There are several reasons plugins might trigger the hang detector:

  • A plugin script (such as ActionScript run inside the Flash plugin) may be in an infinite loop or performing a very long computation.
  • The plugin itself may have a bug (such as a threading deadlock) which causes it to stop responding.
  • The plugin is not deadlocked, but is not processing events quickly enough, causing the event to linger waiting to be processed.
  • The implementation of Firefox out-of-process plugins may be causing a deadlock.

It is this last possibility that is most concerning, and we have pored over our Firefox crash stats studying the hang reports that we receive, trying to categorize the reports into one the categories above. During the long process of Firefox 3.6.4 release candidates, we have identified and fixed several “tophangs”: see bug 561817 for an example.

When hang reports were first submitted to crash-stats, it was very difficult to distinguish full crashes from hangs. It was also impossible to correlate the browser and plugin parts of a hang. Since then, the Socorro team has done an outstanding job of improving the crash-stats UI so that we can analyze hang reports. It is now possible, using the advanced query page, to search for only crash reports, only hang reports, and to limits searches to either the browser process or the plugin process. Individual hang reports are now cross-linked: see the following reports for an example:

In this particular hang-pair, it isn’t immediately clear what is causing the hang. The browser is painting, and during the process of painting a windowless Flash plugin sends the NPP_SetWindow NPAPI call. This is the hanging call. At the time the hang report was collected, the NPAPI thread of the Flash plugin is calling NPN_InvalidateRect, and inside that implementation locking a mutex. But we don’t know, looking at the stacks, whether this is a deadlock on that mutex, or whether the plugin just happened to be making that call at the time the hang detector collected its stack.

In some cases, developers may want to disable the hang detector. If you are using the Flash debugger, or if you are debugging Firefox, you can change the hang detector timeout by changing the preference dom.ipc.plugins.timeoutSecs. Setting a value of -1 disables the hang detector entirely. For more information on setting preferences, see about:config on MozillaZine.

Firefox, Safe From Plugin Crashes

Wednesday, March 3rd, 2010

Today we released the first Mozilla Developer Preview containing multi-process plugins. Firefox is now safe from plugin crashes, on Windows and Linux.

Where Do I Get It?

Here!

How Can I Help?

Warning: the developer preview is alpha-quality and may contain bugs. If you are willing to browse on the wild side, please download and use the preview build as your day-to-day browser. Visit web pages which use plugins. We have done a fair bit of testing with the most popular plugins such as Flash and Silverlight, but there are many less popular plugins which may not be tested at all. If you don’t know what plugins are installed, go to Tools / Add-Ons / Plugins.

If you encounter any crashes, please make sure you submit the crash reporter and try to provide a detailed comment about what you were doing at the time of the crash.

If you encounter any unexpected behavior, please file a bug, including the following information:

  • The website you were visiting
  • Plugin information copied from about:plugins
  • Expected and actual behavior
  • Whether the behavior is fixed when you change the IPC preference (see below).

How Does It Work?

The Mozilla Developer Center has an article describing the underlying architecture: there is a shim layer which acts like a plugin in the browser process and a browser in the plugin process. Function calls are translated into RPC messages passed between the two processes.

One of the key pieces of technology we’ve developed to make message passing more reliable is IPDL. IPDL is a language which precisely describes the messages that can be passed between processes, and allows developers to define a state machine and error handling conditions for messages and resources shared across processes. IPDL layers on top of an IPC stack that Mozilla copied from the Chromium codebase. For instance, this protocol describes the messages associated with a plugin instance on a web page. Each message may be delivered asynchronously, synchronously, or with RPC semantics.

Common Questions

What happens when a plugin crashes?

When a plugin crashes, the Mozilla crash reporter kicks in and submits the crash report to Mozilla. Then we replace the plugin display with the crashed-plugin UI. When you reload the page, we restart the plugin process.

Why don’t you reload the plugin automatically?

We thought about this: however, web page scripts often have state associated with a plugin. If we reload the plugin without reloading the entire page, those scripts will have unexpected state and can get very confused. Overall, it causes fewer problems for the user to simply refresh the page.

What’s the name of the plugin process?

The name of the plugin process in the Windows task manager is mozilla-runtime.exe (mozilla-runtime on Linux).

Why is mozilla-runtime using so much memory?

In general, our automated tests show that Mozilla actually uses less overall memory than it did previously. However, there are some measurement issues which may cause problems: memory which is shared between the two processes, such as mapped memory segments and code, may be counted twice. The Chromium project has had similar problems accurately measuring and presenting memory usage information. If there are particular pages or plugins which show a regression in memory usage, please file a bug!

What about Mac?

MacOS presents some unique challenges: the traditional drawing and interaction model for plugins is very difficult to do across processes. We are working on Mac support for multi-process plugins, and hope to have a preview of this work available soon.

Preferences

Multi-process plugin behavior can be controlled from preferences:

dom.ipc.plugins.enabled.filename

Each plugin can be controlled independently. For example, if Acrobat is causing problems, you can run it in-process by setting the pref dom.ipc.plugins.enabled.nppdf32.dll to false. The filename should be lower-case.

dom.ipc.plugins.enabled (true)

This controls the multi-process behavior for all plugins which don’t have a specific pref set as above.

After these preferences are changed the browser must be restarted.

Choose To Shave

Wednesday, December 2nd, 2009

Johnathan Nightingale, an otherwise upright and honorable man, is leading our world astray. In the name of freedom, adventure, and independence from societal mores, he encourages NSID: No Shaving in December.

Anyone has the freedom to shave or not-shave as they see fit, and I would not dare to take away their right. But do not be hoodwinked by this movement! At its core is not fun-loving independence and adventure. Beneath the surface, behind unshaved faces, lies a sinister purpose: to destroy the conventions upon which proper functioning of society is maintained. It may seem at first glance that shaving is merely a choice, an aspect of personal decoration. But looking deeper, we see that decoration is nothing trivial: the word “decoration” derives from the Latin “decorus”, meaning “fit” or “proper”. Abstaining from shaving is neither fit nor proper in public society, and those who choose this lifestyle should be shunned. Take a stand for decency, wholesomeness, and civic life, and shave regularly and with passion.

Freedom is good. Use it wisely. Choose to shave.

The Best Part of Being a Sister

Thursday, September 10th, 2009

A project Ellie made at school, drawing her sisters

Ellie drew this in kindergarten today. Her younger siblings, from left to right: Claire (curly hair, 4), Maddie (1), Micah (2), Ellie (5), Abigail (3), and Bekah (4 months). I love how she drew Bekah lying down, since she can’t walk yet, and how everyone else is in height order.