Archive for the 'Mozilla' Category

emacs javascript mode

Tuesday, April 15th, 2008

Dear lazyweb,

I would really really love a JavaScript mode for emacs which indented “for each” correctly:

for each(let o in a) {
  // single indent
  process(o);
}

I’d love it enough to buy a 6-pack for the person who made it happen.

History meme

Tuesday, April 15th, 2008

I think we must all have OCD. It’s interesting (to me at least) which things I launch from the GUI (firefox, emacs). Is there a way to get the “most-clicked launchers” in gnome?

tchaikovsky.smedbergs.us:~
bsmedberg $ uname -a
Linux tchaikovsky.smedbergs.us 2.6.23.15-137.fc8 #1 SMP Sun Feb 10 17:03:13 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
tchaikovsky.smedbergs.us:~
bsmedberg $ history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head
271 hg
148 make
122 cd
44 less
43 rm
35 dir
33 exit
27 cvs
25 svn
18 /builds/gcc-dehydra/installed/bin/gcc

Wandering around Mercurial Repositories

Wednesday, March 26th, 2008

One of the things that makes mercurial useful and frustrating at the same time is that history is not linear. History can separate and join at multiple places. Tools such as hg log can’t show the outline of history clearly. There have been several attempts to remedy this situation: hg glog prints a revision graph using ASCII art, and hg view allows browing a repository history in a GUI app written in TK.

I think we can do better, so I started a new project: it is an extension to hgweb which allows browsing repository history graphically in a web browser. You can navigate graphically through revisions, and it will load new revisions on demand. Note that I’ve only tested it with Firefox, and it won’t work in IE (no SVG support):

mozilla-central on office.smedbergs.us.

There is certainly lots more that can and should be done:

  • The layout algorithm is really dumb
  • Revisions should link back to hgweb for viewing diffs, etc
  • Would be nice to zoom out to get a larger picture with less detail

Unfortunately I don’t have any more time to spend on this project. So I’m looking for volunteers to come forward and finish it.

Installation Instructions

The code is here. It is packaged as a single mercurial extension which must be installed in a global hgrc file, and a www/ directory with static files. I believe this extension will work correctly against Mercurial 1.0: I’ve been testing it against a mercurial-crew pull from a couple days ago.

  • In www/navigate.js configure BASEURL for your site
  • In www/index.xhtml populate select-repo with the repositories on your site.

The hgweb extension exists primarily to provide repository information in JSON format. I hope that it will be useful for other kinds of projects as well.

Christians, to the Pascal Victim

Sunday, March 23rd, 2008

Christians, to the Pascal Victim
Offer your thankful praises!

A Lamb the sheep redeems;
Christ, who only is sinless,
Reconciles sinner to the Fater.

Death and life have contended
in that combat stupendous:
The Prince of life, who died,
reigns immortal,

Speak, Mary, declaring
What you saw, wayfaring.
“The tomb of Christ, who is living,
The glory of Jesus’ resurrection;

Bright angels, attesting
The shroud and napkin resting.
Yes, Christ my hope is arisen;
To Galilee he goes before you.

Christ indeed from death is risen,
our new life obtaining.
Have mercy, victor King,
eve reigning!

Amen. Alleluia!

The sequence for Easter Sunday, “Victimae pachali laudes”

QEMU, XEN, and Windows

Monday, March 10th, 2008

Mozilla got me a new computer recently: one that can build Mozilla clobber in 7 minutes, or do a full XPCOMGC rewriting pass in 90 minutes. I wanted to be able to run Windows in a VM on this machine, but I’ve had a really unpleasant time setting up virtualization. This is a tale of what I’ve tried:

  1. I’m running Fedora Core 8.
  2. I told it to install the virtualization group on install, which means I got both the Xen and KVM stacks. Installing the Xen stack automatically sets up grub to boot into a Xen hypervisor.
  3. I tried creating a new VM in the Virtual Machine Manager GUI. I chose two virtual processors.
  4. I tried to install Windows XP by hooking that virtual machine up to /dev/cdrom directly. But the host automount factility kept causing conflicts between the guest and the host. So I copied the Windows CD to a local .iso file and told Xen to use that instead. So far so good, Windows XP tries to install.
  5. Every time Windows tries to restart (which is a lot, when you’re trying to install Windows, updates, and developer tools) the VM hangs. You can’t even shut down the VM from the host GUI: it’s stuck, and you have to reboot the host.
  6. Connecting to the VM from VNC is… difficult. If you use the default option of PS/2 mouse emulation, the guest mouse cursor doesn’t track the host cursor correctly. Configuring the USB tablet emulation fixes this problem. But frequently when typing quickly the keyboard will have doubled keys, missed keys, or the modifiers (shift and control) will simply stop functioning.
  7. Trying to connect from another machine (my mac laptop) is an exercise in disappointment. My old standby “Chicken of the VNC” won’t display anything other than black. A newer client, JollysFastVNC, connects, but I can’t make the “control” key work.
  8. I give up and disable Xen, starting the same process with KVM.
  9. I get a lot closer. The virtual machine can start and reboot successfully. But 2 CPUs run at 100% the entire time, even when the guest VM isn’t doing anything. I read bug reports and see that this is supposedly a problem with very old versions of KVM, but was fixed long before my current version (version 60). Virtual machine performance is terrible.
  10. I try a workaround I found via Google, installing the “Standard PC” driver instead of the “ACPI Multiprocessor PC” driver in Windows. This doesn’t help.
  11. I try allocating only one processor to the VM. This causes Windows to hang on bootup. Going back to 2 CPUs keeps the hang.
  12. I’m frustrated, so I re-install Windows from scratch with one CPU. This makes performance somewhat better.
  13. I discover that I’m not using a bridged network, but an emulated private network with NAT.
  14. I start to read docs about how to set up a bridged network with KVM/QEMU. They all involve launching KVM from a commandline, not from the GUI. And when I try to follow the directions, my local network goes down.
  15. I finally give up using the Virtual Machine Manager GUI. I switch to using the qemu-kvm command line, and get a virtual machine running using that. I can get a bridge network, maybe, but it doesn’t actually work in the guest VM.
  16. I discover that an SDL window in the host is a lot more reliable than VNC.
  17. I discover that the Windows CD I have is an OEM version and isn’t legal on this machine.

    I’m still frustrated. I now have a Windows Vista license that is legal on this machine, but I haven’t installed it yet: I really want a bridged network, and preferably one that comes up automatically every time the host machine starts. I don’t really know much about Linux networking, or even what “a network” means in Linux… I just want a simple option like I used to have in VMWare: “Use bridged network” and it all works… but with better performance because KVM and Xen can both use fully virtualized I/O and multiple CPUs.

    Does anyone know of a good guide to getting this working in a permanent sort of way?

Has GCC Dehydra Replaced Elsa?

Sunday, March 9th, 2008

No.

GCC Dehydra allows us to do analysis passes on our existing code. In the far future it may also allow us to do optimization passes. But it does not have the ability to do code rewriting, and probably won’t gain that ability any time soon. In order to be able to do C++->C++ transformations, you have to have correct end positions for all statements/tokens, not just beginning positions. GCC does not track this information, and making it do so would be a massive undertaking.

Mozilla2 still lives in a dual world where automatic code refactoring is done using the elsa/oink toolchain, while static analysis is taking place using the GCC toolchain.

Things I Learned From Writing a Mercurial Extension

Thursday, March 6th, 2008

I wrote my first mercurial extension today. It will print a log of all the commits you need to get from a changeset A which used to be the only head of a repository to a changeset B which is now the only head of the repository.It’s part of a project to get our buildbots hooked up to mercurial in a sane way. I need to make the same logic available via HTTP. It wasn’t too hard, but there are some tricky/undocumented things in the mercurial API that made life difficult.

  • The wire protocol is a pretty elegant way to ask a server for changesets when you don’t know what the graph looks like yet with relatively few queries.
  • The HTTP version of the protocol returns all the results in a plain-text format, except for the actual changesets which come across as a relatively opaque “bundle”.
  • The between command in the wire protocol doesn’t do what you think it does, or what the documentation seems to say it does. Instead of returning all the changesets between point A and point B, it walks backward through the list returning a selection of changesets along the line. The farther you go back the less often it will pick a changeset to return.
  • There is currently no way to query hgweb for metadata about a particular commit, such as the author, date, or commit message. This is trivial to add, and I will be submitting a patch shortly.
  • With a few extra functions, it should be relatively trivial to write a web repository browser which allows you to nagivate the revision graph graphically and dynamically load information about history as you need it.
  • cmdutil.show_changeset can be used to show changesets the same way hg log does. I haven’t yet figured out how to expose all of the formatting options it accepts as options accepted by my extension.

Statically Checking the Mozilla Codebase

Wednesday, March 5th, 2008

In the header file that declares class nsAutoString, there is an important comment: “Do not allocate this class on the heap”. This rule, buried deep in a header file that almost nobody reads, is a small example of a problem that plagues the Mozilla codebase: it’s easy to write incorrect code.

Mozilla, and XPCOM in particular, uses a meta-language on top of C++. This meta-language of helper classes and typesafe templates allows experienced XPCOM coders to avoid some of the complexities of XPCOM refcounting and memory management. Unfortunately, it is possible, even easy, to use this meta-language incorrectly or inefficiently. The following code, while correct C++, is incorrect “Mozilla C++”:

class Foo
{
private:
  nsAutoString mStr;
  ...
};

void Bad()
{
  Foo *foo = new Foo(); // allocated nsAutoString on the heap!
}

Errors such as this are especially insidious because the code works correctly; it is merely an inefficient use of memory which can add up quickly.

Taras and David have been working on a solution which will allow application-specific rules such as “only allocate nsAutoString on the stack” to be enforced at compile-time. It is called Dehydra GCC. It is a tool which translates the internal GCC representation of C++ code into a JavaScript object model, and allows application authors to write analysis passes as scripts.

This past week I hooked up Dehydra to the Mozilla build system. It is now possible to configure --with-static-checking=/path/to/gcc_dehydra.so and Dehydra will enforce a few basic rules: classes annotated with NS_STACK_CLASS may only be allocated on the stack, and classes annotated with NS_FINAL_CLASS may not be subclassed. For an example of a more complicated analysis, see bug 420933, ensuring that XPCOM methods handle outparams correctly.

Dehydra currently only works on Linux, and building it is a bit complicated: a custom patched GCC is required, as well as a spidermonkey package. Complete directions are available on the Mozilla Developer Center.

Dehydra is still a work in progress. We are working to complete the following tasks:

We are actively looking for hackers to help out with this project. There are many different kinds of tasks people can help with:

  • Write analysis passes to check for common bad-code patterns in Mozilla or other projects
  • Implement a webtool that allows users to browse code exactly
  • Help package up GCC-dehydra in RPMs and .deb packages for easy installation into Linux distros

I am very excited about the prospect of using static checking tools such as this one to improve our code quality and development cycle, and I’m looking forward to new and unexpected uses for this code! In future posts I will cover basics of writing a dehydra script.

Splitting a Changeset/Patch

Wednesday, February 13th, 2008

I have a common problem with revision control:

  1. Clone a tree
  2. Start working on a problem X (say, adding valgrind annotations to MMgc) with edits to GC.cpp
  3. Get distracted and start working on problem Y (say, MMgc crashing) with edits to GC.cpp
  4. I now have unrelated changes in GC.cpp that I’d like to separate.

Dear lazyweb, is there a tool I can use to interactively separate out these changes into two changesets? The solution should work with my mercurial workflow, no “use git-rebase” comments please. I happen to be using patch queues, but a solution that worked on real mercurial changesets or on raw patches would also be acceptable. It happens that usually the patch hunks for problem X and problem Y are completely separate, so a simple tool that let me throw “this hunk for X, this hunk for Y” would probably be ok. Better would be something like a three-way merge tool where I can edit an intermediate state between a fixed start-point and end-point.

Text measurement and SVG

Thursday, January 24th, 2008

In my spare time I’m working on long-needed program for engraving Gregorian chant. (“Engraving” is to music what “typesetting” is for text.) Below are the requirements I’ve been designing around:

  • Produces print-quality output from a text input format
  • in a format that can be embedded into common desktop publishing solutions like Adobe InDesign
  • Multi-platform (either web-based or runs on Win/Mac/Linux)

My current plan is to write the app in JS, provide output in SVG, and deploy over the web. I have a mockup/prototype of the SVG I’d like to produce:

Gregorian chant in SVG. Your browser doesn’t support SVG; please Get Firefox!

There is one major hurdle I haven’t figured out yet: I don’t know how to position the text. The positioning of Gregorian chant is very precise: the first neume (note) of each syllable is placed over the center of the vowel of the syllable. In addition, the spacing of neumes is arranged so that there are precise amounts of whitespace between each element and syllable, and hyphens between syllables are lengthened or shortened as necessary to fit. What is the best way to give a web application precise text measurement of individual letters within a syllable (including any kerning and ligatures)?

Ideally I’d like a way to do this with the user’s own fonts, but if I need to preprocess a font file to get metrics out of it, I guess that’s ok too.

Dear lazyweb, can somebody verify that Adobe InDesign/Quark/your favorite DTP program imports my sample SVG file correctly and usably? I don’t have a copy at the moment, and it’s rather expensive just for an import test…