Déjà Dup Was Worth It

In the few years since I started writing Déjà Dup, I’ve never actually had to use it to recover data from a backup. But now I’m not only the maintainer, I’m also a client!

The other day I accidentally erased the hard drive of my main machine but was able to recover the good bits from my S3 backup. Yay!

In other Déjà Dup news, the new stable 16.0 was just released (available in fine distros that ship GNOME 2.32). This is mostly a bug fix release, but with one nice new feature: if you are running out of space in your backup location, it will delete older backups until there is enough space rather than just yelling at you.

Tracking Software Versions Between Distros

A neat web site that I keep meaning to mention is Open Source Watershed. It’s useful for when you want to see what version of a package is in all the various FOSS distributions.

For example, you can see what versions of wine the various distros use along with some stats about version churn. It’d be nice if they used distro codenames on package pages instead of just saying “ubuntu current” vs “ubuntu future”, but still handy.

What I Do in OEM Services

I realize that I’ve never really blogged about my work at Canonical. Largely because I work in the OEM Services department, and we tend to be hush-hush. OEM means Original Equipment Manufacturer and just means a company that makes and sells computers with their own brand (like HP, Dell, Lenovo). OEM Services is the group that works with OEMs that are interested in preloading Ubuntu. Anyway, with everyone else on the planet talking about their work, it is a good time to share what I actually do and why.

I feel that making it easy to buy a machine with Linux preloaded is the most important task facing the wider FOSS community. This is how we grow the community, how we break into the mainstream.

But working with OEMs also brings more immediate benefits. When working on a specific project, both the OEM and Canonical will have QA resources dedicated to finding bad bugs or usability issues.

OEMs tend to approach QA with a subtly different eye than I do as an independent developer. They tend to try to avoid any situation where a customer will request support, as that costs money. Which means trying to avoid loss of functionality or letting the user shoot themselves in the foot.

But another thing that comes out of such attention is a focus on usability bugs. It feels really good to be able to spend time making a patch for such nits and send it upstream.

For example, here is a random sampling of stuff that I remember working on:

  • Back during the beginnings of a netbook version of Ubuntu, I helped write lots of little patches for applications to make them fit on smaller screens. OEMs would understandably hate it when a program wouldn’t fit on the screen because users wouldn’t be able to click on the dialog buttons and it just looked awful.
  • The dialog prompt for PolicyKit used to always display names like “Michael Terry,,,” which always struck me as an eyesore. It showed that way because it didn’t strip the commas from the information that getpwnam() returned.
  • When the user sets an unusual “how long to idle before blanking the screen” value, that value just didn’t show up at all in the GNOME Power Preferences dialog. This usually doesn’t come up in practice, but arose because an OEM wanted a 15 min delay, which is not a value already in the list.
  • The first time you opened rhythmbox and tried to import a folder, you got a scary “operation not supported” error message. Trying again worked fine. It was a simple fix, but one of those things that you rarely run into and if you do, just kind of shrug off and try again. But it’s nice to be able to fix such first-impressions papercuts.
  • A crash in pidgin if you used a zephyr account, enabled tzc, and didn’t have tzc installed. Not only did it crash pidgin, but X itself! This is one of my favorites because it shows just how useful an OEM’s QA approach to testing is. They really “test to break,” and exercise parts of the systems that don’t get much use. This crash was not triggered by a common use case, but had a really bad effect.

So none of these are earth-shattering fixes or cool new features. Mostly because by the time the software gets to me, it’s already as part of an Ubuntu release and thus mostly functional. I just tend to nibble around the edges. There’s a large untapped market of FOSS users that will never install an OS, and I get warm fuzzies by helping to reach them.

Lost: Minimum Requirements Script

A couple years ago, I remember reading about a script that would scan a source tree and tell you the minimum required version of GTK+ based on the symbols used. I can’t find it anymore. Does anyone know what I’m talking about?

I assume it could be rewritten these days using sexy gobject introspection and thus apply to more than just GTK+. But I’d be happy with the older, basic script.

In exchange for clues about the script, here’s a neat bash tip: I discovered that if you put a space before a command you type on the console, it doesn’t show up in the bash history. It’s not in the output of history and doesn’t show up when you scroll up through your commands with the up arrow.

I’m not sure what the non-malicious uses of that trick are, but there you go.

Translate Your Documentation

As a maintainer, I’ve always been a little sad that my man pages were not translated. Nor where my user help documentation (docbook files). They weren’t translated mainly because:

  • intltool doesn’t support those formats (well, intltool does xml, but not specifically docbook) and
  • the usage is slightly different — help docs need to be generated from po files, but normally those po files are just installed onto the system

Traditionally, projects like GNOME have translators handle docbooks separately from the rest of translations. Partly because of the problems above and partly because docbooks often have images which must be recreated in the correct locale. Gettext just isn’t equipped for that.

But I’m willing to concede the screenshot point. I’m OK with user docs that don’t have screenshots for my simple apps. After all, they’re usually just used to walk through the UI that the user can already see. They don’t generally explain anything.

So, how to integrate documentations to my existing gettext system?

Desired Solution

First, let me briefly review what I want the end product to look like:

  • I don’t want the documentation translations to end up installed on users’ systems in the compiled .mo files. That just wastes lots of space.
  • But I want all the translations to be in the same domain. Multiple domains is a pain for both translators and maintainers. Translators have to go translate two files, not one (I suspect most translators won’t bother-to-translate/notice the other domain), and the domains will have duplicated strings (meaning translators will duplicate work). This means one .pot file and consequently one .po file for each language. This would appear at odds with the previous desire.
  • Man pages and docbook are funky formats. There’s all sorts of syntax that makes breaking the content into logical paragraphs or phrases difficult. I want a tool that can handle that while spitting out gettext files. Intltool’s xml module, for example, doesn’t have enough docbook-specific smarts to make that happen, although it does integrate nicely with gettext.

Enter po4a

There’s a program called po4a that can handle some of the more unusual formats well while consuming po files and generating pot files. It does a really great job, but it has a few shortcomings:

  • it likes to own the .po files
  • it likes to own the .pot file
  • it uses its own list of languages (not po/LINGUAS)

All of those shortcomings will make it hard to have one pot file.

Enter Lots of Makefile Magic

I stole the original bits of Makefile goodness for po4a from dpkg, which used it for its man pages. But I added a bit more to meet my desired goals above.

Basically the sequence is as follows when making a distribution tarball (make dist):

  1. Copy all the .po files from po/ to the help/ directory
  2. Import the contents of po/LINGUAS to po4a’s configuration file in the help directory
  3. Run po4a, generating compiled, translated help documentation from the base versions and the copied po files. This will generate a .pot file.
  4. Filter all the po/*.po files through the standard .pot file, dropping all their help documentation translations. This is so we don’t distribute space-wasting translations that we already ship in the compiled version above.
  5. Merge the generated help .pot file with the standard .pot file in po/ for shipping and distributing to translators.

Ta-da! It’s a bit convoluted, but means that I can have all the benefits of translated help documentation, with none of the disadvantages of multiple .pot files. Of course, I more than doubled my string count, but I assume translators prefer better coverage than not.

The code is mostly in Déjà Dup‘s help/Makefile.am, help/po4a.cfg, and Makefile.am. I encourage you to steal it and use it for your own projects. Too few man pages are translated.

If there are better tools or more elegant methods, I’d love to hear them!

Ubuntu Distribution

When I recently joined the Ubuntu Members team, I was surprised to note the user map:

many flags in North America and Europe, none elsewhere

Seems a bit more concentrated than I expected. I guess the UDS policy of alternating Europe and America locations is more fair than I supposed. (Though in truth, that map suggests it should always be in Europe.)

I know we have rocking LoCo and translation teams pretty much everywhere. As well as developers. Just not many that are both official members and willing to share their latitude and longitude?

Update: So apparently the real map is much more inclusive. The one on the team front page is some optimized subset to make rendering faster. False alarm! Although the algorithm could use tweaking to get a nicer spread.

Turkish Is Crazy

So I was working on a very odd bug at work. For some reason, printing didn’t work when you chose Turkish as your language.

After some digging, I discovered that Turkish is very odd in one particular aspect. The wikipedia entry about it can explain better than me, but suffice to say:

$ LANG=en_US.UTF-8 python -c "import locale; locale.setlocale(locale.LC_ALL, ''); print 'TURKISH'.lower()"
turkish
$ LANG=tr_TR.UTF-8 python -c "import locale; locale.setlocale(locale.LC_ALL, ''); print 'TURKISH'.lower()"
turkIsh

This has implications for all sorts of sloppily written code. Normally it works alright, if all you care about is if two strings are the same thing when lower‘ed, but if you are using lower as part of a hash function and then talking over a network, everything will screw up. I’m surprised anything works in Turkish.

Dogtail and LDTP

So I was looking into an automated test framework for Déjà Dup and naturally, I tried the current contenders: Dogtail and LDTP.

I have to say, I liked Dogtail’s Python interface more, but it kept crashing my at-spi daemon, meaning I had to log out and log back in again. That was enough to keep me disinterested.

So I tried LDTP. It was just fine in terms of robustness (modulo a bug with its UTF-8 support). So I’ve been cooking up a kick-ass test suite framework for Déjà Dup.

The one big problem is that GtkAssistant accessibility support seems broken. It just doesn’t expose the widgets correctly. Which means I can’t automate much of Déjà Dup. Does anyone know atk well? Please fix my bug. :)