Déjà Dup Moves to GNOME’s GitLab

Déjà Dup has happily hosted it’s main project page on Launchpad for the past ten years.

But in the interest of growing closer to GNOME and for the more modern UI & features that GitLab sports, I’m moving the project’s code hosting and downloads to GNOME’s GitLab.

Bugs and translations will remain at Launchpad for now. No change there. Though folks are welcome to also file issues on the GitLab page.

This does not mean Déjà Dup is part of the GNOME project! GNOME is just letting us live on their infrastructure as a “World” module — a project that orbits GNOME but is not part of it.

Backups and Distro Upgrading

tl;dr; I don’t recommend using Déjà Dup to hold your data when you upgrade distros (e.g. from Ubuntu 11.04 to 11.10) without understanding the risks.

I’m the maintainer of the Déjà Dup backup tool that will be included by default in Ubuntu 11.10. So I’m generally biased in its favor. But I am also a cautious person.

My concern stems from the fact that Déjà Dup uses an opaque backup format [1]. Which means that it does not store your data in plain files that you can just copy back into place with the file manager. You’ll need to use Déjà Dup again to restore them [2]. Which is fine if Déjà Dup is working correctly, as it should.

But just from a risk management perspective, I always recommend that people try to have at least one copy of their data in “plain files” format at all times.

So if you back up with Déjà Dup, then wipe your disk to put Ubuntu 11.10 on there, you’re temporarily going down to zero “plain file” copies of your data. And if anything should go wrong with Déjà Dup, you’ll be very sad.

Here are a few recommended ways to upgrade:

  • Use the Ubuntu CD’s built-in upgrade support. It will leave your personal files alone, but upgrade the rest of the system.
  • Use the Update Manager to upgrade your machine. Again, this will leave all your personal files in place.
  • Copy your files to an external hard drive with your file manager and copy them back after install.

In my mind, a backup system’s primary use case is disaster recovery, where going down to zero “plain file” copies of your data is unintentional and an acceptable risk. Intentionally reducing yourself to zero copies seems unnecessary.

Hopefully, all this caution is overblown. I just want people to be aware of the risks.

[1] Déjà Dup uses an opaque format to support a feature set that just can’t be done with plain files:

  • Encryption
  • Compression
  • Incremental backups
  • Assuming little about the backup location (allowing cloud backups)
  • Supporting file permissions, even when backing up to locations that don’t

[2] There are technically ways to recover without going through the Déjà Dup interface. They just aren’t very user-friendly.

Déjà Dup in Ubuntu 11.10

So the result of the UDS session on including Déjà Dup in Ubuntu 11.10 was positive. Assuming no disasters, it will be on the CD!

I’d like to thank Canonical for sponsoring development by providing me work time to develop features and squash bugs!

I want to try to reduce the Ubuntu bug flood by really nailing things down this cycle. So I’d request that Ubuntu users try it and report bugs before the flood gates open. Especially test restoring too, as any bugs with restores are generally critical issues.

Déjà Dup Was Worth It

In the few years since I started writing Déjà Dup, I’ve never actually had to use it to recover data from a backup. But now I’m not only the maintainer, I’m also a client!

The other day I accidentally erased the hard drive of my main machine but was able to recover the good bits from my S3 backup. Yay!

In other Déjà Dup news, the new stable 16.0 was just released (available in fine distros that ship GNOME 2.32). This is mostly a bug fix release, but with one nice new feature: if you are running out of space in your backup location, it will delete older backups until there is enough space rather than just yelling at you.

Déjà Dup News

It’s been a while since I’ve blogged about Déjà Dup. Actually, looking back in my archives, I haven’t blogged specifically about Déjà Dup since it’s first 1.0 release. As it’s now on it’s tenth feature release, I guess it’s fitting to give some news.

Since 1.0, Déjà Dup has continued to rock. Here are some of the new features:

  • Scheduled backups
  • All sorts of crazy backends, like SSH, FTP, WebDav, or samba(!)
  • Restoring from any given point in time
  • A nautilus extension to restore files with a right click
  • Sexy awesome backup and restore wizards that guide a user through setting up a backup
  • All sorts of usability tweaks, bug fixes, and minor improvements

Anyway, the point is, I have not been idle.

And the project is definitely looking for help. If you can translate or write code, please let me know!

Translate Your Documentation

As a maintainer, I’ve always been a little sad that my man pages were not translated. Nor where my user help documentation (docbook files). They weren’t translated mainly because:

  • intltool doesn’t support those formats (well, intltool does xml, but not specifically docbook) and
  • the usage is slightly different — help docs need to be generated from po files, but normally those po files are just installed onto the system

Traditionally, projects like GNOME have translators handle docbooks separately from the rest of translations. Partly because of the problems above and partly because docbooks often have images which must be recreated in the correct locale. Gettext just isn’t equipped for that.

But I’m willing to concede the screenshot point. I’m OK with user docs that don’t have screenshots for my simple apps. After all, they’re usually just used to walk through the UI that the user can already see. They don’t generally explain anything.

So, how to integrate documentations to my existing gettext system?

Desired Solution

First, let me briefly review what I want the end product to look like:

  • I don’t want the documentation translations to end up installed on users’ systems in the compiled .mo files. That just wastes lots of space.
  • But I want all the translations to be in the same domain. Multiple domains is a pain for both translators and maintainers. Translators have to go translate two files, not one (I suspect most translators won’t bother-to-translate/notice the other domain), and the domains will have duplicated strings (meaning translators will duplicate work). This means one .pot file and consequently one .po file for each language. This would appear at odds with the previous desire.
  • Man pages and docbook are funky formats. There’s all sorts of syntax that makes breaking the content into logical paragraphs or phrases difficult. I want a tool that can handle that while spitting out gettext files. Intltool’s xml module, for example, doesn’t have enough docbook-specific smarts to make that happen, although it does integrate nicely with gettext.

Enter po4a

There’s a program called po4a that can handle some of the more unusual formats well while consuming po files and generating pot files. It does a really great job, but it has a few shortcomings:

  • it likes to own the .po files
  • it likes to own the .pot file
  • it uses its own list of languages (not po/LINGUAS)

All of those shortcomings will make it hard to have one pot file.

Enter Lots of Makefile Magic

I stole the original bits of Makefile goodness for po4a from dpkg, which used it for its man pages. But I added a bit more to meet my desired goals above.

Basically the sequence is as follows when making a distribution tarball (make dist):

  1. Copy all the .po files from po/ to the help/ directory
  2. Import the contents of po/LINGUAS to po4a’s configuration file in the help directory
  3. Run po4a, generating compiled, translated help documentation from the base versions and the copied po files. This will generate a .pot file.
  4. Filter all the po/*.po files through the standard .pot file, dropping all their help documentation translations. This is so we don’t distribute space-wasting translations that we already ship in the compiled version above.
  5. Merge the generated help .pot file with the standard .pot file in po/ for shipping and distributing to translators.

Ta-da! It’s a bit convoluted, but means that I can have all the benefits of translated help documentation, with none of the disadvantages of multiple .pot files. Of course, I more than doubled my string count, but I assume translators prefer better coverage than not.

The code is mostly in Déjà Dup‘s help/Makefile.am, help/po4a.cfg, and Makefile.am. I encourage you to steal it and use it for your own projects. Too few man pages are translated.

If there are better tools or more elegant methods, I’d love to hear them!

Dogtail and LDTP

So I was looking into an automated test framework for Déjà Dup and naturally, I tried the current contenders: Dogtail and LDTP.

I have to say, I liked Dogtail’s Python interface more, but it kept crashing my at-spi daemon, meaning I had to log out and log back in again. That was enough to keep me disinterested.

So I tried LDTP. It was just fine in terms of robustness (modulo a bug with its UTF-8 support). So I’ve been cooking up a kick-ass test suite framework for Déjà Dup.

The one big problem is that GtkAssistant accessibility support seems broken. It just doesn’t expose the widgets correctly. Which means I can’t automate much of Déjà Dup. Does anyone know atk well? Please fix my bug. 🙂