Can’t Log Into Precise? Here’s Help!

So apparently I helped break the login screen for a lot of non-English speakers in Ubuntu 12.04 recently. Sorry!

The basic symptom is that regardless of your system keyboard layout, you would get a “us” layout. Which likely meant you couldn’t enter your password correctly.

To work around this, change your keyboard layout to be correct in the upper right keyboard indicator in the login screen.

I’ve pushed a fixed version of unity-greeter to the archive (0.2.0-0ubuntu4). So once you do manage to log in and update your system, you shouldn’t hit this problem next login.

Vala Autotool Tricks

If you have a project that uses Vala and autotools, you are probably using automake 1.11’s native support for the language.

Overall, automake’s support is great. But there are a few oddities that I’ve worked around and thought I’d share here.

Declaring a preferred valac version

Let’s say your project expects valac-0.14. But sometimes people have more than one valac installed, and valac-0.14 may not be their default valac compiler. The AM_PROG_VALAC macro only looks for ‘valac’ in PATH, so you’ll end up with whatever default compiler the user has.

Instead, create an acinclude.m4 file in your project directory. Copy the AM_PROG_VALAC macro from /usr/share/aclocal-1.11/vala.m4 into it. Then:

  1. Change the name to something like MY_PROG_VALAC
  2. Change the AC_PATH_PROG line to something like:
    AC_PATH_PROGS([VALAC], [valac-0.14 valac], [])
  3. Change the AM_PROG_VALAC call in configure.ac to be MY_PROG_VALAC instead

Now configure will first look for the ‘valac-0.14’ executable and then fallback to plain old ‘valac’.

Stop shipping C code

For a long time, I shipped the valac-generated C code with my release tarballs because I did not want distributors to have to deal with valac churn. But nowadays, especially with valac’s six month release cycle, things are calmer.

Not shipping the C code makes tarballs smaller (over 20% in my case), makes it easier for distros to grab upstream patches (they don’t have to create a C patch from your Vala patch), lets you use Vala conditionals (more about that below), and means you’re not shipping something different than you yourself are using.

This only requires a simple change. In each source directory that contains Vala code, add the following bits to its Makefile.am (for the purposes of this example, your Makefile.am is creating the program foo):

  1. Separate out all your .vala code files into a separate variable called foo_VALASOURCES. If you want, you can then use this variable in your foo_SOURCES variable, so you don’t have to specify them twice.
  2. Add the following dist-hook:
    dist-hook:
    	cd $(distdir) && \
    	rm $(foo_VALASOURCES:.vala=.c) foo_vala.stamp

This will delete the generated C code and stamp file when creating a dist tarball.

Use Vala conditionals

One problem with shipping the generated C code is that you can’t use #ifdefs in your code. This is because Vala does not have a version of #ifdef that will “fall through” to the C code. Whether a conditional block of code gets used or not is always decided at valac compilation time.

And since your tarball consumers were compiling the C code, you couldn’t use Vala conditionals. To have an optional project dependency, you had to write little C wrapper functions that did use real C #ifdefs and then call them from Vala. Which is a pain.

But now that you’re shipping straight Vala code, all compilers of your program can now have Vala conditionals decided at compilation time.

Let’s say you have an optional dependency on libunity:

  1. In your configure.ac, find the block of code that deals with having successfully detected libunity and add the following line:
    AC_SUBST(UNITY_VALAFLAGS, ["--pkg unity -D HAVE_UNITY"])
  2. In your Makefile.am files, add $(UNITY_VALAFLAGS) to AM_VALAFLAGS
  3. In your Makefile.am files, add a new dependency to each of your stamp files:
    foo_vala.stamp: $(top_srcdir)/config.h
  4. In your Vala code itself, you can now surround your libunity-using code blocks with #ifs:
    #if HAVE_UNITY
    ...
    #endif

The first step declares the HAVE_UNITY symbol if we’re using libunity. If we’re not, that line won’t ever be reached and UNITY_VALAFLAGS will be empty. The only other interesting bit is step three, which ensures that when ./configure is re-run, valac will also be re-run to pick up any new symbol definitions.

Backups and Distro Upgrading

tl;dr; I don’t recommend using Déjà Dup to hold your data when you upgrade distros (e.g. from Ubuntu 11.04 to 11.10) without understanding the risks.

I’m the maintainer of the Déjà Dup backup tool that will be included by default in Ubuntu 11.10. So I’m generally biased in its favor. But I am also a cautious person.

My concern stems from the fact that Déjà Dup uses an opaque backup format [1]. Which means that it does not store your data in plain files that you can just copy back into place with the file manager. You’ll need to use Déjà Dup again to restore them [2]. Which is fine if Déjà Dup is working correctly, as it should.

But just from a risk management perspective, I always recommend that people try to have at least one copy of their data in “plain files” format at all times.

So if you back up with Déjà Dup, then wipe your disk to put Ubuntu 11.10 on there, you’re temporarily going down to zero “plain file” copies of your data. And if anything should go wrong with Déjà Dup, you’ll be very sad.

Here are a few recommended ways to upgrade:

  • Use the Ubuntu CD’s built-in upgrade support. It will leave your personal files alone, but upgrade the rest of the system.
  • Use the Update Manager to upgrade your machine. Again, this will leave all your personal files in place.
  • Copy your files to an external hard drive with your file manager and copy them back after install.

In my mind, a backup system’s primary use case is disaster recovery, where going down to zero “plain file” copies of your data is unintentional and an acceptable risk. Intentionally reducing yourself to zero copies seems unnecessary.

Hopefully, all this caution is overblown. I just want people to be aware of the risks.

[1] Déjà Dup uses an opaque format to support a feature set that just can’t be done with plain files:

  • Encryption
  • Compression
  • Incremental backups
  • Assuming little about the backup location (allowing cloud backups)
  • Supporting file permissions, even when backing up to locations that don’t

[2] There are technically ways to recover without going through the Déjà Dup interface. They just aren’t very user-friendly.

Why I Ubuntu

I thought I’d share my source of motivation for working on Ubuntu in the hopes that it might inspire others too! (Note that this is my personal opinion; I don’t speak for Canonical in this blog.)

I believe that the world is moving in the direction of widespread technical savvy and ubiquitous technical companions (think smartphones or tablets).

I want such technology to be a force of empowerment, and thereby a force of good for humanity. To me, empowering means cheap, trustable, adaptable, and easy to use.

Open source software trivially fulfills the first three, in ways that proprietary software cannot. And Ubuntu is doing it’s best to fulfill the last.

Cheap

Yes, technically Open Source doesn’t have to be free. But in practice, it is. Proprietary software can be too, usually through ads. But Open Source always is and, more importantly, always will be.

Trustable

With Open Source, it’s easy to have complete confidence even without being an engineer. As a consumer, Open Source means it won’t spy on me and will do only what it says on the tin.

Adaptable

Proprietary software can be adaptable, no question. Not all adaptations require source access, though it does help.

But importantly, Open Source reduces the opportunity cost for creating or even using software. You can stand on the shoulders of giants. Anyone can provide support or contract work.

There is no barrier to entry to modify Open Source software. And most such software is designed specifically to interoperate with other software, so it’s easier to mix and match.

By way of example, think of a fictional school system in India that wants to create a customized version of Ubuntu. Very trivial to do, and they don’t need to seek anyone’s permission.

Ease of Use

This last point has generally eluded the open source community. But it’s the most important in my mind. What good is the most capable technology in the world, if it never improves lives because it is so hard to use?

My goal is empowering, in a utilitarian sense, as many users as possible. I rarely write code for me alone.

This, more than the rest, is why I support Ubuntu. Ubuntu specifically focuses on users and expanding Open Source beyond the chasm. And more than that, Ubuntu has the best chance of all the current efforts to cross the chasm.

Conclusion

When I think of empowering, I don’t tend to dwell on the modern first world. They don’t especially need empowerment. I’m thinking of the less-franchised or even our own sci-fi future, when our relationship with technology becomes even more important. Do you think Geordi would run code on the Enterprise for which he doesn’t have source access?

Also note that this is not a moral argument; I don’t especially consider Open Source a moral directive for these purposes. Users won’t flock to us because Ubuntu is open source, but rather because Ubuntu delights them.

I understand why people work on splinter efforts or other projects, but for me, I think the work that Canonical does with pre-installs, enterprise support, for-purchase apps, Ubuntu One, and user testing is an invaluable addition to the main Ubuntu project. These are how we reach new users.

OnlyShowIn Unity

tl;dr; version: Add Unity to your OnlyShowIn keys.

This is just a PSA that I’ve been doing work in Ubuntu to support Unity’s addition to the set of registered XDG environments.

Which means that in your desktop file, you can now say OnlyShowIn=Unity; to have an application only show up or only autostart in Unity instead of KDE, GNOME, etc. (Or alternatively, use NotShowIn=Unity;)

By and large, Unity uses the same apps and services that GNOME does. So if you currently have an application that does OnlyShowIn=GNOME;, there’s a very good chance it should now be OnlyShowIn=GNOME;Unity;. You might also want to re-evaluate whether you even need OnlyShowIn.

I’ve filed a bunch of bugs against various GNOME apps and patched them in Ubuntu. But I’ve only focused on apps that Ubuntu installs by default.

Using Ubuntu One’s Cloud API

I recently finished up a branch adding support to the backup program duplicity for Ubuntu One. I thought I’d write up how I did it, because some things weren’t obvious.

My needs were simple: Python versions of get, put, delete, and list operations.

Code snippets under GPL-2+ license. These don’t have any error handling, but that’s simple enough to add. Note that for the list and put snippets you will need to either use trunk or a release greater than 0.2.0 of the ubuntuone-couch Python module.

If parts of the snippets are confusing, it may help to consult the official documentation.

Log In

First, you have to log the user into Ubuntu One. There’s a utility class in ubuntuone.platform.credentials to help with this. It’s designed to be asynchronous, but for our simple purposes, we’ll fake synchronicity with a main loop.

_login_success = False
def login(self):
    from gobject import MainLoop
    from dbus.mainloop.glib import DBusGMainLoop
    from ubuntuone.platform.credentials import CredentialsManagementTool

    global _login_success
    _login_success = False

    DBusGMainLoop(set_as_default=True)
    loop = MainLoop()

    def quit(result):
            global _login_success
	    loop.quit()
	    if result:
		    _login_success = True

    cd = CredentialsManagementTool()
    d = cd.login()
    d.addCallbacks(quit)
    loop.run()
    return _login_success

Create Volume

Another piece of set up before we can get to the good stuff is to create a volume if needed. A volume is just a directory in Ubuntu One that can be synchronized (but isn’t by default). It’s OK to attempt to create a volume multiple times, but Ubuntu One will report back an error if you try to create a nested volume.

Note the auth.request call. This adds the required OAuth header to our request so that we can successfully authorize with the server. This is why we must be logged in first, to have access to the credentials that auth.request uses.

def create_volume(path):
    import ubuntuone.couch.auth as auth
    import urllib
    base = "https://one.ubuntu.com/api/file_storage/v1/volumes/~/"
    return auth.request(base + urllib.quote(path), http_method="PUT")

Delete

Alright, we can get down to brass tacks now that we are logged in and have a volume in which to work. Let’s start simple with a file delete request.

def delete(path):
    import ubuntuone.couch.auth as auth
    import urllib
    base = "https://one.ubuntu.com/api/file_storage/v1/~/"
    return auth.request(base + urllib.quote(path), http_method="DELETE")

That was easy!

List

Listing involves requesting the metadata for the parent directory / volume and explicitly asking for information about its children.

def list(path):
    import json
    import ubuntuone.couch.auth as auth
    import urllib
    base = "https://one.ubuntu.com/api/file_storage/v1/~/"
    url = base + urllib.quote(path) + "?include_children=true"
    answer = auth.request(url)
    filelist = []
    node = json.loads(answer[1])
    if node.get('has_children') == True:
        for child in node.get('children'):
            child_path = urllib.unquote(child.get('path')).lstrip('/')
            filelist += [child_path]
    return filelist

Get

This is a little tricky. You have to first ask for metadata about the file to get the actual content path. Then you have to hit files.one.ubuntu.com to get the data itself.

Note we don’t quote the twiddle in the content_path field. It needs to be unquoted.

def get(remote, local):
    import json
    import ubuntuone.couch.auth as auth
    import urllib
    base = "https://one.ubuntu.com/api/file_storage/v1/~/"
    answer = auth.request(base + urllib.quote(remote))
    node = json.loads(answer[1])
    base = "https://files.one.ubuntu.com"
    url = base + urllib.quote(node.get('content_path'), safe="~")
    answer = auth.request(url)
    f = open(local, 'wb')
    f.write(answer[1])

Put

Last, but not least, putting a file on the server. Note that you must specify a content type and length.

def put(local, remote):
    import json
    import ubuntuone.couch.auth as auth
    import mimetypes
    import urllib
    base = "https://one.ubuntu.com/api/file_storage/v1/~/"
    answer = auth.request(base + urllib.quote(remote),
                          http_method="PUT",
                          request_body='{"kind":"file"}')
    node = json.loads(answer[1])
    data = bytearray(open(local, 'rb').read())
    size = len(data)
    content_type = mimetypes.guess_type(local)[0]
    content_type = content_type or 'application/octet-stream'
    headers = {"Content-Length": str(size),
	       "Content-Type": content_type}
    base = "https://files.one.ubuntu.com"
    url = base + urllib.quote(node.get('content_path'), safe="~")
    return auth.request(url, http_method="PUT",
                        headers=headers, request_body=data)

Odds and Ends

Hope that was helpful!

You should be able to figure out how to do other things you’ll need from these snippets and the official documentation. Note that as of this writing, parts of that documentation are out of date. For the moment, trust my snippets over the documentation! I’m told the documentation is being updated.

Note that in terms of HTTP error codes, Ubuntu One seems to return 500 for out-of-space. And if you request a malformed path (like, with two slashes in it), it likes to give back an error code with a login page as an HTML payload.

Déjà Dup in Ubuntu 11.10

So the result of the UDS session on including Déjà Dup in Ubuntu 11.10 was positive. Assuming no disasters, it will be on the CD!

I’d like to thank Canonical for sponsoring development by providing me work time to develop features and squash bugs!

I want to try to reduce the Ubuntu bug flood by really nailing things down this cycle. So I’d request that Ubuntu users try it and report bugs before the flood gates open. Especially test restoring too, as any bugs with restores are generally critical issues.

Yay for Open Source!

So I’ve avoided talking about the latest kerfuffle in Ubuntuland because I might be biased, working as I do for Canonical. But here’s a bit of musing inspired by it.

Progress

I’d like to focus a bit on how awesome the world is. Open source software is becoming a common and accepted tool. A person anywhere in the world can download and install for free a distribution like Ubuntu or Fedora and have a first class experience. It will come packed with powerful software like Firefox and LibreOffice.

You can provision a virtual server from someone like Amazon, fill it with services like a web server, a code repository, you name it. You can go from idea to website in seconds flat.

Need to look something up? Just ask Wikipedia, the rapidly enlarging sum of human knowledge. Want to buy a phone? Pick up a Linux-based Android smartphone.

If you step back a bit, that is amazing. Open source is empowering the world, literally making it a better place. The best (and defining) part is that each advancement can build upon the previous ones.

Design

But the best technical design with the most features isn’t going to change the world — or even just be as helpful as it can be — unless it’s easy and fun to use. That’s why I love that design-driven development is gaining more cachet in open source. I’m glad to see the efforts by Ubuntu’s Ayatana team and GNOME’s design team to cross the chasm and reach new users.

Motivation

Anyway, this is all part of why I work in the open source sphere. Not just to scratch my own itches, but because I believe I can use my skills to leave a lasting mark and to empower others. I’m incredibly blessed to be able to work in such communities and with such companies.

I just find it hard to get worked up over whether buttons are on the left or the right, whether the user can minimize a window, or whether GNOME and Canonical are getting along 70% or 90%.

Not that such discussions aren’t worth having, but it’s easy to get caught up in navel-gazing. We’re all on the same team. We’re trying to change the world, and it’s worth remembering every now and then that we are succeeding!

Now On the Desktop Team

I know I just recently posted about how awesome working in the OEM Services department was, but in the eternal quest for new challenges, I’ve moved over to the Desktop team at Canonical.

But before I get to that, one quick bonus story about a patch that I wrote in OEM Services that I really like. You know how when you plug in headphones with a standard audio jack that the audio immediately switches to headphones only? Nice feature right? Now plug in your USB headset. You’ll notice that it didn’t play through the headset: you have to go to the sound preferences tab and choose it as your output device, which is not intuitive. Same deal with bluetooth headsets.

From the user’s perspective, they should all work the same. They’re all just headsets. So for an OEM project, I wrote a pulseaudio module that tried to do just that: whenever a new pulseaudio sink or source became connected, it would switch to using it. It may need some polish (I’m no pulseaudio maintainer), but it works fine. I’d love to see it applied upstream.

Anyway, so I’m switching to the Desktop team to help work on Quickly and other application developer goodness. Speaking of which if you’re a developer, I have a session tomorrow all about release management as part of the App Developer Week: What’s the deal with time based vs feature based releases? How do I best use these Series and Milestone things in Launchpad? Find out tomorrow in #ubuntu-classroom at 14:00 UTC!