Unfortunately the data from the customers included some \x85 characters, which were breaking the CSV parsing.

0x85 is a control character (NEXT LINE or NEL) in Latin-1, but it's a printable character (HORIZONTAL ELLIPSIS) in Microsoft's code page 1252, which is often mistaken for Latin-1. I would venture a suggestion that the encoding of the customer data was not latin-1 but rather cp1252.

>>> '\x85'.decode('cp1252')
u'\u2026'

3 11

Fast-forward to 2008: there are actual users out there (much to my surprise), I still haven't added the originally-envisioned spit and polish, haven't done anything to foster a development community, am wracked by guilt of not doing my maintainerly duties properly, which leads to depression and burnout. So I do the only thing I can think of: run away from the project and basically ignore its existence for a year. Unreviewed patches accumulate in my inbox.

It seems that the sabbatical helped: yesterday, triggered by a new Debian bug report, I sat down, fixed the bug, implemented a feature, applied a couple of patches languishing in the bug tracker, and released version 0.3 (which was totally broken thanks to setuptools magic that suddenly stopped working; so released 0.3.1 just now). Then went through my old unread email, created bugs in Launchpad and sent replies to everyone. Except Pierre-Luc Beaudoin, since his @collabora.co.uk email address bounced. If anyone knows how to contact him, I'd appreciate a note.

There are also some older changes that I made before I emerged out of the funk and so hadn't widely announced:

• There's a mailing list for user and developer discussions (if there still are any ;).
• GTimeLog's source code now lives on Launchpad (actually, I mentioned this on my blog once).

14 2

The current working directory is my project configuration/state. I run tests here (bin/test), I search for code here (vim -t TagName, find + grep), I run applications here (make run or bin/appname). I can multitask freely, for example, if I'm in the middle of typing an SVN commit message, I can hit Ctrl+Shit+T, get a new terminal tab in the same working directory, and look something up. No aliases/environment variables/symlinks/scripts making changes to config files. I can work on multiple projects at the same time. I can work remotely (over ssh).

Gary Bernhardt's screencasts on Vimeo show how productive you can get if you learn Vim and tailor it to your needs. I have Vim scripts that let me

• See the name of the class and function that I'm editing in the statusbar, even if the class/function definition is offscreen: pythonhelper.vim.
• See all pyflakes warnings and errors in a list as soon as I press F2 to save the file: python_check_syntax.vim.
• Add a "from foo.bar import Something" line at the top of the file if I press F5 when my cursor is on Something, looking up the package and module from ctags: python-imports.vim.
• Switch between production code and unit tests with a single key if the project uses one of several conventions for tests (e.g. ./foo.py <-> ./tests/test_foo.py): py-test-switcher.vim.
• Generate a command line for running one particular unit test (the one my cursor is inside) and copy it into the system clipboard, so I can run that test by Alt-Tabbing into my terminal window and pasting. py-test-runner.vim.
• Open the right file and move the cursor to the right line if I triple-click a line of traceback in a shell (or an email) then press F7 in my gvim window: py-test-locator.vim.
• Compare my version of the code with the pristine version in source control in an interactive side-by-side diff that lets me revert bits I no longer want: vcscommand.vim.
• Highlight which lines of the source are covered by my tests, if I have coverage information in trace.py format: py-coverage-highlight.vim.
• Show the signature of a function/class's __init__ when I type the name of that class/function and an open parenthesis (looked up from tags): py-function-signature.vim.
• Fold code into an outline so I only see names of methods or classes instead of their full bodies: vimrc, function PythonFoldLevel.
• Fold diff files so I can see whole hunks/files and can delete those with a single key (well, two keys -- dd). Useful for reviewing large diffs (tens of thousands of lines): vimrc, function DiffFoldLevel.

Some of these come from www.vim.org, some I've written myself, some I've taken and modified a little bit to avoid an irritating quirk or add a missing feature. Some things I don't have (and envy Emacs or IDE users for having -- like an integrated debugger for Python apps, and, generally, integration with other tools, running in the background).

It's been my plan for a long time to polish my plugins, release them somewhere (github? bitbucket? launchpad?) and upload to vim.org, but as it doesn't seem to be happening, I thought I'd at least put an svn export of my ~/.vim on the web.

13 3

Here's a pattern I've been using in Zope 3 to display multiline user-entered text as several paragraphs:

import cgi

from zope.component import adapts
from zope.publisher.browser import BrowserView
from zope.publisher.interfaces import IRequest


class SplitToParagraphsView(BrowserView):
    """Splits a string into paragraphs via newlines."""

    adapts(None, IRequest)

    def paragraphs(self):
        if self.context is None:
            return []
        return filter(None, [s.strip() for s in self.context.splitlines()])

    def __call__(self):
        return "".join('<p>%s</p>\n' % cgi.escape(p)
                        for p in self.paragraphs())

View registration

<configure
    xmlns="http://namespaces.zope.org/zope">

  <view
      for="*"
      name="paragraphs"
      type="zope.publisher.interfaces.browser.IBrowserRequest"
      factory=".views.SplitToParagraphsView"
      permission="zope.Public"
      />

</configure>

and usage

<p tal:replace="structure object/attribute/@@paragraphs" />

Update: The view really ought to be registered twice: once for basestring and once for NoneType. I was too lazy to figure out the dotted names for those (or check if zope.interface has external interface declarations for them), so I registered it for "*". You should know that this makes the view available for arbitrary objects (but won't work for most of them, since they don't have a splitlines method), and that it is, sadly, accessible to users who may try to hack your system by typing things like @@paragraphs in the browser's address bar. Ignas Mikalajūnas offers an alternative solution using TALES path adapters.

8 4

You need a web server running Apache and SSH. Enable mod_proxy and mod_proxy_connect and add this to the first (i.e. default) virtual host configuration:

<VirtualHost whatever:80>
...

  # allow ssh to localhost over http proxy
  ProxyRequests on
  AllowCONNECT 22
  <Proxy localhost>
    Order allow,deny
    Allow from all
  </Proxy>

</VirtualHost>

On the client side you need proxytunnel. Sadly, it's not packaged for Ubuntu yet, but compiling from sources is trivial. Edit ~/.ssh/config and add an entry for your proxied ssh connection:

Host pmyservername
ProxyCommand proxytunnel -q -p myserver.mydomain.com:80 -d localhost:22

That's it. Now you can ssh pmyservername. (The p prefix is a reminder that I'm using a proxied connection: ssh fridge versus ssh pfridge. Also it reminds me of Terry Pratchett's Pyramids.).

For extra fun (e.g. IRC) use ssh's built-in SOCKS5 proxy: ssh -D 1080 pmyservername. Then tell the apps to use a SOCKS5 proxy on localhost. Since telling each app to use a proxy (and then, later, telling it to stop using it) is a big *pain*, and some apps (e.g. ssh) don't support proxies directly, a wrapper like tsocks is handy. Edit /etc/tsocks.conf and set the default socks server to 127.0.0.1, then use it to run apps:

$ tsocks xchat-gnome
$ tsocks bzr push lp:myprojectname

tsocks is packaged for Ubuntu.

If your hotel doesn't have free WiFi, a prepaid SIM card with 3G access could be cheaper than roaming charges. Apparently you can get one with a virtually unlimited (for a short stay, anyway) data plan for 27 EUR in Amsterdam.

19 0

I want a system

• that is transparent to the user: if my database is at schema version 1, and my code is at version 3, I want it to be automatically upgraded to version 3 on server startup.
• that is not too hard on the programmer: dropping a numbered Python or SQL script in a directory ought to be sufficient to define a transition from schema version X to schema version X+1.
• that handles errors gracefully: makes a backup of the database with the old schema version; runs my script in a transaction and aborts that transaction if the conversion fails (while showing me enough information to debug the problem).
• allows prototyping without having to increment the schema number for every little change I make to the models; I should be the one who decides that a new schema is ready to go out to the world.

I've been glancing at SQLAlchemy-Migrate, since I've been brought up to believe NIHing is Bad. But Migrate is scary. I have to admit that the longer I stare at its documentation, the less I can describe why I think so. All those shell commands—but there's an API for invoking them from Python, so maybe I can achieve my goals. I'll have to try and see.

3 6

I will omit bzr commits from this narrative as it's getting long; you can assume that every self-contained change was committed separately.

tests

First, I want a bin/test script to run the test suite. Pylons uses nose, so we need to tell buildout to install the nosetests script (under a different name, since I'm used to typing bin/test no matter what test runner a project happens to use):

$ bzr diff
=== modified file 'buildout.cfg'
--- buildout.cfg	2009-09-15 19:49:11 +0000
+++ buildout.cfg	2009-09-15 19:49:18 +0000
@@ -8,5 +8,8 @@
 recipe = zc.recipe.egg
 eggs = Pylons
        PasteScript
+       nose
        asharing
 interpreter = python
+scripts = paster
+          nosetests=test

$ bin/buildout
...
Generated script '/tmp/AlliterationSharing/bin/paster'.
Generated script '/tmp/AlliterationSharing/bin/test'.
...
$ bin/test

----------------------------------------------------------------------
Ran 0 tests in 0.276s

OK

ctags

Documentation is good, but sometimes you want to look at the source code of the framework. There's a tool called ctags that builds a database of identifiers. The popular text editors Vim and Emacs can then use the tags database to jump to a definition of any name with a single keystroke (Ctrl-] in vim, M-. in emacs).

Building the tags database is complicated by each Python package being installed into a separate directory. There's a buildout recipe called z3c.recipe.tag that finds those directories and lets you build a unified tags file. We'll also ask buildout to make sure it unzips any packages distributed as .egg files, since ctags doesn't process those:

$ bzr diff
@@ -1,8 +1,9 @@
 [buildout]
 develop = .
-parts = pylons
+parts = pylons ctags
 
 newest = false
+unzip = true
 
 [pylons]
 recipe = zc.recipe.egg
@@ -13,3 +14,7 @@
 interpreter = python
 scripts = paster
           nosetests=test
+
+[ctags]
+recipe = z3c.recipe.tag:tags
+eggs = ${pylons:eggs}

$ bin/buildout
...
Generated script '/tmp/AlliterationSharing/bin/ctags'.
...
$ bin/ctags

omelette

ctags lets you find classes and functions by name; it doesn't let you find packages or modules. There's another recipe, collective.recipe.omelette that creates a tree of symlinks mirroring the Python package structure (here 'unzip = true' also comes in handy):

$ bzr diff
=== modified file 'buildout.cfg'
--- buildout.cfg	2009-09-15 20:04:42 +0000
+++ buildout.cfg	2009-09-15 20:05:30 +0000
@@ -1,6 +1,6 @@
 [buildout]
 develop = .
-parts = pylons ctags
+parts = pylons ctags omelette
 
 newest = false
 unzip = true
@@ -18,3 +18,7 @@
 [ctags]
 recipe = z3c.recipe.tag:tags
 eggs = ${pylons:eggs}
+
+[omelette]
+recipe = collective.recipe.omelette
+eggs = ${pylons:eggs}

$ bin/buildout 
...
$ ls -l parts/omelette
...

The symlink tree is created under parts/omelette/. For example, if you want to see what webhelper tags were available, you can open parts/omelette/webhelper/html/builder.py in your editor and see.

Makefile

This is getting long (and not everyone may be interested¹), but one long post is easier to skip than five medium ones in a row, so I'll continue.

¹ Sorry, Planet Maemo! There's an RSS feed of posts tagged 'maemo', if you can figure out the URL, which is very well hidden by PyBlosxom, *sigh*.

Wouldn't it be nice if new developers could check out your project and start it up with just a couple of commands? Make is a time-tested tool that works well for this:

$ cat Makefile
# Just remember that you need to use real tabs, not spaces, in a Makefile

PYTHON = python

.PHONY: all
all: bin/paster

.PHONY: run
run: bin/paster
        bin/paster serve development.ini --reload

.PHONY: test check
test check: bin/test
        bin/test

.PHONY: tags
tags: bin/ctags
        bin/ctags

bin/paster bin/test bin/python bin/ctags: bin/buildout
        bin/buildout

bin/buildout: bootstrap.py
        $(PYTHON) bootstrap.py

Now all you need to do after checking out is run 'make' to set up a working development environment. 'make run' or 'make test' will also do that, if necessary, so this one-liner is sufficient to get a working Hello World application on port 5000:

$ bzr branch lp:~mgedmin/+junk/AlliterationSharing && cd AlliterationSharing && make run

Try it! You'll get a Bazaar branch with all the history of this little blog project.

1 6

To start a new Pylons project, first create an empty directory. Let's call our new project AlliterationSharing¹, because everybody is sick of 'foo' and 'bar'.

¹ Generated by randomly picking two words from /usr/share/dict/words, then chosen over among 120 other variants that weren't as good.

$ mkdir -p ~/src/AlliterationSharing
$ cd ~/src/AlliterationSharing

Now create a file called buildout.cfg with the following content:

$ cat buildout.cfg
[buildout]
parts = pylons

[pylons]
recipe = zc.recipe.egg
eggs = Pylons
       PasteScript
interpreter = python

Download bootstrap.py to it and run it to get bin/buildout. Note: you can chose which Python version you want to use by running bootstrap.py with it. All other scripts under bin/ will be generated by buildout and will use the same Python interpreter.

$ wget http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
$ python bootstrap.py
Creating directory '.../AlliterationSharing/bin'.
Creating directory '.../AlliterationSharing/parts'.
Creating directory '.../AlliterationSharing/eggs'.
Creating directory '.../AlliterationSharing/develop-eggs'.
Generated script '.../AlliterationSharing/bin/buildout'.

Run bin/buildout to install Pylons into your sandbox.

$ bin/buildout
Installing pylons.
Generated script '.../AlliterationSharing/bin/paster'.
Generated interpreter '.../AlliterationSharing/bin/python'.

Aside: buildout has this very nice feature where it can share Python packages between projects. This will save you enormous amounts of time that would otherwise be spent downloading and unpacking eggs. To make use of this facility, create a file ~/.buildout/default.cfg with

$ cat ~/.buildout/default.cfg 
[buildout]
eggs-directory = /home/mg/tmp/buildout-eggs
# XXX replace /home/mg with the full path of *your* home directory
# it would be much nicer if buildout let me use ~ or $HOME
# see https://bugs.launchpad.net/zc.buildout/+bug/190260

Another useful trick is to pass the -N flag to bin/buildout, which will tell it not to bother looking for newer versions of packages on the Internet when there's already an existing version installed in your eggs directory.

Back to business: now you've got two new scripts: bin/python and bin/paster. You can use the first one to play with the interactive Python console where you can now import pylons and all the dependencies; it has no other value.

Now is a good point to add the files you've created into a version control system. I'll arbitrarily use Bazaar.

$ bzr init .
$ bzr add bootstrap.py buildout.cfg
$ bzr ignore bin parts eggs develop-eggs .installed.cfg
$ bzr commit -m "Create AlliterationSharing project"

Run bin/paster create -t pylons to create a skeleton project.

$ bin/paster create -t pylons asharing
$ bzr ignore *.egg-info
$ bzr add asharing
$ bzr commit -m "Generated project files with paster create"

Now paster creates a directory structure that I don't like:

AlliterationSharing/
  buildout.cfg
  bin/
  asharing/
    setup.py
    README.txt
    MANIFEST.in
    asharing/
      __init__.py
      config/
      controllers/
      templates/
      public/

I'd like the README and setup.py to be in the top level, and I dislike repeating 'asharing' twice in directory names. I'll move some files around

$ cd asharing/
$ bzr mv development.ini docs MANIFEST.in README.txt setup.* test.ini ../
$ bzr rm ez_setup.*
$ cd ..
$ bzr mv asharing src
$ bzr ci -m "Moved some files around"

Now the tree looks like this:

AlliterationSharing/
  buildout.cfg
  setup.py
  README.txt
  MANIFEST.in
  bin/
  src/
    asharing/
      __init__.py
      config/
      controllers/
      templates/
      public/

We have to tell setup.py where to find the source tree

$ bzr diff
=== modified file 'MANIFEST.in'
--- MANIFEST.in	2009-09-13 13:04:00 +0000
+++ MANIFEST.in	2009-09-13 13:05:59 +0000
@@ -1,3 +1,3 @@
-include asharing/config/deployment.ini_tmpl
-recursive-include asharing/public *
-recursive-include asharing/templates *
+include src/asharing/config/deployment.ini_tmpl
+recursive-include src/asharing/public *
+recursive-include src/asharing/templates *

=== modified file 'setup.py'
--- setup.py	2009-09-13 13:04:00 +0000
+++ setup.py	2009-09-13 13:04:40 +0000
@@ -17,7 +17,8 @@
         "SQLAlchemy>=0.5",
     ],
     setup_requires=["PasteScript>=1.6.3"],
-    packages=find_packages(exclude=['ez_setup']),
+    packages=find_packages('src', exclude=['ez_setup']),
+    package_dir={'': 'src'},
     include_package_data=True,
     test_suite='nose.collector',
     package_data={'asharing': ['i18n/*/LC_MESSAGES/*.mo']},

(I'm not sure if you also need to change package_data and/or setup.cfg; it's possible that I left i18n in a broken state. Can somebody comment on this?)

And we have to tell buildout that we've got a new Python package to enable in the project environment

$ bzr diff buildout.cfg 
=== modified file 'buildout.cfg'
--- buildout.cfg	2009-09-13 12:57:21 +0000
+++ buildout.cfg	2009-09-13 13:08:05 +0000
@@ -1,8 +1,10 @@
 [buildout]
+develop = .
 parts = pylons
 
 [pylons]
 recipe = zc.recipe.egg
 eggs = Pylons
        PasteScript
+       asharing
 interpreter = python

Now you can re-run bin/buildout and start your hello-world project

$ bzr commit -m "Include the new package in the build"
$ bin/buildout -N
$ bin/paster serve --reload development.ini

Happy hacking!

To be continued: telling buildbot to create bin/test; using ctags and omelette.

0 7

(Recorded with byzanz. My gif-fu is nonexistent or I would make it loop, but with a sufficiently long delay at the end to avoid irritation. Now you have to reload the whole page if you missed the animation.)

I'm somewhat ambivalent about the animation effect. On one hand, shiny! On the other hand, hitting tiny clickable areas is not good usability. Still, shiny!

Footnotes are kind of a personal pet-peeve of mine.

0 9

svn co svn+ssh://svn.zope.org/repos/main/plone.z3cform/trunk plone.z3cform
cd plone.z3cform
python2.4 bootstrap.py
bin/buildout
bin/test -pvc

Now suppose you want to change the buildout environment somehow, e.g. use the current development version of zope.testing instead of whatever is specified in buildout.cfg. Don't edit the existing buildout.cfg (you might accidentally commit your local debug changes), instead create a new cfg file, e.g. test.cfg:

[buildout]
extends = buildout.cfg
develop += ../zope.testing

[versions]
# override any existing version pins
zope.testing =

Now re-run buildout

bin/buildout -c test.cfg
bin/test -pvc

And the tests should be run with the newest zope.testing.code.

Only this does not work with plone.z3cform, and I have no clue why. It generally works with other packages (at least those that use the zc.recipe.testrunner rather than collective.recipe.z2testrunner). Buildout is like that sometimes :(

1 4

Updated gtkeggdeps, the interactive Python package dependency browser. Collaborated with Thomas Lotze, who maintains the engine (tl.eggdeps) that gtkeggdeps wraps, to resolve API mismatches. Moved the sources to launchpad.net, added a test suite, made it use zc.buildout for convenient development.

Moved the source repository of gtimelog, the simple desktop time tracker, to launchpad.net. Failed to do anything else with it. :-(

Tried to work on xdot, wrestled with git-svn merges, failed abysmally. Asked upstream to upload xdot to PyPI.

Released ZODB Browser, but this deserves a separate post.

Sent a bunch of pyflakes patches from my old branch upstream, created trac tickets for the rest. Wrestled with bzr-svn merges, failed abysmally.

4 1

Question: are there any df-like utilities that filter out all the crap and show only interesting bits? The standard df as well as pydf both display 8 lines instead of 1. Discus is worse: it shows 20. GUI utilities like Baobab also suffer from this confusion, especially bind mounts.

Ironically, Ubuntu's update-motd gets confused by Ubuntu's private user directories and displays disk stats for ~/Private as if it were a real partition.

0 4

The question, slightly paraphrased: given a class

class LameContainerOld:
    def __init__(self):
        self._items = {'bar':'test'}
 
    def __getitem__(self, name):
        return self._items[name]
 
    def __getattr__(self, attr):
        return getattr(self._items, attr)

why does the 'in' operator work

>>> container = LameContainerOld()
>>> 'foo' in container
False
>>> 'bar' in container
True

when the equivalent new-style class raises a KeyError: 0 exception? Also, why does __getattr__ appear to be called to get the bound __getitem__ method of the dict?

>>> container.__getitem__
<bound method LameContainerNew.__getitem__ of {'bar': 'test'}>

What actually happens here is that LameOldContainer.__getattr__ gets called for special methods such as __contains__ and __repr__. This is why (1) the 'in' check works, and (2) it appears, at first glance, that you get the wrong __getitem__ bound method. If you pay close attention to the output, you'll see that it's the __getitem__ of LameOldContainer; it's just that repr(LameOldContainer()) gets proxied through to the dict.__repr__ when you don't expect it:

>>> container
{'bar': 'test'}

2 4

First: don't be scared by the list of comment-related plugins on the PyBlosxom site. There's only one important plugin: comments. All others depend on it and enhance its functionality. The last three or four times I was about to add comments to my blog I got scared at step one: evaluate the available plugins. Don't repeat my mistake!

Second, follow the instructions carefully. There's no shortcut.

Third, fix what's broken. Be prepared to debug the source code. print >> sys.stderr, "message" is your friend.

Fourth, fiddle with the look (CSS and HTML).

Fifth, write a blog post and eagerly await your first comments.

Step 3 screams for an explanation, doesn't it? Problem 1: the comments plugin requires that you use categories in your blog. I'm not (I'm holding out for tags). Workaround: comment out if entry['absolute_path'] check in cb_story and cb_story_end.

Problem 2: the AJAX post returns "Empty response from server". Workaround: modify cb_story_end to call readComments directly if entry['num_comments'] is None, since cb_story, which usually does the read, is not called during the AJAX post.

Problem 3: if you enable comment moderation (by setting comment_draft_ext to a different value from comment_ext), the AJAX post returns "Empty response from server" once more. Workaround: modify cb_prepare to notice this case and set data['moderated'] = True, create a new template comment-moderated and render it in cb_story_end just like the preview template is rendered; also modify __shouldOutput to return True when rendering comment-moderated.

I'll post patches to the pyblosxom mailing tomorrow, unless I forget. It's 6 am already, and I'm kind of sleepy. I just hope I haven't inadvertently broken my RSS feed or flooded any planets.

Oh, and a helpful hint: don't name the post you're writing comments.txt, or the #comments anchor will point to the start of the story instead of the comments.

0 5

I upgraded a buildbot slave to Ubuntu 8.04 (Hardy) recently and now I'm getting a strange intermittent failure: sometimes cp -r /local/dir /nfs/mounted/dir fails ("process killed by signal 1", i.e. SIGHUP).

I wonder if NFS is relevant or incidental to the issue?

Google finds an old thread from 2005, with a workaround (usepty=False), but I'd like to understand the problem before applying random fixes.

So far three different build steps doing cp -r have failed during 10 days. I've now changed them all to cp -rv, so I can at least see if the failure is in the middle of the copy or at the end, if it fails again.

Update: so far 4 build steps have failed on 6 separate occasions:

May  5 02:31: cp -r local-dir1 nfs-mounted-dir1  
May  6 02:31: cp -r local-dir1 nfs-mounted-dir1  
May  6 04:33: cp -r local-dir2 nfs-mounted-dir2  
May 15 02:00: cp -r local-dir3 nfs-mounted-dir3  
May 17 04:32: rm -rf nfs-mounted-dir4            
May 20 04:31: rm -rf nfs-mounted-dir4            

I see no particular correlation between step duration and results, e.g. the rm -rf step usually takes between 2.2 and 4.6 seconds. The two SIGHUPs happened after 2.4 seconds.

They all make no output. When I changed the cp steps and added a -v, they stopped failing, but that could be just a coincidence.

We're having an email conversation with Jean-Paul Calderone ("exarkun") about the possibility of this being PTY-related, with no clear resolution so far.

And, hey, now this blog supports comments ;)

0 6

I received a free PDF copy of this book from Packt Publishing, with the understanding that I'll post a review on my blog. This is it. Short summary: it is a good book marred by a lot of mostly inconsequential little mistakes. I'd give it four stars out of five.

Aside: the PDF that I could download was personalized and had my name and address in the footer of every page. A very nice form of DRM that did not restrict my software choices for reading the book (Evince and also PDF Reader on Nokia Internet Tablets).

I bring it up here because it seems that Packt could've also applied fixes for the known errata to the personalized version, yet missed that opportunity. Perhaps it's technically more difficult than slapping a footer on every page. Or maybe it's better if everyone buying the book, whether in paper or in PDF, gets to see the same text.

The author (Tarek Ziade) covers a wide range of topics in the book, ranging from syntax (probably useful for those who've been programming in Python for quite a few years, and didn't have the time to keep up with the language changes before picking up this book) to style, source code organization, project infrastructure, software life cycle, documentation, testing and optimization, and finally ending with a review of some of the popular design patterns. The middle parts were the most interesting for me personally. I learned a thing or two, disagreed with the author on a few minor points (which are mostly a matter of preference), and managed to finish the book despite constant irritating little pricks I feel when I notice an error (I confess I'm a pedant. A missing space after a colon drives me up the wall).

As an example of the disagreement: I have an aversion to code-generating tools where you have to edit the generated code by hand. I could say more, but this is a topic for another time. Next, I strongly dislike sudo easy_install since it scribbles onto the part of the filesystem exclusively reserved for your OS's package management tools. And I don't think porting the original 23 design patterns to other programming languages is a good way to describe what those languages are about. (Also, set tabstop=4 in your .vimrc? Heresy! The Right Thing To Do is set softtabstop=4, as all right-thinking Vim users will doubtlessly agree. All hail the one true text editor! Oh dear, now I'm glad I don't have comments on this blog...)

The goodies: Chapter 1 (the bits about PYTHONSTARTUP on page 19) gave me persistent history for my interactive Python prompt, nicely complementing the coloured prompt and tab-completion I already had snarfed from somewhere else on the net (probably Peter Norvig's Python IAQ). Chapter 12 provided good examples of how to do profiling for time (page 281) and memory (page 291). I like Tarek's @profile decorator (measure time, pystones and memory at the same time). My profilehooks module was not mentioned, *sniff* ;-). Chapter 13 told me about Queue.join and task_done that snuck into the stdlib with Python 2.5 without me noticing.

I haven't mentioned topics covered in the book that I was already familiar with, such as setuptools, virtualenv, zc.buildout, Sphinx, Nose, Buildbot, or Mercurial. Yet, in my opinion, those are the most useful parts of the book. The breadth of the topics is amazing: I could hardly think of something that every serious Python programmer should know that isn't wasn't mentioned. I believe the depth was exactly right: mention solutions that are available, show how they feel when used and what they can do, point to the relevant web page and then stop. And not only tools, the descriptions of workflows (how to organize your source trees, how to develop software consisting of multiple packages, how to make releases), while hardly universal, are invaluable.

One thing prevents this from being a perfect book: errata. At around page 95, according to my notes, I invented a new metric of book quality: WTFs per page, It's closely related to WTFs per minute, but independent of your reading speed. At around page 165 I got tired of making a note of every little thing that I noticed and started just reading. This was considerably more enjoyable. I hope there's a second edition will all the bugs shaken out. To that end, I should go through my notes again and submit them via the online errata form. Yay, more work...

3 3

Here's what you can do if the GUI app in question doesn't have that menu item, and you don't remember what it's called, and you're the same sort of a crazy command-line person that I am:

$ xprop|grep PID

Click on the app's window. Watch that shell command return a line that looks like

_NET_WM_PID(CARDINAL) = 807

$ ps 807  # substitute the real number

You'll see the command name, e.g.

  PID TTY      STAT   TIME COMMAND
  807 ?        S      0:02 /usr/lib/indicator-applet/indicator-applet --oaf-acti

Now you can run

$ ubuntu-bug /usr/lib/indicator-applet/indicator-applet

If you're not running Jaunty, you'll need to do one more step to find the name of the Ubuntu package:

$ dpkg -S /usr/lib/indicator-applet/indicator-applet
indicator-applet: /usr/lib/indicator-applet/indicator-applet

You can use that with apport-cli or on the Launchpad online bug reporting form.

For some (many?) programs you can short-circuit this trail by looking at the WM_CLASS property instead of the _NET_WM_PID property:

$ xprop|grep CLASS
WM_CLASS(STRING) = "indicator-applet", "Indicator-applet"

While there is no requirement for the window class name to match the name of the Ubuntu package or the name of the executable, it may give you a reasonable guess.

0 7

After a little hiccough at the beginning, the upgrade was the smoothest Ubuntu upgrade I've ever had: I spent those two and a half hours browsing the web, watching screencasts and chatting on IRC, while update-manager worked in the background. Firefox was mostly very responsive, only stuttering when update-manager got around to unpacking openoffice.org-common. There were few debconf or conffile questions (one from sysstat 2 minutes into the upgrade, then a conffile question after 1 hour and 20 minutes, then two more after 15 minutes, and one more 5 minutes later. And the last one 10 minutes later). There were no ugly theme changes or failing GNOME applet error messages during the upgrade. Near the end X-Chat automatically started showing new-style notifications (beautiful!) and Firefoxes nicely asked to be restarted with a fold-down notification bar.

Nice. Now, after a reboot things were not so nice: I couldn't login. After typing in my password and a couple of mode changes I was kicked back to the GDM prompt. I panicked and started logging into the text consoles and trying to run startx, quite in vain, since when I just tried gdm again it worked fine.

The intel video driver feels slower, as promised by the release notes, but it's acceptable as long as you don't try to rotate the external screen. Then it's horrible and unusable—a regression since Intrepid. I'll have to retry with UXA.

Compiz failed to enable a plugin (GNOME Compatibility), so a couple of key bindings didn't work (Alt+F1 to get the menu, Alt+F2 to get the run dialog, my custom keybinding to open a terminal) until someone on FreeNode told me what to enable.

The Flash plugin is now swfdec, and it is unable to cope with Youtube music videos—the sound is all choppy. I'm wondering if this is swfdec's fault, pulseaudio's fault (it's common knowledge that all audio problems stem from pulseaudio, right? ;) or X.org's fault (top shows it's X that's eating 90% CPU when swfdec is trying to play a video).

A lot of very irritating bugs are gone. I don't need to restart Compiz after playing with xrandr. X doesn't crash after I play with xrandr. Two-finger scrolling with the Synaptics touchpad doesn't produce phantom scroll-down-17-pages events when I take my fingers off the touchpad. The GNOME panels don't migrate to the external screen when I play with xrandr (but one of them jumps from the bottom to the top when I play some more). The new splash screen has a pretty gradient for its progress bar (but is displayed off-center, maybe because I added vga=872 to my GRUB kernel options list to avoid ugly stretching of text consoles). X.org no longer distorts the aspect ratio of 1024x768 when stretching it to fit the 1280x800 screen—now I get sensible black bars on the sides. The new notification bubbles are beautiful!. I could stare at them all day. (But the new indicator applet is ugly.)

Overall I'm happy. A bunch of very irritating bugs were replaced with a smaller bunch of somewhat less irritating bugs. The intel video slowdown scares me a bit, though, but the prettiness of the notification bubbles outweigh everything else. What can I say, I like pretty things—Ooh, shiny!

0 6

Leap of knowledge: since I know lazr.enum was created by the Launchpad.net team I could safely assume they were keeping the sources in Launchpad. Therefore I was pretty sure I could get them with

$ bzr branch lp:lazr.enum

so I ran that command and it worked.

Next I looked at setup.py to see how it produces the long_description field. It was reading the contents of a couple of text files, one of them being src/lazr/enum/README.txt. I looked at that and saw a .. toc-tree: directive that does not exists in plain docutils (it's a Sphinx extension).

I added up a couple of lines to setup.py to strip that out, tested it (with setup.py --long-description > test.rst; restview test.rst) committed to my local branch, and created a bug report in Launchpad. Then I was a bit lost, since I didn't know how to make my fix available. Attach a patch? Maybe, but I wanted to see if this distributed version control thing is good for anything else.

I thought that first I'd make that branch public, and then see if there was a way to link it to the bug report. I ran

$ bzr push lp:~mgedmin/lazr.enum/pypi-fix

which took a few seconds to create a new public branch on Launchpad with my fix in it (it would be nice if I didn't have to explicitly specify my Launchpad username and the project name—both of which bzr already knows—and just specify the name of the branch). Then I went back to my bug report and saw an option to link it to a branch. There was a search field in the popup that found my "~mgedmin/lazr.enum/pypi-fix" easily enough when I pasted it into the search box.

After clicking on the branch, I saw a "propose a merge" option. I did that and Launchpad sent an email to the developers asking them to merge my fix.

I made one mistake, I think: I should've created the bug report first, and then mentioned the bug number in my commit message (with bzr commit --fixes=NNN, although here I'm suddenly not sure if the bug number should be left bare, or prefixed with something like "lp" to indicate it was a Launchpad bug number?).

Other than that it was a pretty smooth experience. When will I be able to do that for Ubuntu packages?

0 6

Also wanted: a spam-resistant comments plugin. (Maybe some/all of them are? I was always afraid to try.) And a tags plugin (categories are too limiting). And useful page titles. And a pony.

Update: Laurence Rowe and Alexander Artemenko suggested I check out cssutils. Alexander also pointed out that a similar question was asked on stackoverflow. Peter Bengtsson recently posted a similar tool built with lxml and regexps. Meanwhile, I took the plunge and installed a comments plugin, and even somehow managed not to get drowned in spam.

0 7