Codehelp blog

Emdebian SVN r2618 includes my first working implementation of em_installtdeb. Barring a single annoyance message from dpkg, the script works.
The warning arises from the need for a modified "source" tarball that can be uploaded alongside the .deb, containing only the sources needed for this specific translation: debian/control, debian/changelog, po/$lang.po /* if any */ and po/$GETTEXT_PACKAGE.pot. Where a package contains more than one po directory, all POT files are included and all po files for the requested translation. This allows translations to be updated independently of the upstream package.
I would advise using a test directory if you do want to try it, not your main package building directory.
Output (using gpe-expenses as the demo package) in translator mode:

$ em_installtdeb sv
dpkg-deb: building package `gpe-expenses-locale-sv' in `../gpe-expenses-locale-sv_0.1.4-1_all.deb'.
dpkg-source: warning: .orig.tar name gpe-expenses_0.1.4.tdeb.tar.gz is not _.orig.tar (wanted gpe-expenses_0.1.4.orig.tar.(?:gz|bz2|lzma))
dpkg-source: building gpe-expenses using existing gpe-expenses_0.1.4.tdeb.tar.gz
dpkg-source: building gpe-expenses in gpe-expenses_0.1.4-1.diff.gz
dpkg-source: building gpe-expenses in gpe-expenses_0.1.4-1.dsc
dpkg-genchanges: including full source code in upload
Format: 1.7
Date: Fri, 07 Dec 2007 00:08:39 +0000
Source: gpe-expenses
Binary: gpe-expenses-locale-sv
Architecture: source all
Version: 0.1.4-1
Distribution: unstable
Urgency: low
Maintainer: Neil Williams
Changed-By: Neil Williams
Description:
gpe-expenses-locale-sv - sv translation for gpe-expenses
Changes:
gpe-expenses (0.1.4-1) unstable; urgency=low
.
* New upstream release
* use machine operable debian/copyright, migrate to standards 3.7.3
Files:
781a60eccc4396ee3f0f86a01150e755 316 misc extra gpe-expenses_0.1.4-1.dsc
f9dd946db27ff6077f8e85f7100d3ea4 5863 misc extra gpe-expenses_0.1.4.tdeb.tar.gz
894f5789ee0348abfa61a5e7342ed6a7 20 misc extra gpe-expenses_0.1.4-1.diff.gz
56f8d22c30cb5502633ff4d26862db78 4532 misc extra gpe-expenses-locale-sv_0.1.4-1_all.deb
Package-Type: tdeb
Changes file: ../gpe-expenses_0.1.4-1_tdeb.changes
signfile ../gpe-expenses_0.1.4-1.dsc 0x28BCB3E3

You need a passphrase to unlock the secret key for
user: "Neil Williams (Debian) "
1024-bit DSA key, ID 28BCB3E3, created 2002-01-27


signfile ../gpe-expenses_0.1.4-1_tdeb.changes 0x28BCB3E3

You need a passphrase to unlock the secret key for
user: "Neil Williams (Debian) "
1024-bit DSA key, ID 28BCB3E3, created 2002-01-27


Successfully signed dsc and changes files
neil@holly:gpe-expenses$ dpkg -c ../gpe-expenses-locale-sv_0.1.4-1_all.deb
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/share/
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/share/locale/
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/share/locale/sv/
drwxr-xr-x neil/neil 0 2007-12-19 16:09 ./usr/share/locale/sv/LC_MESSAGES/
-rw-r--r-- neil/neil 6419 2007-12-19 16:09 ./usr/share/locale/sv/LC_MESSAGES/gpe-expenses.mo
-rw-r--r-- neil/neil 1796 2007-12-19 16:09 ./usr/share/locale/sv/LC_MESSAGES/qofexpensesobjects.mo

Hmm, that isn't right - need to check on the ownership of those files - might be enough to use fakeroot.

$ dpkg -I ../gpe-expenses-locale-sv_0.1.4-1_all.deb
new debian package, version 2.0.
size 4532 bytes: control archive= 302 bytes.
230 bytes, 9 lines control
Package: gpe-expenses-locale-sv
Source: gpe-expenses
Version: 0.1.4-1
Architecture: all
Maintainer: Neil Williams
Installed-Size: 44
Section: misc
Priority: extra
Description: sv translation for gpe-expenses


em_installtdeb is a debhelper add-on created by Emdebian to create translation packages (tdebs). em_installtdeb is intended to separate out the individual translation files from the current Debian packages into packages without any translation files and a series of tdeb locale packages, one per translation. Generated packages use the syntax: $package-locale-$language_code_$version_all.deb
Certain language codes need to be modified to make acceptable components of a debian package name. Underscores are converted to hyphens, '\@' is converted to '+' and all codes are made lowercase. These changes only apply to the package name, the installation location is unchanged.
The locale package must use GETTEXT_PACKAGE for the eventual filename of the binary translation file - not the $dh{MAINPACKAGE}. GETTEXT_PACKAGE is determined by upstream, not Debian. When building the whole package, the binary translation file may be in debian/tmp/usr/share/locale/$lang/LC_MESSAGES but when in translator mode, this location is not available. Instead retrieve GETTEXT_PACKAGE from the POT filename. debian/control is needed for various dpkg and debhelper calls only.

The "source" for translators therefore includes:
debian/control
debian/changelog
po/$lang.po /* if any */
po/$GETTEXT_PACKAGE.pot

Note that no rules file exists. (i.e. dpkg-source -x will complain when trying to extract it later so a new wrapper is needed (not difficult).)
em_installtdeb will try to generate the necessary POT file(s) and then create a $package_$version_tdeb.tar.gz containing the source files.
The default action is to process all available po files.
The above example uses translator mode where LANG_CODE is passed to only handle that language. In translator mode, em_installtdeb builds a tdeb package, a source tarball, a .dsc and a .changes file (and signs the .dsc and .changes). In default mode, em_installtdeb builds the package(s) but leaves the source, .dsc and .changes to the rest of the build process.
LANG_CODE can either be the name of the locale as specified in the PO filename (fr, en_GB, sr@Ltn etc.) or the name of the locale as specified in the eventual package name (fr, en-gb, sr+ltn etc.).
If a po file already exists for this locale, em_installtdeb processes just that one translation and builds a single tdeb package.
If the po file does not exist, em_installtdeb exits with an error. (Copy the POT file from the original source to create a new po file for the new locale and edit it to include the new translation, then run em_installtdeb again.)

Output in default mode:
$ em_installtdeb
dpkg-deb: building package `gpe-expenses-locale-cs' in `../gpe-expenses-locale-cs_0.1.4-1_all.deb'.
dpkg-deb: building package `gpe-expenses-locale-de' in `../gpe-expenses-locale-de_0.1.4-1_all.deb'.
dpkg-deb: building package `gpe-expenses-locale-pt' in `../gpe-expenses-locale-pt_0.1.4-1_all.deb'.
dpkg-deb: building package `gpe-expenses-locale-sv' in `../gpe-expenses-locale-sv_0.1.4-1_all.deb'.
dpkg-deb: building package `gpe-expenses-locale-vi' in `../gpe-expenses-locale-vi_0.1.4-1_all.deb'.

Next stage is to fold this into emdebian-tools so that the emdebian patch files can be cleaned up and various complex patches to debian/rules can also be cleaned up.
Finally, note that this only supports gettext and only supports program translations. This is an Emdebian script written with an eye on usage in Debian but primarily intended to meet Emdebian requirements. As outlined before, Emdebian does not want language packs or translation grouping. One locale, one package.
See also langupdate in Emdebian SVN.

Further to my musings on a debhelper-type script to handle tdebs, I'm considering two modes:

1. maintainer mode - the default. With no options, operate on all .po files in all identifiable gettext directories (if -d /^po.*/), creating absolutely minimal control data on-the-fly, installing the directories and the binary translation data (.gmo) and building the .deb for each translation (with nothing else in the package). Emdebian will use this mode when cross building.


2. translator mode - when passed the name of a translation locale (either in the gettext form, e.g. sr@Ltn, or the filename form, sr+ltn) only builds a package for that po file. Should be relatively simple to create upload wrappers that can handle single translations in a similar manner to current uploads from porters.

Likely that some kind of update tool is going to be needed to retrieve the updated translations from the locale repository and merge back into the upstream package - which in turn means some form of "translation source" in the repository (the po file as a gz maybe?). Need to keep the POT file somewhere as a source file too so that new translations can be created easily. Such a tool would also need to be distribution-neutral as upstream may not be using Debian at all. Likely to investigate linkage with the Translation Project on an opt-in basis. (Not all upstreams use TP).
I can see an apt-locale package in the offing . . . - get source, build a single tdeb for a new or updated translation, query the locale cache, partner to langupdate . . .
Still not convinced on the script name: em_installtdeb because this script also generates stuff and the 't' is almost hidden in the name. (Eventually, I'd seek inclusion into debhelper and change the prefix accordingly).
Also not sure how to handle the package version string - this isn't commonly in the POT file (and may often be wrong or apply to the upstream version) but some Debian-compatible versioning will be needed. May need a stump debian/changelog that is discarded during building and which is generated within maintainer mode - i.e. held in the tdeb source.
Yes, I know some of this diverges from the Debian TranslationDebs approach but this is done with good reasons:

1. Emdebian does not want more than just the translation binary files in the tdeb package.


2. To get a working implementation soon, Emdebian is doing without the .tdeb suffix and going with .deb


3. No dependencies at all


4. No "language packs" as Emdebian needs full control over individual translation files, not over mere collections of localisation support files.

The blog has been quiet but I'm not.
Sometimes things happen in a flood but, in my life, that only happens after a prolonged period of thinking, planning, breaking, fixing, swearing and testing. Right now, I'm in one of those intervening periods when lots of things are just concepts and it also coincides with a period of more-than-usually stressful paid work. (Those who are familiar with healthcare will be aware of the national panic that ensues when things close down for just two days).
So what am I conceptualising?

• Emdebian tdeb implementation - including a new debhelper script to replace emlocale and do the actual package generation work instead of a complex shell script in debian/rules


• libgda3-3 migration for libqof1 which is being combined with better shlibdeps and cross build support in QOF itself - using QOF as a "test package" for all the other stuff I'm doing so that QOF cross builds with just:
  PKG_CONFIG_LIBDIR=/usr/arm-linux-gnu/lib/pkg-config dpkg-buildpackage -aarm (until dpkg includes the pkg-config setting when it will cross build with just d-bp -a)
  


• Emdebian autobuilder support for target packages


• Emdebian support for debian-xcontrol

I've got a few minor things to do in the meantime but expect another flurry of visible activity towards the end of the week.

I've just uploaded the first (hopefully) working version of langupdate to Emdebian. Full details on the emdebian website.
There are only a few packages in the emdebian locale repository right now. langupdate will be part of the default Emdebian GPE setup.
This is all very experimental. Emdebian needs tdebs urgently so the decision was taken to proceed on an experimental basis and then feed back to the Debian tdeb implementation and fold the Emdebian code into the Debian tdeb support.