Codehelp blog

emsource/emdebuild machine variants

Extending the customisation support in Emdebian so that the Emdebian package now becomes the source package for a machine specific variant build. Using the same principles as the machine variant handling in emsandbox, customised builds would apply patches over the Emdebian source to modify dependency chains, remove unwanted binaries etc.. emsource would use the existing Emdebian package (i.e. the package must have been cross built for Emdebian already) as the source, providing an unpacked package with the Emdebian patches already applied. emsource would use the ../machine/$machine/$variant directory instead of trunk/. Patches from the machine/$machine/$variant/f/foo/ subdirectory of your Emdebian work directory would then be applied. emdebuild would update those patches because the relevant $foo.old directory would contain the Emdebian source. The Debian source would always be available in trunk/ for comparison. Using the Emdebian package as source means that the variant build tree will not contain any .svn/ or CVS/ directories so machine/variant builds can use your preferred RCS. Expected in emdebian-tools >> 0.7.0.

langupdate and tdeb packages

The current Emdebian package search script will be adapted to handle the buildd.emdebian.org/locale repository and omit locale packages (tdebs) from the default search and package count. locale packages will be migrated into the locale repository with a cron task, removing them from the main repository at the same time. This way, uploads proceed as normal, no modifications are needed for the .changes file and the main repository apt cache is kept as small as possible. langupdate itself will use apt options to direct apt to the langupdate cache of locale packages. Unlike the main apt cache, this cache is removed after each langupdate run (actually it will probably be removed just before the packages are unpacked). langupdate will call apt to do the download and actual installation. Whilst this does mean that the installed tdebs will show up in the apt and dpkg cache data, it does prevent the other 90% of locale packages from also ending up in the main apt cache. Besides, the user should be able to see which tdeb packages are installed using the normal dpkg -l queries. tdeb packages are very small but hopefully apt-get autoclean will clean up the downloaded package files. langupdate will also automatically remove any tdebs and translation files for unsupported locales. Finally, langupdate will be run for the first time by the eventual Emdebian installer - as soon as the user has selected a timezone and set of supported locales. langupdate may well be extended in future to support handling the zoneinfo data from tzdata and the gconv data from glibc. langupdate currently uses libglib2.0-0 (because I like using it) but if anyone fancies developing it in plain C (or probably C++ to fit in better with apt), let me know. Code is in Emdebian SVN.

These are all just 'first-builds' so problems could well lie ahead but I have now built a set of cross built packages for Emdebian (ARM) that edos-debcheck says is installable for a basic GPE installation. Only one package remains as a problem - gpe-filemanager builds fine but the current dependencies of libgnomevfs mean that gconf is required which doesn't give an option to disable ldap which is something I would rather do without. Certainly, existing GPE installations do without ldap but that could just be because these tend to use older versions of the particular libraries.
The GPE team status report covers the rest of the package set with links to the Debian and Emdebian packages.
I've got a few ITP's to close and a few more to open and close before the full GPE toolset is available (and I need to work out what has happened to the source package for gpe-terminal) but things are looking good.
All the cross building problems were in the base packages and a few of the lower level system libraries - the X libraries and GPE applications cross build very easily, thanks mainly to CDBS support in emdebian-tools.
The hard task lies ahead - configuring the packages so that Emdebian can support pre-built installation images for specific devices with their specific hardware configuration (and nothing else).
I'm expecting a 70% reduction in the amount of space required for the same package set relative to Debian.
When browsing the Emdebian package set you may think that 1,736 packages (or so) is a lot but this is mainly down to the Emdebian Tdeb implementation which splits out each translation for each package into a package of its own.
The next stage of the Tdeb process is to migrate those packages into a separate repository and finish the development of langupdate. Then, each user can choose to install only the .mo files required for their specific needs instead of getting 250Mb of translations for languages that are unsupported on their machine anyway.
The main deviation from the existing tdeb proposal is that Emdebian needs finer granularity so that gettext translations are separate from translated manpages - because Emdebian doesn't include any manpages.

I'd better put some details of this somewhere, so here goes.
Moray Allen and I have formed a Debian GPE team at Alioth. The idea is to gather all the GPE packaging stuff onto a mailing list as a single point of contact for all things related to GPE - the G Palmtop Environment.
I'm now busy updating the GPE packages to add the Debian GPE team as Uploader and taking advantage of these uploads to also add cross building support for all these packages for Emdebian as well as the normal upload tasks for keeping up with the Debian Menu transition and fixing lintian warnings that arose since the last upload, etc.
My basic idea is that everything under the "Debian GPE team" banner should be cross-buildable - it's only logical.
Once dpkg is fixed to once again allow cross-building properly, I'll upload these packages to Emdebian and lots more of their dependencies.
Next task is to file a few more ITP bugs and make good on my existing ones so that GPE can be a complete environment within Debian and Emdebian.

Only an outline so far but most of this code already exists in Emdebian SVN.

1. C/C++ application: langupdate using libapt-pkg and libglib2.0-0
   
   • Reads supported locales and silently installs only the supported translations for all installed packages, with fallback support where configured.
   
   
   • Silently removes all translations for all unsupported locales.
   
   
   • First run in langupdate postinst
   
   
   • Removes package translations on the next run after package removal.
   
   
   • Separate /etc/langupdate/tdeb.sources.list and /var/cache/langupdate/lists/ to keep apt cache size down. Manual changes will be overwritten so maybe it ought to live in /var/ or /usr/share/.
   
   
   • One source for each locale root, main_en, main_fr, main_pt etc. All sources maintained by langupdate directly, according to the list of supported locales.
   
   
   • Each source contains all translations for locales within scope: en_GB in en, pt_BR in pt etc. to support fallback correctly.


2. XS-Package-Type: tdeb with a file suffix of .deb (not .tdeb) to reduce the number of patches needed for a working implementation.


3. Server-side perl script: langmigrate that parses the .changes file:
   
   • Removes tdebs from existing repository.
   
   
   • Moves incoming tdebs into new layout.


4. No descriptions, no dependencies. All tdebs are Priority: extra and Architecture:all. Possible support for a tdeb1 version suffix for updates that are not yet in Debian. (Allows for translation updates without package uploads.)


5. Plan to remove (or support an alternative package) dependency on libglib2.0-0 but OK for Emdebian at the moment due to targetting of GPE.


6. langupdate itself Priority: optional and not part of the default emsandbox


7. langupdate is a native package, uploaded to and maintained in Emdebian. Migrate into Debian once infrastructure is ready.


8. langupdate has little or no output, only a help option and possibly support for --clean but everything else automated.

This is built on the emlocale method (from emdebian-tools) and emlocale will be modified to fit this plan in 0.5.4.

langupdate (and langmigrate) cannot be part of emdebian-tools themselves, so will be separate packages.

(and yes, it is 4am, I'm not sleeping well due to a viral infection.)