Codehelp blog

I've got a few fixes to go into drivel 3.0.1 but I've been trying to decide on whether to re-factor the underlying code - libsoup specifically. Drivel has it's own way of handling xmlrpc messages and this method doesn't work well with the libsoup2.4 changes, so currently it backports some code from libsoup2.2. This isn't ideal for all the usual reasons and I've been waiting for time to sort it out.

Time is lacking. My change of career has dramatically changed the amount of time I have for upstream code and as long as drivel continues working for most people, I'm not sure how much time it deserves.

I've done a lot of refactoring in drivel already, for the GTK3 transition mainly, but this libsoup stuff is much more awkward because it raises new bugs with each blog engine that drivel tries to support. It's not helped when some blog engines retain broken XML practices that mean that the standard libsoup2.4 support simply fails to post a message with properties. (I'm looking at you livejournal.com).

Anyone fancy helping out with a branch where the old xmlrpc code can be replaced by soup_value_hash_insert? I've made an SVN branch for soup24 changes at SourceForge. There's a mailing list too (v.quiet).

I'm going to update a few screenshots, complete a few more tests and then release 3.0.1.

1. My Orange 3G mobile broadband USB modem won't operate properly with gnome-ppp because it doesn't use a real username or password and needs more customised chatscript support than gnome-ppp appears to be able to support.


2. I stopped using network-manager-gnome and started using wicd instead due to problems with nm-applet at that time, which doesn't see the USB modem (and doesn't drop wlan0 for it either). I'm not sure if nm-applet would now work with the USB modem.


3. I've got the modem working with sudo pon orange but I wanted a GUI.

Instead - and, frankly, just as an experiment to get used to auto-detecting things - I decided to write a little program that puts a status icon in place if HAL reports no modem capability, then raises a simple window to manage the connection when the modem becomes available. (I had a shell script using zenity that was almost good enough but without automated launch.)

The window is trivially simple, it just lists the available connections in /etc/ppp/peers and has a toggle button to connect or disconnect the selected connection. Almost no error checking, no configuration support for the pon scripts themselves. I might add a default connection using gconf later. Yes, I know it's HAL but PolicyKit/DeviceKit just seemed too complicated when it was simpler to use libhal's own dbus support.

I'm happy to just use this for me but if anyone else is interested in this kind of app, I might be persuaded to put the source code somewhere public and continue development with a view to uploading to Debian. Naturally, if gnome-ppp or wicd could be adapted to replace it, that would be good too. (I'll file wishlist bugs if it isn't just me who wants this - I only need this functionality for a few months and a quick and dirty version is enough just for me. By the time the bugs are fixed, I probably won't be using mobile broadband any more.)

Further to my previous entry of putting the house on the market, it's now definite that I'm relocating to Cambridge to work as a self-employed application developer with Toby!Churchill.

Time to start hunting for accommodation in Cambridge tomorrow . . .

Yes, aptitude can list packages by priority, but what I wanted was a simple list of just package names. I've got two ways of getting a bare list:

Updated with many thanks to Phil Hands.

$ aptitude search '~prequired'|sed -ne 's/^i [ A] \([^ ]*\).*$/\1/p' | tr '\n' ' ' | sort -u|fold -s ; echo

#!/bin/sh
# Licence: GPL-3+
FILE=`find /var/lib/apt/lists/ -name '*unstable*Packages'`
LIST=`grep-dctrl -n -s Priority '' $FILE |sort -u`

for P in $LIST; do
  [ "optional" != "$P" -a "extra" != "$P" ] || continue
  echo "Priority: $P"
  grep-dctrl -n -s Package,Priority "$P" $FILE | sed -ne '/^'$P'$/!h;/^'$P'$/{g;p}' | tr '\n' ' ' | fold -s
  echo
done

The advantage with the script is that is lists all packages in all priorities except optional and extra and can be restricted to individual Packages files and therefore to individual repositories and components. (Change the find expression if you have more than one repository specifying unstable or adapt the script to accept a path to the relevant Packages file on the command line.)

I really ought to have used perl because pattern matching in perl is a
lot more natural to me than using sed or awk. I do tend to chain a
couple of sed commands, just so that it's easier to work out which bit
is broken when it breaks.

The final version of this script will end up in emdebian-grip-server,
converted to perl with POD content, translated and reading the Packages
files using Debian::Packages::Compare.

$ ~/scripts/list-priority
Priority: important
adduser apt-utils apt aptitude bsdmainutils libbz2-1.0 cpio cron libcwidget3
debian-archive-keyring dhcp3-client dhcp3-common dmidecode ed libgdbm3 gnupg
gpgv groff-base ifupdown iproute iptables iputils-ping logrotate libept0
libsigc++-2.0-0c2a libtasn1-3 libusb-0.1-4 man-db manpages module-init-tools
nano libncursesw5 net-tools netbase netcat-traditional libnewt0.52 whiptail
libssl0.9.8 libpopt0 libreadline5 libreadline6 readline-common rsyslog
tasksel-data tasksel libwrap0 info install-info traceroute udev vim-common
vim-tiny wget libxapian15
Priority: required
libacl1 libattr1 base-files base-passwd bash coreutils debconf-i18n debconf
debianutils diffutils dpkg e2fslibs e2fsprogs libcomerr2 libss2 libc-bin libc6
findutils gcc-4.3-base gcc-4.4-base libgcc1 libstdc++6 grep gzip hostname
lsb-base libdevmapper1.02.1 lzma liblocale-gettext-perl libselinux1 libsepol1
libtext-charwidth-perl libtext-iconv-perl libtext-wrapi18n-perl libunwind7 mawk
libncurses5 ncurses-base ncurses-bin libpam-modules libpam-runtime libpam0g
perl-base procps sed sensible-utils login passwd libslang2 initscripts sysv-rc
sysvinit-utils sysvinit tar tzdata bsdutils libblkid1 libuuid1 mount util-linux
zlib1g
Priority: standard
apt-listchanges at bash-completion bc dc bind9-host dnsutils libbind9-50
libdns53 libisc50 liblwres50 bsd-mailx libsasl2-2 libdb4.8 libdb4.5 libdb4.6
libdb4.7 debian-faq doc-debian doc-linux-text libc6-i386 locales exim4-base
exim4-config exim4-daemon-light exim4 file libmagic1 gettext-base libgnutls26
libgpm2 libkeyutils1 libgssapi-krb5-2 libgssrpc4 libk5crypto3 libkadm5clnt6
libkadm5srv6 libkdb5-4 libkrb5-3 libkrb5support0 less lsof libcap2 libedit2
libevent-1.4-2 libgc1c2 libgcrypt11 libgpg-error0 libgssglue1 libidn11
liblockfile1 libnfsidmap2 librpcsecgss3 libunwind7-dev libxml2 m4 mime-support
mlocate mutt ncurses-term ftp telnet nfs-common libldap-2.4-2 openssh-client
patch libpci3 pciutils libpcre3 perl-modules perl portmap procmail
python-central python-minimal python python2.5-minimal python2.5
python-reportbug reportbug wamerican libsqlite3-0 tcpd texinfo time ucf w3m
whois

Hmm, next step is to check the "Priority: standard" set for Emdebian - it looks far too large for what would be standard on an embedded device. LDAP will be dropped for one - and what is a -dev package doing in there? libunwind7-dev isn't even installed on my amd64 Debian desktop!

Probably check the important and required listings too, then tweak what we need to change via the archive overrides for Emdebian Grip.

Come to think of it, I might need a PHP version of this to read live data from the repository, documenting the changes.

Sometimes, the main task at any one time is simply going to take time and cannot be shortened. With my poor network connection, this tends to occur any time I need to do network tasks like multistrap and debootstrap tests, archive updates and the rest. The problem is that my network connection is so bad, most apt type tasks completely saturate all available bandwidth, especially when downloading a single large package like those built from gcc or OOo.

Thankfully, gpdftext has provided an intermittent task that is useful, intermittent, non-network dependent and relatively straightforward (so as not to be too distracting from the main task) without being sufficiently routine to make me think about automating it.
I've downloaded some 70 public domain novels as A4 PDF and I'm gradually working through them in gpdftext to tidy up the chapter headings and reformat as an A5 PDF, ready to be transferred back to my ebook reader.

This version of gpdftext isn't released yet (it's current SVN, 0.1.0) as it's in string freeze. However, it provides a fairly robust test of the next release.

Current list includes: Little Women, Fairy Tales of Hans Christian Andersen, Lady Susan, Northanger Abbey, Pride and Prejudice, Sense and Sensibility, Peter Pan (Peter and Wendy), Lorna Doone: A Romance Of Exmoor, The Gap in the Curtain, The Thirty-Nine Steps, Through the Looking Glass (And What Alice Found There), Fanny Hill: Memoirs of a Woman of Pleasure, Moll Flanders, Robinson Crusoe, The Further Adventures of Robinson Crusoe, A Tale of Two Cities, David Copperfield, Great Expectations, Little Dorrit, Oliver Twist, The Life And Adventures Of Nicholas Nickleby, The Adventures of Sherlock Holmes, The Casebook of Sherlock Holmes, The Hound of the Baskervilles, The Memoirs of Sherlock Holmes, The Return of Sherlock Holmes, Cranford, The Wind in the Willows, King Solomon's Mines, Tess of the d'Urbervilles, The Hunchback of Notre Dame, Declaration of Independence, Just so Stories, Kim, The Jungle Book, The Man Who Would be King, The Arabian Nights, Lady Chatterley's Lover, Sons and Lovers, Women in Love, The Story of Doctor Dolittle, The Call of the Wild, The Game, White Fang, Moby Dick, Anne of Green Gables, The Well at the World's End, Pygmalion, An Inquiry into the Nature and Causes of the Wealth of Nations, East is West, Strange Case of Dr Jekyll and Mr Hyde, The Jewel of Seven Stars, Anna Karenina, The Adventures of Huckleberry Finn, The Adventures of Tom Sawyer, The Kama Sutra, 20,000 Leagues Under the Sea, The Aeneid of Virgil.

That should be enough for now.... (21 converted so far).

GPE packaging (at least for my own packages in GPE) has been added to svn.debian.org (alternative browser front end - my preference) and available via svn://svn.debian.org/pkg-gpe/.

Some packages have been converted to dpkg source format 3.0, although only for purposes of using the upstream .orig.tar.bz2 directly without repacking and for the purposes of testing svn-buildpackage 0.7.1 support.

I've got a new, fast, server and I've migrated my homepage to Drupal6. This has given me a chance to experiment with i18n updates.

I've got a section for translations and I can write blog entries for this section using drivel. The new i18n blog is a separate feed, not syndicated on planet, intended for anyone who wants to know about the translation status of any of my packages - whether upstream or Debian. POT files will be uploaded for each package and each message (should have) links to the VCS where the current PO files can be found.

I could post continual messages to the debian-i18n mailing list but this way I can put specific content (like messages about which parts of the project I'm changing) as well as updates on which translations are lagging etc. I hope it'll be easier for translators to find the relevant data instead of having to search through the debian-i18n mailing list archive.

I'm not sure about comment handling within the i18n section at this stage. I think I'd prefer to get input via normal email and bug reports.

This, more general, i18n blog category will remain on my main blog and syndicated with planet. I'll use this for posts that aren't specific to one particular project.

gnome-doc-utils has a fairly idiosyncratic layout - help/package.pot, help/lang/lang.po - which doesn't suit things like podebconf-report-po that I want to use to call for translations and updates.

This little snippet creates the relevant directory from a gnome-doc-utils Makefile.am:

report-po: Makefile
if [ ! -d po ]; then \
mkdir po/ ; \
fi
cp $(PACKAGE)-help.pot po
cd po ; \
for l in $(DOC_LINGUAS); do \
if [ ! -L $$l.po ]; then \
ln -s ../$$l/$$l.po . ; \
fi; \
echo -n $$l: ; \
msgfmt -c --statistics $$l.po; \
rm -f messages.mo; \
done


Also, I thought gnome-doc-utils updated the PO files automatically but I couldn't get it to do it, so I've got this snippet too:


check: update-po

update-po: Makefile
for l in $(DOC_LINGUAS); do \
xml2po -u $(top_srcdir)/help/$$l/$$l.po $(top_srcdir)/help/C/$(PACKAGE).xml; \
done