Sunday, May 27, 2012

Installing Debian 6 onto a VIA Epia 5000-L and IDE HDD

First boot with GUI
LXDE + Debian 6.0.5 on 10-year-old Epia 5000-L motherboard
I have a 10-year-old mini-ITX motherboard that I'm cobbling with a few parts into a useful system.

In this post, I discussed testing the old motherboard, and getting it to boot.

Since it arrived without any storage attached, in this post I discussed the hurdles getting it to boot Debian 6 from a USB drive. That wasn't easy, since the motherboard predated most forms of USB-boot, and the BIOS didn't support USB-HDD, the current standard for bootable USB.

Next, in this post, I discussed the adventure of using that USB boot to bootstrap a Debian install into an IDE-connected SD card. This didn't work out - one bootable IDE connector was defective, and a second worked...but wasn't bootable. I may return to this, since 3.5-inch HDDs are enormous - almost as big as the motherboard!

So I finally broke down and spent a few dollars on a 40GB HDD. This has the big advantage that the HDD can be partitioned for experimentation. Installing to the HDD is essentially the same process as installing to the SD card.




Lessons Learned

Hooked up:
Left: PS/2 Mouse, PS/2 Keyboard, VGA Monitor,
           USB Wireless-g dongle
Bottom: Power button
Right: Power to HDD, Power from wall, IDE to HDD
1) The -i486 architecture is important. This motherboard is too old to support cmov, cx8, and pae. All three of those are in the latest Ubuntu kernels. cmov and cx8 are part of the i686 instruction set, so specifying a i486 kernel gets rid of them. pae is a new default since Debian 6.0. So a Debian 6.0 kernel compiled for x486 will be useful on the motherboard...until I get better at compiling my own kernels.

2) Debootstrap does not include the wireless-tools package, so a bootstrapped system won't have wireless access. Adding wireless-tools is trivial if you remember to do it: During the chroot session when installing the kernel and grub and configuring the accounts and fstab and hostname, install wireless-tools and configure /etc/network/interfaces too.




For the first HDD experiment, I installed Debian 6 + LXDE onto a partition of the HDD.

Startup test: 45 Seconds from power-on to text-login prompt, 95 seconds to power-on to GDM login prompt.

Next tests: I want to try Debian Live for running the whole system in RAM. And I want to try modding the latest Ubuntu kernel to remove those three features and try Unity on it.

Saturday, May 12, 2012

Ubuntu Brainstorm and UDS-Q

Greetings, fellow Brainstormers,

I have, as usual, been monitoring the latest Ubuntu Developer Summit from afar. And I'm pleased to see a lot of Brainstorm Ideas popping up in various sessions.

That's one of Brainstorm's great purposes - to provide a range of ideas ready for discussion by developers.

- If you were UDS participant, please let us know how Brainstorm can help you better next time. Even if you didn't use Brainstorm at all, we would love to know why you didn't consider it as a preparatory source. There is always room for improvement, and relevance to you is one of our big goals! Feel free to send an e-mail to the moderator list, or e-mail me privately, or leave a comment on this blog post.

- If you weren't a UDS participant, even a remote one like me, set aside some time to plow through the various session records. A lot of decisions get made in those rooms! And some -usually the ones important to you- of them don't get discussed widely afterward. If you see a project that interests you, get involved! Ubuntu grows from community contributions and participation. Brainstorm Ideas are merely the first step.

Here's a sampling of the Ideas I saw in action on just Monday! Many are probably from non-Brainstorm (parallel) origin...but I don't care about the origin, I care that it got talked about. I care that it improves Ubuntu.

Sound Theme, especially customizing:
     http://summit.ubuntu.com/uds-q/meeting/20655/sound-theme/
     http://brainstorm.ubuntu.com/idea/3665/

Embedded Device images and development:
     http://summit.ubuntu.com/uds-q/meeting/20498/foundations-q-embedded-rootfs/
     http://brainstorm.ubuntu.com/idea/21841/

Hardware Testing:
     http://summit.ubuntu.com/uds-q/meeting/20383/hardware-q-distributed-testing/
     http://brainstorm.ubuntu.com/idea/19151/
     http://brainstorm.ubuntu.com/idea/24693/
     http://brainstorm.ubuntu.com/idea/29704/
     http://brainstorm.ubuntu.com/idea/21604/
     http://brainstorm.ubuntu.com/idea/29367/
     ...and several more.

Cheers, and keep talking!

Tuesday, May 1, 2012

More about Bug 931181



Back in this post, I discussed Launchpad Bug #931181, a SHA1 implementation mistake that broke Debian/Ubuntu Simutrans multiplayer support. So Ubuntu and Debian players could share a game, but cold not connect to any other non-Debian server.

16 Feb 2011, the original patch that broke network play was applied.
12 Feb 2012, I reported the bug.
19 Feb 2012, a Simutrans dev noticed that the issue seemed to be endianness in the SHA1 hash used to communicate paksets between client and server.
21 Mar 2012, Version 111.2.2 of Simutrans is released
29 Apr 2012, the Debian package maintainer created a patch to fix the bug, and applied it to the 111.2.2 upload.

Today, Debian sid and testing (wheezy) and Ubuntu 12.10 have the patched version...but it still doesn't work.

For a clean set of tests, I installed a Debian 7 (testing) VM.

1) Test the Packages: Install the stock Debian Testing packages of simutrans 111.2.2 and simutrans-pak64 111.2 right out of the repos.

# apt-get install simutrans simutrans-pak64
$ simutrans -debug 3 &> s-package.log


Result: Fails to connect to the a.64.simutrans.entropy.me.uk server game. The UI claims the same pakset incompatibility as before. Here's the logfile.

As a check, let's take a look at one SHA1 hash:

PostOffice -> sha1 = 263C0CB8C858272A12FF54FC64F57292AF8ADAB7



2) Test the manual install: Install the upstream Simutrans versions of Simutrans 111.2.2 and Pak64 from the release announcement on the upstream website.

$ mkdir manual
$ cd manual
$ wget http://sourceforge.net/projects/simutrans/files/simutrans/111-2-2/simulinux-111-2-2.zip
$ wget http://sourceforge.net/projects/simutrans/files/pak64/111-2/simupak64-111-2.zip
$ unzip simulinux-111-2-2.zip
$ unzip simupak64-111-2.zip
$ chmod -R +Xr simutrans/pak       # This fixes the pakset's permissions
$ simutrans/simutrans -debug 3 &> ~/s-manual.log

Result: Succesful connection to the a.64.simutrans.entropy.me.uk server game. Here's the logfile.

Let's compare SHA1 hashes to see if the original issue (Launchpad Bug #931181, SHA1 endinaness ) has been fixed, or if this is a separate issue...

PostOffice -> sha1 = 263C0CB8C858272A12FF54FC64F57292AF8ADAB7

The hashes are the same. The original SHA1 issue has been fixed. There is a different bug or misconfiguration.

3) Is the Pakset good? Let's eliminate a possibility -- the packaged Pak64. Use the manually-installed Simutrans from #2 with the packaged Pak64 from #1.

$ simutrans/simutrans -debug 3 -object /usr/share/games/simutrans/pak &> ~/s-crossover.log

Result: Succesful connection to the a.64.simutrans.entropy.me.uk server game. Here's the logfile. The packaged pakset is not the problem. It's the correct version, and compiled properly. The problem is in Simutrans itself somewhere....

How to build a Simutrans .deb package manually

Simutrans, as I've written before, is a fun game.

Currently it's in active development, with new features and bug fixes arriving several times each year.  But Ubuntu's repository system is based on six-month snapshots; it cannot keep up with the pace of Simutrans releases.

That causes a big problem for multiplayer games: The server requires clients to be on a similar pakset and release, and the server operators usually keep up with the releases. So Ubuntu users are locked out of multiplayer games - the versions in the Ubuntu repos are simply too old.

So I want roll my own .deb of the latest version of Simutrans. This is the first step - eventually a PPA of the backported latest release would be cool.



1) Create a VM for the build environment. For this, I used a VM of my current install, Ubuntu 11.10. But you can use anything back to Debian 6, of course.

The VM serves several purposes. It provides the appropriate build environment for each version of Ubuntu. It also provides a safe place to test the final package installation. And, of course, it protects my main system.

Beyond the basic setup, you should probably clean up the desktop shortcut bar; you don't need Software Center or LibreOffice, but you do need Terminal on there. You should also set Software Sources to automatically download-and-apply security updates, you can safely ignore all non-security updates.

Also, install the following packages needed for a useful Simutrans build environment:
$ sudo apt-get install build-essential devscripts debhelper libsdl1.2-dev 
libsdl-mixer1.2-dev zlib1g-dev imagemagick libpng-dev libbz2-dev libssl-dev subversion

Do NOT install the simutrans package from the Ubuntu repositories. If you do install it, for example to confirm that it works, remove it.


2) Prepare the working directory for Simutrans and download the sources

First, let's set up the Simutrans working directory

~$ mkdir Simutrans_Development
~$ cd Simutrans_Development
~/Simutrans_Development$ wget http://ftp.us.debian.org/debian/pool/main/s/simutrans/simutrans_111.2.2-1.debian.tar.xz 
~/Simutrans_Development$ tar -xvf simutrans_111.2.2-1.debian.tar.xz 

Increment the version number by editing the changelog in each Debian directory.

~/Pak64_Development$ cd ~/Simutrans_Development
~/Simutrans_Development$ dch -v 111.2.2-0ppa1

simutrans (111.2.1-0ppa1) oneiric; urgency=low

  * New upstream release

 -- me <me@example.com>  Wed, 25 Apr 2012 07:23:32 -0500

Edit the Debian rules file to reflect the most recent upstream release, then download a new source code tarball, uncompress the tarball, and copy the debian folder into the new folder.

~/Pak64_Development$ cd ~/Simutrans_Development
~/Simutrans_Developmen$ nano debian/rules

SVNREV  = 5583
VERSION = 111.2.2


~/Simutrans_Development$ make -f debian/rules get-orig-source
tar -xvf simutrans_111.2.2.orig.tar.zx
cp -r debian/ simutrans-111.2.2/

Finally, a progress check: The directory should look something like this:

~$ ls Simutrans_Development/
debian             simutrans_111.2.2-1.debian.tar.xz
simutrans-111.2.2  simutrans_111.2.2-1.orig.tar.xz


3) Prepare the working directory for Pak64 and download the sources

~$ mkdir Pak64_Development
~/Simutrans_Development$ cd ~/Pak64_Development
~/Pak64_Development$ wget http://ftp.us.debian.org/debian/pool/main/s/simutrans-pak64/simutrans-pak64_111.2-1.debian.tar.xz
~/Pak64_Development$ tar -xvf simutrans_111.2-1.debian.tar.xz

Increment the version number by editing the changelog in each Debian directory.

~/Simutrans_Development$ cd ~/Pak64_Development
~/Pak64_Development$ dch -v 111.2-0ppa1

simutrans-pak64 (111.2-0ppa1) oneiric; urgency=low

  * New upstream release

 -- me <me@example.com>  Wed, 25 Apr 2012 07:23:32 -0500

Edit the Debian rules file to reflect the most recent upstream release, then download a new source code tarball, uncompress the tarball, and copy the debian folder into the new folder.

~/Simutrans_Development$ cd ~/Pak64_Development
~/Pak64_Development$ nano debian/rules

SVNREV  = 811
VERSION = 111.2
UPSTREAM_SVNREV = 810M # see README.source

~/Pak64_Development$ make -f debian/rules get-orig-source
tar -xvf simutrans-pak64_111.2.orig.tar.xz
cp -r debian/ simutrans-pak64-111.2/

Finally, a progress check: The directory should look something like this:

~$ ls Pak64_Development/
debian                                 simutrans-pak64-111.2
simutrans-pak64_111.2-1.debian.tar.xz  simutrans-pak64_111.2.orig.tar.xz


4) Build Simutrans

We're ready to try building the package. This is where the troubleshooting really starts.

~$ cd Simutrans_Development/simutrans-111.2.2
~/Simutrans_Development/simutrans-111.2.2$ debuild -us -uc

First  Problem: debhelper 9.
 
dpkg-checkbuilddeps: Unmet build dependencies: debhelper (>= 9)
dpkg-buildpackage: warning: Build dependencies/conflicts unsatisfied; aborting.
dpkg-buildpackage: warning: (Use -d flag to override.)
debuild: fatal error at line 1348:
dpkg-buildpackage -rfakeroot -D -us -uc failed

The debhelper package was originally version 7.x. According to the git log, it incremented up to 9 in March 2012, long after the build environment (that's why we review the log!). We can try the -h flag to avoid this...or unapply the patch that incremented debhelper.
 
~/Simutrans_Development/simutrans-111.2.1$ debuild -us -uc -d

Second Problem: Lintian errors.
 
Now running lintian...
W: simutrans source: newer-standards-version 3.9.3 (current is 3.9.2)
W: simutrans: binary-without-manpage usr/games/simutrans-nettool
Finished running lintian.

Lintian errors are a good sign - it means everything else worked! Both warnings in this case can be safely ignored...though a good citizen would also file bugs with patches to fix those warnings.

Take a look at the Simutrans_Development directory now. See all those ready-to-install deb packages?
 
ian@VM-11-10:~/Simutrans_Development$ ls
debian                                simutrans_111.2.2-0ppa1_i386.deb
simutrans-111.2.2                     simutrans_111.2.2.debian.tar.gz
simutrans_111.2.2-1.debian.tar.xz     simutrans_111.2.2.orig.tar.xz
simutrans_111.2.2-1.dsc               simutrans-data_111.2.2-0ppa1_all.deb
simutrans_111.2.2-0ppa1_i386.build    simutrans-makeobj_111.2.2-0ppa1_i386.deb
simutrans_111.2.2-0ppa1_i386.changes

Do not install the simutrans_111.2.1-1_i386.deb or simutrans-data_111.2.1-1_all.deb packages yet. If you try, they will install a pak64 package from the repositories instead of the one you are about to build.

5) Install makeobj

makeobj is a package (simutrans-makeobj_111.2.1-1_i386.deb) that is built with the Simutrans game, and used in turn to create the paks. Install the deb that was just created before building pak64.

~/Simutrans_Development$ sudo dpkg -i simutrans-makeobj_111.2.2-0ppa1_i386.deb 

6) Build Pak64


We're ready to try building the pak package.

~$ cd Pak64_Development/simupak-64
~/Pak64_Development/simupak64$ debuild -us -uc

Problem: Makefile patches failed

The file debian/patches/buildsystem has several patches that get applied to the makefile. As the makefile changes over time, the patch program may not find the right place to apply some patches. Those changes to the patch must be applied manually.

patching file Makefile
Hunk #2 FAILED at 46.
Hunk #3 succeeded at 65 (offset 2 lines).
Hunk #4 FAILED at 84.
2 out of 4 hunks FAILED -- saving rejects to file Makefile.rej

Hunk #2 was a straightforward change to one line (adding the word "zip").
Hunk #4 had the old version number instead of the new.

Upon the second try, the pak64 build was successful. You can see the deb package in the directory listing:

~$ ls Pak64_Development/
debian                                     simutrans-pak64_111.2.1-1.dsc
simutrans-pak64_111.2-0ppa1.debian.tar.xz  simutrans-pak64_111.2.1-1_i386.build
simutrans-pak64-111.2                      simutrans-pak64_111.2.1-1_i386.changes
simutrans-pak64_111.2-0ppa1_all.deb        simutrans-pak64_111.2-1.orig.tar.xz
simutrans-pak64_111.2-1.debian.tar.xz      simutrans-pak64_111.2-1.orig.tar.xz

7) Test Install

When installing package-by-package, make sure to install in the right order!

sudo dpkg -i Pak64_Development/simutrans-pak64_111.2-0ppa1_all.deb
sudo dpkg -i Simutrans_Development/simutrans-data_111.2.2-0ppa1_all.deb 
sudo dpkg -i Simutrans_Development/simutrans_111.2.2-0ppa1_i386.deb


I fire up Simutrans...and it works!

8) Removing Simutrans

When the test is complete, and I want to remove simutrans:
sudo dpkg -r simutrans
sudo dpkg -r simutrans-data
sudo dpkg -r simutrans-pak64