Solution to prc-tools on AMD64 and other 64-bit machines
Tags: google, linux, servers, VMware
If you’re like me, moving to the latest 64-bit hardware has made an ENORMOUS difference in my productivity. I’ve moved all of my personal machines and servers to AMD64/4600+ machines with a minimum of 4gb of RAM.
The problem is that not everything moves over so seamlessly; case in point: prc-tools. The problem with prc-tools not functioning on AMD64 has nothing to do with John Marshall, the maintainer of prc-tools… John is a great person, once you get used to how he works and how he expects bug reports and submissions <ducking from johnm’s swing>
I use prc-tools in several-dozen hourly cron jobs to build Palm software for projects like Plucker and several others, and I wanted to decommission the existing AMD32 machine that was doing those builds up to this point to migrate everything to the faster, less power-hungry AMD64 machines.
I didn’t really want to have to move everything to the new AMD64 machines EXCEPT this one AMD32 machine powered up @400W 24×7 just to build Palm software. That would cost me too much money each month in power costs for a server which isn’t really being used for anything other than cranking out hourly builds of Palm software.
prc-tools is a series of patches to the mainline gcc compiler that we’re all used to using on Linux and other POSIX systems like BSD and Solaris. The problem is that the prc-tools patches are mated to gcc-2.95, which was released back in July 31 of 1999. 64-bitness didn’t even exist back in 1999, 8 years ago.
I’m also not the only one with this problem.
I started patching up gcc’s configuration files to detect 64-bit procs a bit better, but it dead-ended quite early. The autoconfiscation process doesn’t even detect the architecture via config.guess. Dropping in a more-recent config.guess and rewrapping configure.ac helped a little bit, but it died further on in the process. Iterative fixes got it quite far, but eventually I had to dive into gcc itself to patch it, and that’s an area I leave to more-seasoned experts than myself.
./configure [...] Configuring for a x86_64-unknown-linux-gnu host. Invalid configuration `x86_64-unknown-linux-gnu': machine `x86_64-unknown' not recognized Invalid configuration `x86_64-unknown-linux-gnu': machine `x86_64-unknown' not recognized Unrecognized host system name x86_64-unknown-linux-gnu.
The deeper I went, the deeper into the innards of gcc the problems went. I’m not the person to go fixing an 8-year old compiler to work with current 64-bit processors, so I had to come up with a different approach to building the software.
I went through several dozen different ideas, including running gcc in Wine, using an older Linux with native gcc-2.95/prc-tools inside VMware and rsync‘ing the compiled .prc files back out and many other things. Nothing was really scalable or workable in the long term.
Enter chroot and schroot!
Since I run Debian on the build servers, it already had most of the tools I was going to need to build the chroot jails.
The first thing you’ll want to do is install chroot and schroot and set a path for your chroot. In my case, I’m using /var/chroot/. Inside there, are several dozen directories for various chroots I use. In this case, we’ll build one called ‘debian_32bit’.
apt-get install chroot schroot
Debian includes a tool called ‘debootstrap’, which lets you bootstrap a basic Debian system anywhere, and you can execute it as follows:
debootstrap --arch i386 sarge /var/chroot/debian_32bit http://ftp.debian.org/debian/
As it begins working, you’ll see it retrieving packages, unpacking them and building a full Debian 32-bit, i386 system in /var/chroots/debian_32bit for you. This will take awhile, so you might want to be patient.
I: Retrieving Packages I: Validating Packages I: Resolving dependencies of required packages... I: Resolving dependencies of base packages... I: Found additional base dependencies: liblzo2-2 I: Checking component main on http://ftp.debian.org/debian... I: Retrieving adduser I: Validating adduser I: Retrieving apt I: Validating apt I: Retrieving apt-utils [...]
Let those install. By this point, depending on your bandwidth and disk speed, your i386 Debian chroot should be complete.
If your bootstrap above fails with the following errors:
W: Failure while configuring base packages. W: Failure while configuring base packages. W: Failure while configuring base packages. W: Failure while configuring base packages. W: Failure while configuring base packages.
You need to make sure you’re trying to build a chroot using Debian’s ‘sarge’ flavor, and not ‘sid’ or ‘unstable’. You can always install ‘sarge’ in the chroot, then update your sources.list and upgrade to ‘sid’ when you’re done. For now, as of the date of this post, Sid is too unstable to use for building Palm software. Stick with Sarge for now.
A successful install via debootstrap should end like this:
[...] I: Extracting util-linux... I: Extracting whiptail... I: Extracting zlib1g... I: Installing core packages... I: Unpacking required packages... I: Configuring required packages... I: Installing base packages... I: Base system installed successfully.
We should test that it works first, of course:
neptune:~# cd /tmp neptune:/tmp# chroot /var/chroot/debian_32/ neptune:/#
You should now be returned to the prompt, with no errors. Your chroot jail needs some tweaking before you’ll be able to use it for building Palm software.
See if you can ping something simple from inside the chroot, like google.com. If you can, you’re almost done!
neptune:/# ping -c2 google.com PING google.com (72.14.207.99) 56(84) bytes of data. 64 bytes from eh-in-f99.google.com (72.14.207.99): icmp_seq=1 ttl=242 time=230 ms 64 bytes from eh-in-f99.google.com (72.14.207.99): icmp_seq=2 ttl=242 time=189 ms --- google.com ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1004ms rtt min/avg/max/mdev = 189.937/210.260/230.584/20.328 ms
We need to update the default sources.list file to get the latest version of Sarge’s Debian packages in the chroot (remember, NOT Sid!):
vi /etc/apt/sources.list
Change the line that reads:
deb http://ftp.debian.org/debian/ sarge main
To read as follows (add the bolded items):
deb http://ftp.debian.org/debian/ sarge main contrib non-free
Now we update the package listing:
neptune:/# apt-get update Hit http://ftp.debian.org sarge/main Packages Hit http://ftp.debian.org sarge/main Release Get:1 http://ftp.debian.org sarge/contrib Packages [89.0kB] Get:2 http://ftp.debian.org sarge/contrib Release [85B] Get:3 http://ftp.debian.org sarge/non-free Packages [120kB] Get:4 http://ftp.debian.org sarge/non-free Release [86B] Fetched 209kB in 4s (45.1kB/s) Reading Package Lists... Done
Install some necessary tools:
neptune:/# apt-get install apt-file prc-tools pilrc Reading Package Lists... Done Building Dependency Tree... Done The following extra packages will be installed: libapt-pkg-perl libconfigfile-perl libglib1.2 libgtk1.2 libgtk1.2-common libx11-6 libxext6 libxi6 perl perl-modules prc-tools-arm prc-tools-m68k prc-tools-utils xfree86-common xlibs-data Suggested packages: ssh libterm-readline-gnu-perl libterm-readline-perl-perl pilot-link kpilot x-window-system-core x-window-system Recommended packages: perl-doc prc-tools-doc The following NEW packages will be installed: apt-file libapt-pkg-perl libconfigfile-perl libglib1.2 libgtk1.2 libgtk1.2-common libx11-6 libxext6 libxi6 perl perl-modules pilrc prc-tools prc-tools-arm prc-tools-m68k prc-tools-utils xfree86-common xlibs-data 0 upgraded, 18 newly installed, 0 to remove and 0 not upgraded. Need to get 19.8MB of archives. After unpacking 60.7MB of additional disk space will be used. Do you want to continue? [Y/n]
Let that process finish. While its unpacking files, you may messages that LC_ALL is not set. They’ll look like this:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
To fix these, just add the following to root’s .bashrc INSIDE the chroot:
export LC_ALL=C
From here, assuming you have the latest Palm SDK installed and configured via ‘palmdev-prep’ inside the chroot jail, you should be ready to build software using it.
To build Palm software INSIDE the chroot jail FROM OUTSIDE the chroot jail, you’ll need schroot (we installed that earlier). To run chroot binaries from outside the chroot, do something like this:
# schroot -c debian_32 -p /usr/bin/file /bin/bash I: [debian-ia32-556d548b-14eb-4bf5-9cd6-cce96d170171 chroot] Running command: [/usr/bin/file /bin/bash] /bin/bash: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.4.1, dynamically linked (uses shared libs), stripped
That’s it. If you want to build software interactively, just chroot to your new chroot jail in /var/chroot/debian_32. If you want to do it in an automated fashion, you can put it all in a script INSIDE the chroot jail, and run schroot against that script.
Easy!