random neuron misfires

Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/ubuntu/+source/command-not-found
Please include the following information with the report:
No module named CommandNotFound
Traceback (most recent call last):
  File "/usr/lib/command-not-found", line 10, in 
    from CommandNotFound import CommandNotFound
ImportError: No module named CommandNotFound
Python version: 2.5.2 final 0

So the command /usr/lib/command-not-found could not find a required Python module called CommandNotFound, and instead crashed. Well that’s a well-thought out tool.

I was recently building a copy of Ice 3.0.1 from ZeroC on x86_64 and ran into the following error message fairly early on:

/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.2.3/libstdc++.so when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.2.3/libstdc++.a when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.2.3/libstdc++.so when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.2.3/libstdc++.a when searching for -lstdc++
/usr/bin/ld: cannot find -lstdc++
collect2: Error: ld returned 1 exit status
make[2]: *** [../../lib/libIceUtil.so.3.0.1] Error 1

But I clearly had libstdc++ version 5 and 6 installed:

# COLUMNS=50 dpkg -l | grep libstdc
ii  libstdc++5     1:3.3.6-15ubun The GNU Standard C++ Library v3
ii  libstdc++6     4.2.3-2ubuntu7 The GNU Standard C++ Library v3
ii  libstdc++6-4.2 4.2.3-2ubuntu7 The GNU Standard C++ Library v3 (development

This works on i386 and 32-bit Linux, so I sought out a fix, and the fix was simple; install g++-multilib:

$ sudo apt-get install g++-multilib
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  g++-4.2-multilib gcc-4.2-multilib lib32gomp1
Suggested packages:
  lib32stdc++6-4.2-dbg lib32mudflap0
The following NEW packages will be installed:
  g++-4.2-multilib g++-multilib gcc-4.2-multilib lib32gomp1
0 upgraded, 4 newly installed, 0 to remove and 70 not upgraded.
Need to get 1017kB of archives.
After this operation, 4854kB of additional disk space will be used.
Do you want to continue [Y/n]? y

After doing this, you may receive the following error:

/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.2.3/../../../libbz2.so when searching for -lbz2
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.2.3/../../../libbz2.a when searching for -lbz2
/usr/bin/ld: skipping incompatible /usr/bin/../lib/libbz2.so when searching for -lbz2
/usr/bin/ld: skipping incompatible /usr/bin/../lib/libbz2.a when searching for -lbz2
/usr/bin/ld: skipping incompatible /usr/lib/libbz2.so when searching for -lbz2
/usr/bin/ld: skipping incompatible /usr/lib/libbz2.a when searching for -lbz2
/usr/bin/ld: cannot find -lbz2
collect2: Error: ld returned 1 exit status
make[2]: *** [../../lib/libIce.so.3.0.1] Error 1

This is due to a similar issue, and the libraries you want are in /usr/lib64/, not /usr/lib.

The fix for this is seems to be buried in Cyrillic on this thread. The fixes proposed there did not work for me, so I had to hack and slash the Make.rules.Linux file and the various Makefile targets below to get that working.

I woke up this morning to very slow response time on my server, and decided to check the statistics. I graph these things with a great deal of detail so I can see precisely when it happened and begin narrowing down where I need to go to fix it or report it upstream.

In this case, my incoming connections went from under 500/second to well over 3,000/second. Owch!

You can see the “wall” of traffic growing from our normal traffic rate to this enormously-increased rate:

I checked all of the services, logs and protocols and didn’t see anything out of the ordinary. I started shutting down services one at a time and regenerated the graphs, to see if I could see any change.

One thing I noticed was that I had poppassd open on the public port. Not a huge problem, but it was something that was unnecessary on the public interface. I locked that down with iptables:

$IPT -A INPUT -s ! 127.0.0.1 -d ! 127.0.0.1 -p tcp -m tcp --dport 106 -j DROP

But as I looked further, I noticed even more:

netstat -tulpn | grep LISTEN

This showed that I had Squid listening on the public interface as well (0.0.0.0:3128). I jumped to the squid logs and was shocked to see that they were scrolling so fast that I couldn’t even read them. Ut oh!

Apparently some enterprising young spammer found my squid instance and decided to try to hijack it for his own needs. It was already locked down internally in my squid.conf to restrict use from only my block of IPs, but he was hammering it with 8,466 separate IPs trying to use it to send spam on port 25.

# cat access.log* | cut -b20-300 | grep ':25' | perl -lne 'print /((?:\d+\.){3}\d+)/' | sort | uniq  | wc -l
8464

Damn! There goes a few gigabytes of bandwidth that were eaten in the last 11 hours while I was sleeping.

I locked that down in a similar fashion:

$IPT -A INPUT -s ! 127.0.0.1 -d ! 127.0.0.1 -p tcp -m tcp --dport 3128 -j DROP

A bit more poking around with nmap, netstat, Webmin, HotSanIC and other tools allowed me to lock down some other services that incorrectly bind to the public interface and not the internal interface.

The result is that we’re back to normal:

One last piece needed my attention. Because this was an active spam attack, propagated using the IP of my server as a vector, I had to make sure to check my mail logs and delist myself from the various RBLs who had listed me as a spammer for sending out 43,745 separate spam attempts through my IP in a matter of hours.

SpamCop originally listed me, but I corrected that, and a few others. I also reported it to my provider so they can be sure to keep a closer eye on it.

You can see the drop-off on the far right of the last two graphs above and in the traffic graph below.

Problem solved.

ThinkPad laptops include a proprietary interface called SMAPI BIOS (System Management Application Program Interface) which provides some hardware control functionality that is not accessible by other means.

This driver exposes some features of the SMAPI BIOS through a sysfs interface. It is suitable for newer models, on which SMAPI is invoked through IO port writes. Older models use a different SMAPI interface using “thinkpad” module from the “tpctl” package.

Unfortunately, out of the box it does not build on Ubuntu or Debian systems, because of the way those Linux distributions structure their kernel source and headers in /lib/modules/…/ and /usr/src/…/

It is a simple one-line patch to fix the tp_smapi Makefile to get these package to build on stock Ubuntu/Debian systems:

--- Makefile.orig       2008-03-21 14:07:34.604072067 -0400
+++ Makefile    2008-03-21 14:13:07.669091731 -0400
@@ -3,7 +3,7 @@
 X:=$(shell false)
 KVER        := $(shell uname -r)
 KBASE       := /lib/modules/$(KVER)
-KSRC        := $(KBASE)/source
+KSRC        := /usr/src/linux-headers-$(KVER)/
 KBUILD      := $(KBASE)/build
 MOD_DIR     := $(KBASE)/kernel
 PWD         := $(shell pwd)

Basically you need to tell the KSRC variable that the kernel source is in /usr/src/linux-headers-1.2.3-foo/ and not in /lib/modules/1.2.3-foo/source/.

If you want to build tp_smapi with HDAPS support (and you probably do, if you’re on a laptop), then you’d build it as follows:

make load HDAPS=1
make install HDAPS=1

Depending on your particular configuration, you can then put tp_smapi and hdaps in your /etc/modules file to load at boot-time.

I personally use tp_smapi to set the charging and discharging thresholds of my Thinkpad batteries (3 separate Thinkpads at this point; T61p, T42p and X61s).

$ sudo echo 30 > /sys/bus/platform/drivers/smapi/smapi/BAT0/start_charge_thresh
$ sudo echo 90 > /sys/bus/platform/drivers/smapi/smapi/BAT0/stop_charge_thresh

The values above tell tp_smapi to begin charging the battery when it reaches BELOW 30% of its total capacity, and to stop charging the battery when it reaches ABOVE 90% of its total capacity.

There’s a lot of other things to play with in here, and here is a table showing some of those options, courtesy of ThinkWiki.

Here’s what my battery details are on the T61p as I type this:

$ cat /proc/acpi/battery/BAT0/info 
present:                 yes
design capacity:         84240 mWh
last full capacity:      86790 mWh
battery technology:      rechargeable
design voltage:          10800 mV
design capacity warning: 4339 mWh
design capacity low:     200 mWh
capacity granularity 1:  1 mWh
capacity granularity 2:  1 mWh
model number:            42T4511
serial number:           21169
battery type:            LION
OEM info:                SANYO

As I continue to shave and optimize my hybrid working environment to gain more productivity out of the limited hours I have in every day, I’ve been working on consolidating my calendaring needs across the three platforms I currently use (soon to be 4):

1. Linux Development environment (I do everything on Linux)
2. Windows Financial environment (I use this for Microsoft Money, some Office 2007 work and several Palm conduits and plugins that don’t have Linux equivalents)
3. Web
4. Mac OS X (not yet introduced into my workflow)

I have a lot of calendar items that need to be in various places so I can get to them when I’m working on that platform. This means when I’m on Linux, I need to see my calendar in Evolution and J-Pilot.

When I’m in Windows, I need to see the same calendar in Microsoft Outlook.

When I’m in a browser or not on my native Linux or Windows machines, I need to be able to see my calendar in Google Calendar.

All three platforms must reflect the same EXACT data, without being out of step with any other. So far, this is working very well, using my Palm Treo680 as the middle-man delivery mechanism.

Recently it came to my attention that I need to have clients see where my free/busy time is, and start booking their own slots of my time into the free spots that I haven’t personally blocked out yet.

To do this, I’ve had to leverage and expose my Google Calendar to the public.

On Windows, I’m using a tool called “CompanionLink for Google Calendar” to get my calendar data from Treo → Outlook → Google Calendar. It’s non-free, but it had a decent trial period and its usefulness won me over, so I registered it.

But there was no obvious way to get Evolution to read back those calendars, so I could see when external people (i.e. friends, clients) were adding things to my calendar to book my time for them.

Enter “evolution-webcal”, a seldom-discussed binary that lives in /usr/lib/evolution-webcal/ on most GNU systems (/opt/gnome/lib/evolution-webcal/ on SuSE)!

Basically all you have to do to get Evolution to read in your Google Calendar calendar files is the following (all on one line):

$ /usr/lib/evolution-webcal/evolution-webcal \
http://www.google.com/calendar/ical/p%23weather%40group.v.calendar.google.com/public/basic.ics

The way you get the .ics calendar URL you see above, is by:

1. Log into your Google Calendar account
2. Click the little “V” chevron to the right of your target calendar in the “My Calendars” block on the left sidebar
3. Select “Calendar Settings” from the popup menu
   
4. Towards the bottom of the “Calendar Details” tab on the right side, you’ll see two sections: Calendar Address and Private Address. Click the [ICAL] item on either of these (choose wisely)
   
5. Cut and paste the URL provided in the popup window into your evolution-webcal command above.

That’s it. Now Evolution will have your new Google Calendars listed under the “On the Web” category in the Calendar’s view.

Now I can see the same thing in Evolution as I see in Outlook as I see in Google Calendar, without any discrepancies. Whew!

If Mozilla Thunderbird is your thing, there’s an extension to Thunderbird called “Provider for Google Calendar” that allows you to read/write to your Google Calendars from within Thunderbird.

It’s a start. Now where did I put those extra 32 hours I need in every day again?

If you’re using the latest version of VMWare Workstation, and try to configure it to boot to an ISO image file, VMWare will crash with the following error message:

/usr/lib/vmware/bin/vmware: symbol lookup error: 
/usr/lib/gtk-2.0/2.10.0/filesystems/libgnome-vfs.so: 
undefined symbol: g_get_user_special_dir

To solve this problem, simply launch VMWare with the following variable in your environment:

VMWARE_USE_SHIPPED_GTK=no vmware

Now you’ll get proper, working VMWare Workstation, using your native GTK+ on the system.

I’ve been building kernels for a very long time. In fact, I even wrote my own kernel HOWTO describing how to do it.

When I install a new Linux distribution from original media, the first thing I do, is replace the stock kernel with my own custom build, which is optimized for my own environment (tuning HZ, removing unused drivers, patching some other interfaces, etc.).

When I moved from Debian to Ubuntu, building kernels became a problem. All of the kernels I’d try to build from the upstream source would fail to complete a boot. At first, I thought it was something with mkinitrd(1). I tried to fix that and it would still fail to complete a boot.

So I started looking for some better options, and came up with this very small HOWTO:

If you’re like me, moving to the latest 64-bit hardware has made an ENORMOUS difference in my productivity. I’ve moved all of my personal machines and servers to AMD64/4600+ machines with a minimum of 4gb of RAM.

The problem is that not everything moves over so seamlessly; case in point: prc-tools. The problem with prc-tools not functioning on AMD64 has nothing to do with John Marshall, the maintainer of prc-tools… John is a great person, once you get used to how he works and how he expects bug reports and submissions <ducking from johnm’s swing>

I use prc-tools in several-dozen hourly cron jobs to build Palm software for projects like Plucker and several others, and I wanted to decommission the existing AMD32 machine that was doing those builds up to this point to migrate everything to the faster, less power-hungry AMD64 machines.

I didn’t really want to have to move everything to the new AMD64 machines EXCEPT this one AMD32 machine powered up @400W 24×7 just to build Palm software. That would cost me too much money each month in power costs for a server which isn’t really being used for anything other than cranking out hourly builds of Palm software.

prc-tools is a series of patches to the mainline gcc compiler that we’re all used to using on Linux and other POSIX systems like BSD and Solaris. The problem is that the prc-tools patches are mated to gcc-2.95, which was released back in July 31 of 1999. 64-bitness didn’t even exist back in 1999, 8 years ago.

I’m also not the only one with this problem.

I started patching up gcc’s configuration files to detect 64-bit procs a bit better, but it dead-ended quite early. The autoconfiscation process doesn’t even detect the architecture via config.guess. Dropping in a more-recent config.guess and rewrapping configure.ac helped a little bit, but it died further on in the process. Iterative fixes got it quite far, but eventually I had to dive into gcc itself to patch it, and that’s an area I leave to more-seasoned experts than myself.

Today I decided to fire up the Sony Playstation 3 again to get some more Linux hackery done, and noticed there was an OS update on the PS3 side which adds the new Folding@Home Playstation Client to the default GameOS firmware.

In a word: Nice!

As you can see from the statistics, the PS3 is outperforming all of the other systems out there, by several orders of magnitude. I decided to fire it up and see.

The first thing you notice is the very spiffy interface to the Folding@Home client on the PS3 side… it has a real-time 3D globe with all of the active nodes lit up like little dots of light on the map (which you can rotate/zoom with your game controller), and a real-time protein “carbon-chain” configuration zipping and zooming through its calculations.

I’ve only started a few hours ago, but I’m already almost 1/3 of the way through my first work unit. Its beating my AMD64 machines by about 300%. Its very impressive!

Speaking of AMD64 machines, there’s a small problem: Folding@Home doesn’t provide a client for AMD64. They have Intel and Intel SMP clients, which will run on FreeBSD and OpenBSD, but not without a little help.

On AMD64 Linux, you’ll need to install the ia32 libraries (ia32-libs, appropriately enough, on Debian and Ubuntu). If you don’t, you’ll just get a “File not found” error when you try to run the ‘fah5’ binary. This binary lists itself as:

$ file fah5
fah5: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.0, 
dynamically linked (uses shared libs), for GNU/Linux 2.0.0, stripped

On AMD64 FreeBSD, you can’t run the SMP Linux client, even in Linux emulation, because it outputs a binary called ‘FahCore_78.exe’, which FreeBSD then cannot parse or execute. That binary lists itself as:

 $ file FahCore_78.exe
FahCore_78.exe: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, 
statically linked, stripped

You’ll need to install the client from the port in ‘/usr/ports/biology/linux-foldingathome’ and run it as “FoldingAtHome”, which (oddly-enough), tries to write to /usr/local/share/foldingathome/. This means if you install this as root from ports (which you probably will), you won’t be able to write to this path as a normal user. You can fix that with some chown(1) mojo, or just run the client as root. Your choice.

But there’s a downside to all of this wonderful Folding@Home CPU crunching… power consumption.

As Carl Nelson details, the PS3 running 24×7 crunching WU for Folding@Home is eating about 138KWh of power. Where I live and work in New London, CT… we’re paying about $0.13 USD for power. If I was to keep this PS3 running 24×7, that’s going to cost me $17.94 extra per-month, just to fold with the PS3. That doesn’t take into account the other power and servers I have running here (several of them also recently added to the team).

But this thing runs VERY hot, especially when folding.

For that problem, there’s a whole new market of PS3 cooling products. This one seems to be the best I’ve seen so far. 4 separate, parallel cooling fans in a mountable bracket that draws the air out of the PS3 and keeps it cool, prolonging its life. It will eat some more KWh, but its probably worth it in the long run.

The irony here, is that you’re not really being eco-friendly by eating 138KWh per-month of the local power grid, but you ARE helping to cure cancer and other ailments… so it can’t be all bad, right?

In a previous blog entry, I described how to get TrueCrypt working on your Linux machine with ext2/ext3 or XFS filesystems.

If you’ve upgraded your kernel, you may run into trouble with the kernel module. You might see an error like the following:

$ sudo truecrypt /dev/sdi1
Enter password for '/dev/sdi1': 
FATAL: Module truecrypt not found.
Failed to load TrueCrypt kernel module

Here’s how to fix it..

If you fetch the latest TrueCrypt source, unpack it, and go into truecrypt-${VERSION}-source-code/Linux (I’m using v4.3 here), and edit ‘build.sh’ in your favorite editor. In this file, you’ll see a check for the kernel source’s .config, that looks like this:

if [ ! -f "$KERNEL_SRC/.config" ]
then
        if [ -f /proc/config.gz -o -f /boot/config-$KERNEL_VER -o -f /boot/config-$(uname -r) ]
        then
                echo -n "Configure kernel source according to the system configuration? [Y/n]: "
                read A
                if [ -z "$A" -o "$A" = "y" -o "$A" = "Y" ]
                then
                        echo -n "Configuring kernel source in $KERNEL_SRC... "
                        
                        if [ -f /proc/config.gz ]
                        then
                                zcat /proc/config.gz >$KERNEL_SRC/.config || exit 1
                        else
                                if [ -f /boot/config-$(uname -r) ]
                                then
                                        cp /boot/config-$(uname -r) $KERNEL_SRC/.config || exit 1
                                else
                                        cp /boot/config-$KERNEL_VER $KERNEL_SRC/.config || exit 1
                                fi
                        fi
              
                        make -C $KERNEL_SRC oldconfig /dev/null || exit 1
                        echo Done.
                fi
        fi

        if [ ! -f "$KERNEL_SRC/.config" ]
        then
                error "Kernel not configured. You should run make -C $KERNEL_SRC config"
                exit 1
        fi
fi

The outermost test is what we need to remove in this file. Simply comment out the first two lines of that block, as follows (unified diff, you can apply this with patch(1) on your Linux system). I’ve bolded the appropriate lines above. Just comment them out, or apply the diff below to patch this file.

--- build.sh.orig       2007-04-15 14:15:10.000000000 -0400
+++ build.sh    2007-04-15 13:25:18.000000000 -0400
@@ -72,8 +72,8 @@
        exit 1
 fi
 
-if [ ! -f "$KERNEL_SRC/.config" ]
-then
+# if [ ! -f "$KERNEL_SRC/.config" ]
+# then
        if [ -f /proc/config.gz -o -f /boot/config-$KERNEL_VER -o -f /boot/config-$(uname -r) ]
        then
                echo -n "Configure kernel source according to the system configuration? [Y/n]: "
@@ -104,7 +104,7 @@
                error "Kernel not configured. You should run make -C $KERNEL_SRC config"
                exit 1
        fi
-fi
+# fi
 
 if [ ! -d "$KERNEL_SRC/include/asm" ] && grep -q modules_prepare $KERNEL_SRC/Makefile
 then

On most machines, you won’t have the full kernel source tree installed… let’s fix that first.

I’m running 2.6.20 here, and the numbers below will reflect that. Replace your running kernel version with that below:

sudo apt-get install linux-source-2.6.20
cd /usr/src/
sudo tar jxvf linux-source-2.6.20.tar.bz2
sudo ln -s linux-source-2.6.20 linux
sudo cp /boot/config-2.6.20-12-386 /usr/src/linux/.config
cd linux
sudo make oldconfig

Now you need to go back into your TrueCrypt source directory and type ‘sudo sh ./build.sh’, and follow the prompts. It may take a long while to build, but let it run and do its work. You should see the following output (your own output may vary, but it should compile the kernel modules and userland tools cleanly.

$ sudo sh ./build.sh 
Checking build requirements...
Configure kernel source according to the system configuration? [Y/n]: 
Configuring kernel source in /usr/src/linux-source-2.6.20... .config:1390:warning: trying to assign nonexistent symbol SATA_INTEL_COMBINED
.config:3548:warning: trying to assign nonexistent symbol RTL818X
Done.
Building internal kernel modules (may take a long time)... Done.
Building kernel module... Done.
Building truecrypt... Done.

Once it builds correctly, you can run the install script:

desrod@purity:/tmp/truecrypt-4.3-source-code/Linux$ sudo sh ./install.sh 
Checking installation requirements...
Testing truecrypt... Done.

Install binaries to [/usr/bin]: 
Install man page to [/usr/share/man]: 
Install user guide and kernel module to [/usr/share/truecrypt]: 
Allow non-admin users to run TrueCrypt [y/N]: y
Installing kernel module... Done.
Installing truecrypt to /usr/bin... Done.
Installing man page to /usr/share/man/man1... Done.
Installing user guide to /usr/share/truecrypt/doc... Done.
Installing backup kernel module to /usr/share/truecrypt/kernel... Done.

That’s it.. you’re done. You should now be able to run TrueCrypt to mount your encrypted drive using my other instructions.

Good luck!