random neuron misfires

I’ve been noticing a sharp increase in trackback spam from zombied Windows machines. The interesting part of this is that they “broke” the URL by lowercasing it… the 2q2000 should be 2Q2000.

Well, after over 2,000 hits from over 248 unique IPs, I decided to take some retaliatory action. I wanted to try to resolve the IPs down to a really granular level (ISP name, gps coordinates, whatever), but there really aren’t good tools to do this yet (not without lots of manual checks).

Originally I wanted to craft up a quick mod_rewrite rule to bounce the “broken” request to the valid one.. but that would be espensive on the Apache side. Since I value elegance and simplicity, I just created a symlink from the original directory to the “broken” directly. Problem solved!

So now instead of getting thousands of 404 requests per day from these countries, each request goes to a valid page which either earns us some revenue for impressions, or causes Google to index our pages by initiating the hit.

Either way, a win for us.

I purchased an external 2.5″ drive from a company online, which is basically a Yahoo! store behind it. The drive enclosure I received wasn’t exactly what I had ordered. I wanted the IEEE1394a (Firewire 400) model, and received the IEEE1394b (Firewire 800) model. Since I don’t have fw800 ports, I’d have to use the usb2 connection. The problem there, is that Linux doesn’t support the Oxford 922 chipset on this drive. I mentioned this in a previous entry.

I went out and purchased a bilingual cable to make the drive work, and it did, though it was not bus-powered, as the original product spec sheet indicated. It doesn’t work with USB bus power or with Firewire bus power. After about a week, the power supply on the external enclosure just stopped working outright. It takes a lot of weird twisting and turning of the power cable and the drive enclosure itself to find the exact position that lets it power up. I should just be able to plug it in and not worry about it.

I received an email this morning from Yahoo! asking me to rate the vendor that sold me this drive. . I rated them with an “F”. Their customer service is non-existant, their entire website has zero contact information, there is no customer service number, there is no information on their privacy or returns policy, and many other shady things.

5 minutes after I submitted the review, I get a phone call from someone representing the company. He basically asserted that I was “an IDIOT“, and that I couldn’t read, and many other interesting and colorful profane phrases. Instead of finding out why I rated him with an “F”, he just proceeded to try to insult my intelligence with slander and libel. I calmly tried to explain that the device wasn’t what I thought I was getting (bus-powered, IEEE1394a), and that the PSU had died on the unit.

After several more insults and swears from this individual, I asked if he knew what slander was, and he said he did, and basically suggested I “go ahead and sue” or something to that effect. Then he tells me he is recording the whole conversation (also illegal, since he did not notify me of this at the beginning of the call, which is required by law. He wasn’t recording anyway, another series of lies).

Then he breaks down and tells me to send the drive enclosure back to him for a full refund, because “…he doesn’t want to do business with people like me.” I’m not sure if he was being discriminatory there or just ignorant, but these things didn’t bolster his side of the case.

We terminate the call and then I see in my email, a copy of a message he apparently sent to several others, calling me a “fucking idiot“, and then states “As posted on our checkout page prior to any purchase we do not do business with IDIOTS” (obviously their checkout page says no such thing, another lie on his part). I’ve made a local copy of their entire website, just in case they try to get smart and change what it says.

Here is a copy of the relevant parts of that email (maybe it wasn’t supposed to go to me?)

Date  Wed Apr 6 09:18:56 PDT 2005

Mark Reason  Other (every pages has our policy, and
our checkout page is very clear what a fucking idiot!)

It seems that you must not know how to READ, please
try to read your invoice and send item back for refund.

As posted on our checkout page prior to any purchase
we do not do business with IDIOTS

TKS

I’m sure if I was to send him the enclosure back (I will be anyway, certified mail, through the USPS, not Fedex or UPS, since it is defective) that he would just keep it, and my money, and not refund anything. The Better Business Bureau in Clearwater, FL has already been notified, as has my local BBB in CT.

I don’t tolerate or appreciate lies, deceit, insults, slander and libel being wrongfully directed at me. This person has absolutely no idea who he is dealing with.

In any case, if you are online looking for peripherals, avoid this company as much as you can. They’re shady, and they clearly don’t care about their customers.

My shiny new Thinkpad T42p came with an UltraBay Slim DVD-RW drive, which is nice and fast. Unfortunately, it also came with a 60gb primary IDE drive, which isn’t enough to hold my source, documents, projects and all of my VMware images for development and testing. On my Thinkpad T23, I had always used the UltraBay slot to hold a second IDE drive for these images.

This puts me in a quandry, because I’d like to use the DVD-RW, but also use my VMware images..

I decided to look into getting an external 2.5″ usb2.0/Firewire enclosure to hold a spare 60/80gb drive for the VMware images. I went with one based on the Oxford chipset, because I’d read that they were the fastest external chipset out there for these kind of enclosures.

I received the enclosure yesterday, and there were a few problems with it right off the top:

• The Firewire interface was 1394b, not 1394a (Firewire 800 vs. Firewire 400). I only have Firewire 400 peripherals, so this connection is useless to me, and the Firewire 800 pcmcia cards out there don’t have a usb2.0 combo interface, so I’d be swapping out pcmcia cards for each device. Not fun.
• The enclosure doesn’t run without external power when using the usb2.0 connection. One of the selling points of this enclosure was that I could use it without having to carry around a separate power “brick” with me when I travel. With usb2.0 being the only possible connection interface, this was important. Unfortunately, it requires a wall plug to power up the drive when using usb2.0. Ugh.
• It doesn’t work with Linux. Double-ugh. I thought it would “Just Work™”, but apparently not. I found this informative post from someone who had done some pretty extensive testing, and found that the Oxford 922 chipset is buggy, and doesn’t work with Linux. I wish I found this before I bought the enclosure!

The errors I’m getting look like this:

Apr 11 17:04:15 angst kernel: usb 5-2.6: khubd timed out on ep0in
Apr 11 17:04:15 angst kernel: usb 5-2.6: device descriptor read/8, error -110
Apr 11 17:04:15 angst kernel: usb 5-2.6: new high speed USB device using ehci_hcd and address 10
Apr 11 17:04:20 angst kernel: usb 5-2.6: khubd timed out on ep0in
Apr 11 17:04:20 angst kernel: usb 5-2.6: device descriptor read/8, error -110
Apr 11 17:04:25 angst kernel: usb 5-2.6: khubd timed out on ep0in

These are fatal, and the drive isn’t recognized. I use a LOT of external usb and firewire peripherals, so this was unusual for me. I tried about 7 different kernels and different suggestions from the community about ACPI and noapic at boot, but those didn’t seem to help.

So now I’ve got an enclosure that seems flaky with Linux, and doesn’t have the interfaces that work with the rest of my peripherals.

But.. I found a possible solution, a bilingual cable! Basically this cable takes Firewire 800 (1394b) and transforms it to Firewire 400 (1394a).

It was fairly cheap, so I ordered it. Hopefully this will let me use the drive on bus-power only, and in a way that Linux will recognize. Otherwise, I’m going to have to eBay the device. The company I purchased it from had a policy (that only showed up after my order was confirmed) that allows repair or replacement of the same device, but no exchanges. Ugh.

A couple of years ago, I purchased a Western Digital external combo drive to back up my laptops and a couple of the critical servers here. It was also partitioned for holding the digital images we take with our Minolta DiMAGE 7Hi. It was only a mere 120gb of capacity, but it lasted for quite a long time… but it was time to upgrade it.

The enclosure has two interfaces: usb2.0 and Firewire 400 (1394a). It works great, and has served me well for the couple of years I’ve had it. No complaints at all with it.

I recently went out and bought two Maxtor MaxLine Plus II 250gb drives; one for the main server, and one to replace the 120gb drive in the WD enclosure.

The upgrade of the external enclosure’s drive went pretty smoothly (full details of the disassembly), and recognizing the new drive went smoothly. I proceeded to back up 3 of the servers here to the drive, including making a duplicate copy of what was on the 120gb WD onto this new 250gb drive. I made sure to verify the backups to be sure things were intact. I’ve had a LOT of bad luck with storage and computer peripherals in general, so I was taking no chances.

The other drive went into the main server here, and that wasn’t so easy. I did an rsync of the existing running data to the Maxtor while installed in the primary slave location. So far, so good. I wanted to chroot to that drive’s mountpoint and just re-run lilo to create a working mbr on the slave, but that didn’t work so well.

Ok, second plan: switch the drives, boot the server to KNOPPIX and chroot from there, and run lilo. Nope, of course not. My KNOPPIX disks, which I use almost weekly were all no longer recognized in the CDROM drive in the server. In fact NO cdrom was recognized in that drive. Arg!

So I had to put the original drive back in as slave, switch the bios to allow me to boot to that second drive, and then re-ran lilo from there, which put the right mbr on the master. Whew. A few hiccups with some startup scripts, and I was back in business. The drive is pushing about 1gb/sec. over cache, and 49mb/sec. over disk reads. Not bad at all.

Once I wiped the servers after doing the backup, I stupidly decided to try to defrag the ext partition. It was ext3, so e2defrag barfed on it. I used tune2fs to take off the has_journal and dir_index bits from the drive metadata, and tried again.

This time it got as far as calculating the inode indices, then crashed. Ut oh. I ran e2fsck on the drive, and it segfaulted about 70% into the process. Double-ut-oh! I ran it several times, all segfaulting in the same place. Running it under gdb produced the following barf:

     0xb7fcf45b in ext2fs_unmark_generic_bitmap () from /lib/libext2fs.so.2

Rut-roh! So I decided to yank all of the data off of the backup drive onto other systems with enough free space to hold it, and reformatted it to XFS instead. After restoring the data across, all seems well.

Whew!

Well, we finally did it. After debating for months about whether or not to get an SUV, a truck, or a minivan, we’ve decided that a minivan makes the most sense right now in our lives with Seryn and our frequent travel and storage needs.

We’ve test-driven many vehicles in our quest, including the Hummer H2, Toyota Sienna, and others. The Hummer H2 is a funny vehicle.. exhorbitantly overpriced, absolutely horrible gas mileage (6mpg, so low in fact, that they can’t even print the mileage on the invoice sticker), and its heavy as a tank. Unfortunately, its about as useless as a tank too. Hummers are really REALLY fragile too. They are so vastly different from the original Hummer, that its funny they chose to use the same name.

The Toyota Sienna was a really nice van too. Smooth ride (though a bit loud in the cabin), lots of power outlets, and an obscene amount of cupholders. I think I counted 18 of them throughout the car. It doesn’t really look like a minivan either, which is a plus, but at $42k for the options we wanted, we might as well just get a full-blown Toyota Sequoia instead.

We also had the pleasure of renting a “Dodge Grand Caravan” for a trip to Buffalo, NY.. land of snow. For the most-part, Erika and I liked it. It was fairly roomy (though we packed it fully of Christmas goodies, luggage, and Seryn’s stuff), and it had the “Stow-n-Go” seats that fold flat under the floorboards. Of course, we got the “baby blue” model. Yuck.

The van handles and drives well, even for a 2-wheel-drive model (AWD is available, but not on the rental we had). There was room to get up and walk around inside, and the captain’s chairs were fairly comfortable. Its still a minivan, but it wasn’t bad.

So today we went down to the local Chrysler dealer and looked at their “Town & Country” minivan.

Note: The Chrysler Town & Country and the Dodge Grand Caravan are exactly the same, except for the company logo on the front grille. Everything inside is identical… seats, dash, floormats, everything!

The vans they had were fairly nice, though we didn’t get to test drive it. We decided that we could save about $12,000 if we went with a 2005 program version, vs. buying a brand new 2005 off of the lot. With the savings, we could also end up trading up for their “Limited” model, with leather, sunroof, and all the extras.

So that’s what we did, and the dealer is trying to find one in our color and style for us as I write this. We put a deposit down to secure the van if he can find it, and if he can, we’ll probably be proud owners of a new 2005 Chrysler Town & Country minivan.

NOT in baby blue, of course.

I’ve finally broken down and added one much-needed piece of equipment to my office… a commercial quality cross-cut paper shredder.

Speaking as someone who has had his identity stolen (along with pieces of my mail stolen from my mailbox every day for about 3 years), and being paranoid about security in general, its ironic that I haven’t had one of these before.

I’ve been doing quite a bit of research on the various models, quality, blade lifetime, number of pages they’ll swallow, and everything else about them. I finally decided that any shredder is better than having no shredder at this point, so I trundled down to my local Staples Office Supply, hoping to find something I could use.

Staples, contradictory to most of their inventory, had about 12 shredder models in stock, and ready for my review. I went through each one them, weighing their pros and cons, usability, physical weight and dimensions (after all, it does have to fit in my office), and the rest of the literature.

I finally settled on a “no-name” brand I’d never heard of, because the blades themselves looked really beefy compared to the others, and because it cut the smallest pieces (4mm x 25mm), and cost. There was one other model there that cut slightly smaller pieces, but was twice the height, and twice the price. It wasn’t really worth it. As one of my friends use to say… “paranoia is a deep pit”

I’ve already put it through its paces, and about 150 pieces of personal material that was in my “To Be Destroyed” queue, and its faring well. It even ate the stapled document I accidentally put through it (I didn’t see the staple until it was too late). It happily ate some older credit cards and lots of other pretty thick material.

So far, I’m pretty impressed. I just wonder how long these blades will last before they need to be sharpened, or I need to replace the unit. Time will tell, I guess. I’ve bought some “shredder blade oil” also, just to be sure the blades stay nice and lubricated as they do their job.

Now to see if I can’t do something useful with these bags of shreddings.

I’m very pedantic about maintaining a clean server and working environment, server-side. This means making sure things are running at top-speed, optimized, without anything that could slow down or harm the user experience. After all, I don’t have the bandwidth that Google has at their disposal, so speed and responsiveness is important. With over 50,000 total hits a day to all of the domains we host, it really is important to keep all of the sites quick and snappy.

For example, the Plucker website used to have a little iconic images next to each horizontal menu option. That’s 18 little icons that are sent to the clients when they visit the page. In most cases, these are already in their local cache, so its not a problem, but that’s also 18 extra round-trips to the webserver per-client. Removing those images from being sent increased the response time of the servery by a LARGE amount, and reduced the number of trips the clients have to make to the server to get the full site. It was just a small tweak, and there are more coming in other parts of the sites and domains I host.

But one thing that has been really irritating me, are the overly-abusive spiders, crawlers, harvesters, and robots that slam the site daily, nightly and at all hours. I have scripts set up that parse the logs and find people who are running spiders that don’t read robots.txt, or those which slam me too fast… and I check the logs every single day.

A lot of people think they’re smart by forging their UserAgent string to match that of a “real” browser, but it can clearly be shown that the hits to 20 pages in 3 seconds, proves that they’re not real humans reading that content. For those people, they get a nice big fat firewall rule to block them… only for a few hours/days, until they shape up, or fix their broken spider.

I found another one today, one from Korea apparently, calling itself “W3CRobot/5.4.0 libwww/5.4.0”. Over 3,577 hits in one day from that one beast. Obviously I blocked that entire /24 CIDR also. 221.148.44.0/24 gone. Plonk!

But two of the biggest “commercial” abusers are Yahoo!’s “Slurp” crawler, and Microsoft’s own “msnbot” crawler.

I specifically restrict Yahoo!’s crawler from reaching many parts of sites I host (such as the online CVS repositories, deep-linking into the mailing list archives, and other places). Of course, it reads, parses, and ignores my robots.txt file for each of these domains. Nice.

msnbot is an even worse abuser. It reads, parses, and follows restricted links in robots.txt. I’ve heard a rumor that they do this so they can get “more pages” than Google has in their index, so they can claim they index “more” of the Web. I have a trap set up in robots.txt specifically to catch them:

# Do NOT visit the following pathname or your host will be
# blocked from this site. This is a trap for mal-configured
# bots which do not follow RFCs.
User-agent: *
Disallow: /cgi-bin/block_crawler.pl

If they decide to ignore that, and follow that link, they’ll trigger the script to add a “Deny From” rule in .htaccess, and log it with the date and time of the block in the .htaccess file, so I can check it later to see who and what caused the denial rule.

Its pathetic.

Between both of these spiders (and its not just 2 instances, each of these two have several dozen parallel instances hitting the server simultaneously), they hit me about every 2 seconds, 24 hours a day, every day.

Until today…

I blocked them both, at the firewall, the whole /24 CIDR. Now they can’t even get to port 80. I’ll let them stew on that for a couple of weeks, and remove the block. They were hitting me so hard, that they were over 80% of my total traffic… and all of the domains I host get a LOT of hits.

There are over 400,000 posts on msnbot’s abusive behavior, as returned by Google. I fully expect, and get, ineptitude from Microsoft, but I would have thought Yahoo!’s staff was more clued than that… but I guess not. I may just end up throttling both of those back with some UserAgent detection and some sleep() calls on various pages, to keep them from hitting me so fast.

The fun never stops. I hope the users appreciate all of the extra work I go through, to keep their browsing experience fresh, fast, and snappy.

Recently, I’ve been offered a job working as a Solaris system administrator, answering “tickets” from a queue, writing up documentation, business procedures, and other similar things for a local company about 5 miles away from where we live. The opportunity looks fun, but my Solaris experience is a bit rusty. I’m great with Unix in general, and I’m stellar with Linux, but my Solaris experience is a bit lacking. It isn’t something I can’t pick back up fast, I just haven’t used it in many years.

I haven’t made a decision on the job yet. The initial pass at the salary they offered was a bit lower than I expected, but a few negotiations have brought it around to where I’m comfortable with it. A few more details, and I’ll be able to make a decision.

So I took some time to start re-learning Solaris 10 on x86, so I can refamiliarize myself with it. I installed it in VMware and did the full install. It topped out at about 3.5GiB of disk space, not too bad. The full install took about 3 hours. It really unpacks slow.

(Actual screenshot of my version running within vmware)

The Java Desktop, a new option with Solaris 10, is very slick, very nice, and reasonably fast, even while being emulated without any accellerated video drivers, on my ageing P3/1133MHz laptop (yes, that laptop).

Getting the right pieces working was not as easy as I thought…

First, I had to download the binary X server from this page. Since there is no text-mode browser on Solaris by default (even when installing everything), I had to use a Perl trick… I configured CPAN on the machine, and installed LWP::Simple, so I could use the ‘getstore()’ function to fetch the binary X server over http. While in CPAN, I installed a few other useful modules I will be using later on.

Then I installed the binary X server, and then followed these other instructions to get the right resolution on my laptop’s screen (1400×1050), which required creating a local file that didn’t exist, and reconfiguring XSun with kdmconfig.

After that, I was in business… full 1400×1050 resolution at 65k colors inside VMware, running Solaris 10 x86.

Now off to break it and fix it and break it again. I need a faster laptop..

Useful links:
http://supportforum.sun.com/sunos/index.php?t=msg&goto=1509&rid=0#msg_1509
http://www.tools.de/solaris/xf86/
http://www.samag.com/documents/s=9338/sam0410j/0410j.htm
http://www.bolthole.com/solaris/vesa1400x1050.html

Well, it had to happen… Seryn got hurt. Not just any hurt though, she dislocated her left elbow in a fall.

She’s just at the age where she’s learning to crawl and pull herself up to a kneeling position. She likes to use the bars on her crib to practice this. She also uses the couch when she’s crawling on the floor. She’ll pull herself up to a kneeling position at the front of the couch, so she can see what’s on the seat cushions of the couch.. most of the time the cats napping.

The cats are starting to learn how to tease her too, which gets her going even more. She has a great time with them.

But she doesn’t know how to recognize when balance is required, or is faltering.

She was pulling herself up to a kneeling position at the front of the couch, reached for the cat on the seat cushion, and slid over to one side, falling mostly on her arm and chest. It wasn’t a big fall, probably 10″ or so, but I looked at her and her arm was kind of bent to the side in a very weird, uncomfortable-looking position.

You know that gut feeling when you think to yourself.. “Self, that doesn’t look right…”. I got that feeling, and a sort of cold washed over me.

I picked her up fast and sat her on her butt, and she started screaming a scream I hadn’t heard in months. Ear-piercing, glass-shattering screaming. Her arm was just kind of hanging there. I thought she pulled a muscle or something like that, so I put her in her high-chair, and she started screaming more and more. I felt absolutely awful.

What do you do? She’s too young to say “It hurts over here, daddy!”, or to tell me what kind of pain she’s feeling. I couldn’t see anything obvious (and I’ve had a LOT of injuries in my life, including many broken bones), but I’m not a doctor, and I’ve never dislocated an elbow. I can’t even imagine what that must feel like, especially at that age.

After calming her down a bit, I took her clothes off and compared her arms. I was looking for a dislocated shoulder or bruising or something obvious like that. I didn’t see anything.

After she had a nap and some food, she was still not using that arm. Being the paranoid parent I am, I started thinking maybe she hit her head and had a mini stroke or something, and lost the arm function, neurologically. She wasn’t even trying to use the arm, even subconsciously.

So when Erika came home, we went off to the doctor, who immediately diagnosed it as “Nurse Maid Elbow”, and proceeded to set it for her.

Setting a 7-month-old’s dislocated elbow involves a complicated process of holding her hand palm up, and pushing her arm into her sholder, until it pops. Let me tell you, Seryn did not like that.. and the glass panes in the doctors office felt that pain when she let out that scream.

She seems to be a lot better now, and she’s using the arm to crawl with and grab things, so we think its healing.

Whew.

I found this quote buried deep within a Salon article:

“While these events are perplexing, and at least potentially dangerous, a presumed link to terrorist activity is, even if impossible to discount, still premature and wrongheaded. Alas that’s a bit like whispering into a hurricane here in 2005 America, where the T-word has been spliced into the very DNA of our collective societal psyche.”

“Thanks to one day’s events more than three years ago, we’ve come to exist in a full-on reversion mode in which every anomaly that’s at once potentially harmful and not instantly solvable takes automatic cover beneath the dark cloak of “terrorism” — a paranoid pathology that shows no sign of relenting.”

“We’ve concocted an upside-down religion, choosing to invest our faith in the cunning of an invisible adversary while disparaging our own voices of reason and good sense. At heart it’s an old story, fear of the unknown, taken to new and self-destructive heights in a politically charged climate.”

And further down in the article, even more:

“One school of thought proposes that terrorists have no desire to knock off a plane with lasers, only to scare us into thinking they do.”

And..

“Here we have cleaners and caterers able to board and roam through aircraft with no security screening whatsoever, yet people are worried about laser beams? Our priorities are insane.”