random neuron misfires

Back in October of 2007, I started writing a little tool to build MFA 2.0 sites on the fly.

This tool (in Perl of course), allows me to create a new WordPress blog targeted to a very specific niche, populate the WordPress database with hundreds/thouands of articles that target that niche, and some other fancy things with lots of trickery under the hood. My Diabetes Information and Acne Skin Treatment websites are two examples of works I created in about 30 minutes with this tool back in October.

The article sites that I point to for content are attempting to drive traffic to their site and they implement all sorts of tricks on the server-side to try to thwart spidering and bots. They want “real humans” to read their content.

So I came up with the idea of using a random proxy server for each request. It slows down the speed with which I can spider articles, but it also doesn’t put me on an automatic block/ban list.

The problem with public proxy lists is that they become stale very quickly, so I needed a way to make sure every proxy I use is alive, valid and accepting connections to the remote site I’m querying for article content.

Enter my return_proxies() function in Perl, which does just this:

sub return_proxies {
        my $link        = 'http://proxy-site/list.txt';

        my $ua = LWP::UserAgent->new;
        my $rand_browser = random_browser();
        $ua->agent($rand_browser);

        my $req         = HTTP::Request->new(GET => $link) or die $!;
        my $res         = $ua->request($req);
        my $status_line = $res->status_line;
        my $html        = $res->content;

        my $t           = HTML::TreeBuilder->new_from_content($html);
        my @output      = map $_->as_HTML, $t->look_down(_tag => 'td', class => qr/dt-tb?/);
 
        my @proxies;
        foreach my $ip (@output) {
                (my $address) = $ip =~ /((?:\d+\.){3}\d+\:\d+)/;
                push @proxies, $address if $address;
        }

        # print Dumper(@proxies);
        return @proxies;
}

I call this in my fetch_page() function like this:

        my @proxies     = return_proxies();
        my $rand_proxy  = "http://$proxies[rand @proxies]“;
        $ua->proxy(['http', 'ftp'], $rand_proxy);

So far it works very well, no issues at all that I’ve seen.

Obviously there’s a lot more to it than just this… but I can’t give away all of the secrets to my code, can I?

Gas Ascent

Gas prices are now up 11.71% from last week. We’ve gone from $3.32/gallon to $3.75/gallon in 7 days and the price keeps on rising. My local gas station goes up $0.01 to $0.03/day, every day.

Fixing Dovecot SSL Certificates

My local Dovecot certificates expired, so I had to re-gen some new ones… but the problem is that Debian’s dpkg-reconfigure for the dovecot-common package is a bit botched, and complains that there are SSL certs already, and does not re-gen new ones.. even when I manually rm them from /etc/ssl/certs/.

The solution? Do it all manually, of course.

First, check your existing certificate’s validity and expiry:

$ openssl x509 -in /etc/ssl/certs/dovecot.pem -noout -text| grep -A2 Validity
        Validity
            Not Before: Apr 21 12:21:07 2007 GMT
            Not After : Apr 21 12:21:07 2008 GMT

If the expiry is past-due, find and delete the existing dovecot certs:

find /etc/ssl -name 'dovecot.*' -exec rm {} \;

…and re-gen new ones… like this (this is where it gets ugly, but follow along one command/step at a time):

cd /etc/ssl/certs
PATH=$PATH:/usr/bin/ssl
HOSTNAME=`hostname -s`
FQDN=`hostname -f`
MAILNAME=`cat /etc/mailname 2> /dev/null || hostname -f`

# These next lines get run all in one command, from open parenthesis to end parenthesis. 
(openssl req -new -x509 -days 365 -nodes -out $SSL_CERT -keyout $SSL_KEY > /dev/null 2>&1 <<+
.
.
.
Dovecot mail server
$HOSTNAME.$DOMAINNAME
$FQDN
root@$MAILNAME
+
)

Now you should have two shiny new certs stored in:

/etc/ssl/certs/dovecot.pem and /etc/ssl/private/dovecot.pem

It's a good idea to examine these with the openssl command above, just to be sure they're correct. You should now see something like the following:

$ openssl x509 -in /etc/ssl/certs/dovecot.pem -noout -text| grep -A2 Validity
        Validity
            Not Before: Apr 23 06:30:49 2008 GMT
            Not After : Apr 23 06:30:49 2009 GMT

These now get poked into your /etc/dovecot/dovecot.conf file

ssl_cert_file = /etc/ssl/certs/dovecot.pem
ssl_key_file = /etc/ssl/private/dovecot.pem

The reason I had to go through this rigmarole was because the Thunderbird nightly build I am now using to import all of my IMAP mail archives to Gmail outright rejects the cert if it has expired.

Previous versions would issue a warning that the cert was expired, and you could continue anyway. These new versions are much less permissive, and outright block access.

To import your new Dovecot cert into Thunderbird, you have to jump through a couple of hoops.

Launch Thunderbird and go to Edit → Preferences → Certificates.

Click on the "View Certificates" button and go to the "Authorities" tab.

Click "Import" here and a file picker dialog will pop open. Navigate to your dovecot.pem certificate (the one that was put into /etc/ssl/certs/, not the one in /etc/ssl/private), and import that.

Ignore my ugly icons in these screenshots, they're 'broken' because I'm testing something in the background.

After this has been imported, if you go back to the Authorities tab and scroll down a bit, you should see a new certificate listed for Dovecot, as shown here:

If you select this certificate and click on "View", you should see something like the following:

Voila! Now you can use your Thunderbird nightly build against your local IMAP without connections being denied.

Stupid, silly Gravatars

I noticed that there is a new WordPress out with over 70 fixes. That alone plus the security issues closed merits an upgrade for me across all of the user, professional, MFA and other blogs I run and host for people.

The biggest bug fixed in this one has been affecting me for weeks now, and I've reported it and am glad to see they claim to have fixed it.

But there was something wacky with my theme, which caused user's avatars to be invisible. I trundled over to the main Gravatar site to look for answers, and saw that they have their own WordPress plugin to handle that.

I downloaded the plugin, installed it and was not surprised to find that it didn't work at all.

Sigh.

So I cracked open the source, and noticed that it flat-out was non-functional. Even their instructions say to use the following syntax:

<img src="<?php gravatar("R", 40, "http://www.somewhere.com/heatvision.jpg"); ?>" alt="" />

That url they provide in their instructions leads to a 404, because it doesn't exist. Easy problem to solve, but the code itself is never reached.

Pouring through more templates and WordPress source revealed the subtle answer:

<?php echo get_avatar($comment, 40 ); ?>

I wrapped that in a proper div with some quick inline styles, and now avatars work for public comments on posts:

<div id="gravatar" style="float:left;margin-right:0.5em;"><?php echo get_avatar($comment, 40 ); ?></div>

I wish some projects would test their code before they release it, or at the very least before they claim that it works "...just like this...".

A transcript of FBI Director Robert Mueller’s exchange at a House of Representatives hearing with Rep. Darrell Issa hit c|Net News recently. Issa is a California Republican that made his fortune by founding Directed Electronics, a publicly traded company that sells car alarms and home theater loudspeakers.

Issa also is a member of the House Intelligence Committee, which is holding a closed hearing on Thursday devoted to the Bush administration’s so-called Cyber Initiative. In January, President Bush signed a pair of secret orders–National Security Presidential Directive 54/Homeland Security Presidential Directive 23–that apparently deal with detecting and preventing Internet disruptions.

In short, the FBI wants the ability to intercept and monitor traffic going INTO Internet backbone choke-points. They’re already tapping these now, but they want the ability to “legally” intercept these communications.

tjstork had some strong words in a post he made in a thread on Slashdot about this exact issue today:

I do not know in my right mind how, it became permissable for George Bush to undermine civil liberties in the same way that we always argued it was wrong for Democrats to do.

Liberty and Freedom do not care about political affiliations and political parties. If a federal practice is wrong, it is wrong regardless of which party does it. If we do not want Hillary Clinton or Barrack Obama or Bill Clinton reading our e-mail, then we should not tolerate George Bush or John McCain doing it either. Doing so only undermines the very essence of the rule of law and the fabric of our democracy. It is the totalitarian regime that justifies itself through personality, not the free one.

We conservatives have many differences with our fellow liberal americans and we always will. However, the very thing that makes us American, the idea, as Jefferson said, “We are endowed with certain inalienable rights … To secure these liberties, governments are instituted among men”, is under assault and in the name of a rival that frankly is not nearly the equal of the rivals that we have faced in the past. We overcame the British Empire to secure our independence. We fought the Barbary Pirates, our own Civil War, Imperial Germany, and Nazi Germany, and then put our cities on the nuclear firing line against the dark stain of Communism… and we NEVER once entertained turning America into a land of checkpoints and identity requests.

What is going on now in our country is madness. America is not supposed to be a place where guys with machine guns are walking around train platforms, asking if you have a driver’s license with federal approved features. America is not supposed to be the place where the government collects data on all of its citizens.

Yeah, the muzzies blew up the world trade center, and its sad that those people died. But, the British burned our nation’s capital to the ground, the Germans sunk the Lusitania, the Japanese bombed Pearl Harbor and captured an army of 80,000 men of ours. We’ve been attacked before and we’ll be attacked again, and what makes America special is that we keep our freedoms, rather than surrender them.

There’s a million dead soldiers rolling over in their graves because we have so easily surrendered every freedom they fought for. It’s an insult to them, to our national heritage, to turn our country into some sort of crappy police state because a few muslims with box cutters give us the willies.

Support those candidates, regardless of party, that promise to end the Dept of Homeland Security, promise to repeal the USA PATRIOT ACT, and join me in a call for a Constitutional Amendment that bars the Federal Government from intercepting any electronic communications within its borders, unless it can prove before a court that those communications are with another nation with which the USA might be in a state of war.

And another reply from an AC in the same thread:

The current generation has no idea what their ancestors fought and died for. To them, the Constitution is that “dumb thing they had to learn for some test back in school.”

The words of the Constitution, the rights it promises, the beauty and eloquence of the promises themselves — all lost on a generation that mocks those who correctly punctuate their MySpace page.

So who does that leave to defend these freedoms? Us. Every geek who’s ever learned something from the Internet, every one of us who’s ever spent a night on IRC going over a disassembly with some fellow hackers from around the world — every single one of us is threatened by this sort of crap. And every one of us should work to fight it. Whether it’s refusing to turn over traffic logs, enabling mandatory SSL for the sites you administer, or just teaching your family members how to use GPG, there’s something that you (yes you — the one reading this right now) can do to make this sort of illegal, immoral, borderline fascist spying ten times harder.

Realistically though, this sort of monitoring is coming. They may not get it this time around, but you’ll be damned sure they will eventually. The ‘net’s too big a “free speech zone” for them to ignore now. We won’t be able to stop them from getting access to our data, so we’ve got to be ready. Assume that everything you send and receive is monitored. Act accordingly. If the FBI wants to spy on all citizens, then their next war will be virtual. The average citizen doesn’t know what AES is. He doesn’t know how to check a SHA-256 hash. He doesn’t know why SSL is useful. He can’t send an e-mail protected by GPG. But we can. And in the coming war… well.. I guess that makes us the terrorists. So ready your arms, fellow terrorists, and let the jihad begin.

I think they speak for many of us.

As if we needed more reasons NOT to vote Hillary Clinton into office, here comes another one:

Democratic presidential Hillary Clinton has threatened to “obliterate” Iran if it launches a nuclear attack against Israel as she fights for her own political survival.

“I want the Iranians to know that if I’m the president, we will attack Iran,” Senator Clinton told ABC News, asked what she would do as president were Iran to launch a nuclear attack on Israel.

”In the next 10 years, during which they might foolishly consider launching an attack on Israel, we would be able to totally obliterate them.”

She’s not even President, and she’s running her mouth like world politics is some sort of game where she can just throw around our already-stretched-thin military like plastic soldiers on a game board in her office.

It is also important to note that Hillary was only one of TWO SENATORS out of 100 that declined to vote to remove the retroactive immunity clause from the Telecommunications Wiretapping law. Do you really want to elect someone who SUPPORTS the illegal and immoral wiretapping of American citizens? I most-certainly do not.

Barack Obama is no better, and neither of them are getting my vote this year (nor is McCain).

People this dangerous should not be granted power by the people, ever.

I use a lot of rdf/rss/atom feeds to keep up with the trends in general news, gadgets, software, releases, productivity and many other things.

Until very recently, I was using an add-on to Firefox called “Sage“. Sage has a lot of really slick features which I have grown accustomed to… the largest of which is the ability to see all the feed summaries at a glance, on one page, without scrolling. It looks like this:

I like the way it docks to Firefox and is accessible or tucks away on the left sidebar with a simple Alt-Z key combo.

But a friend recently asked me if I’d tried Google Reader, and honestly… I’d never heard of it. He lauded the benefits of it being tightly integrated with Google’s suite of other products and applications.

As I usually do when someone suggests that I jump ship from my comfortable daily-use tools, I remained skeptical. But I decided to give it a try anyway.

The first thing I noticed, was the clunky interface. Ugh. Here’s what the above feed from Plucker’s feeds looks like in Google Reader:

An enormous waste of space, IMHO.

But as luck would have it, someone else felt the same way about that problem and solved it with a Greasemonkey script.

As I fumbled through the rest of the interface, I began importing my feeds one-by-one from my Sage sidebar into the Google Reader sidebar. I got about 90% of the way through, grumbling that Google Reader needed an ‘Import’ feature, and realized that Google Reader has exactly that… buried in the Settings page. It would have even imported my Sage OPML file, if I knew that earlier.

Why they didn’t decide to make it an option right ON the sidebar where you create and manage feeds, I’ll never know.

So I’ve imported all of my feeds and manually created what Google Reader refers to as “Folders” to organize them so they look like my Sage feeds and structure. Google Reader actually supports drag-n-drop here between “Folders”, which is nice and makes it easy to organize LOTS of feeds.

So now onto the bugs I’ve found in the last 30 minutes or so of playing with it:

1. No way to sort the feeds in the sidebar by name/date/etc. itself. You can sort the feed articles on the right side, but not the feeds themselves.
2. Google Reader refers to “Folders” in one place (main page dropdown), but calls these same items “Tags” in another place (Settings page). Why not make it behave like Gmail, where you can “tag” feeds and set colors to each of them? At the very least, use the same terms in all of the relevant places.
3. No way to remove/hide or “archive” read items from view on the right side panel. Huge oversight.
4. I can’t seem to style or change the interface at all. Sage Styles are quite spiffy.
5. Can’t “undock” the interface, like I can with Sage.

I’m sure there are other issues, but I’m still green with using it… I’ll repost more when I have a chance to really beat this up.

The only reason I’m exploring this particular tool, is to try to carve out more time and productivity using feeds.

I drive by my local gas station every morning on the way to the train, and over the last 3 days I noticed a sharp rise in the price of gas:

Tuesday night: $3.43
Wednesday morning: $3.44
Wednesday night: $3.44
Thursday morning: $3.45
Thursday night: $3.64

Gas jumped $0.21 in a matter of 3 days, $0.19 of that in less than a day.

Why is the price of gas so high, and still rising?

The statistics show that production is up and demand is down (although China is booming, which has some impact).

4/17/2000 $1.63
4/16/2001 $1.75 (+06.86% from 2000)
4/15/2002 $1.59 (-09.15% from 2001)
4/14/2003 $1.79 (+11.18% from 2002)
4/19/2004 $1.99 (+10.06% from 2003)
4/18/2005 $2.43 (+18.11% from 2004)
4/17/2006 $2.99 (+18.73% from 2005)
4/16/2007 $3.08 (+02.93% from 2006)
4/14/2008 $3.61 (+14.69% from 2007)

(Source: Energy Information Administration).

I’d love it if someone could take this table, import it into a spreadsheet and do a per-country breakdown to see the trending of gas prices rising and falling.

We have more oil stockpiled than we need, and we’re not using it as fast as we used to, so why has the price of gas gone up 54.85% since 2000.

It makes no sense whatsoever. Gas prices go up sharply, but come down very slowly. We can’t sustain this market if the price of gas is going to keep increasing like this.

I found this neat gadget over on zFacts:

And before the Canadians and Europeans jump on me for ranting about paying such a “cheap” price for gas, let me remind you that YOU have more-efficient vehicles, better EPA standards and your public transportation is FAR better than even our best in the US. Your roads also allow and encourage people to walk, cycle, rollerblade and so on.

Here in the US, in most cities and towns.. you’re not only discouraged from using bicycles/walking on the roads, it’s downright forbidden.

When I go to the stores now, I don’t even use plastic bags for most of my purchases. Plastic bags (and plastics in general) are made from petroleum, and petroleum comes from… you guessed it:

Oil.

It’s always about oil.

I’ve written about this before a little bit, but now it’s reached a head.

I call my credit card companies every few months and have them lower my APR and raise my credit limit, based on my stellar credit rating and spending history. All of my credit cards… with the exception of the Amazon/CHASE card, are well, WELL below 10% vAPR (prime + Xx.xx%).

My Amazon/CHASE card is Prime + 21.99%, based on the my credit report and my “other equities”.

I called them and after being bounced around on the phone through 6 different people, I finally was put in touch with an “account manager” (the second one I’d spoken to on that call) who said it was..

“related to my other credit cards, the spending on those cards, the APR of those cards, and also any other loans or bills I pay for with income or earnings”.

This includes a car loan, telephone bill, utilities bill and so on.

WHAT?!!!

So what they’re saying, is that because I have a stellar credit rating, and because I keep low balances on my other cards, have VERY low APR on those other cards, and have NEVER been late or missed a payment on any credit card or bill… that I’ve “earned the privilege” of being selected to have my APR tripled on my Amazon/CHASE card.

How lovely of them.

In fact, while I was on hold on the telephone, I logged onto myfico.com and bought my 3 credit scores online (if you use the code 7yrsale, you get 25% off of the cost because it is their 25th anniversary), and quoted those scores to the woman who I was speaking to. Because my credit rating is marked as “Very Good”, I’ve earned the right to have the APR of someone with a credit score rated as “Poor”.

Earlier this morning, I called another one of my credit card companies, and had them lower the vAPR that they were charging me (already below 10%), and they dropped it by another 3% to FAR below 10%, no problems, no questions asked, right there on the phone while I was talking to her.

Why does Amazon/CHASE insist on charging such a ridiculous APR? I did manage to squeak out one clue: Every credit card CHASE supplies is an “unsecured loan“, and so they make their customers suffer.

This goes right along with a documentary I saw recently called “Maxed Out“, which details how these companies continue to raise the interest rates to a level just beyond what people are able to reach, so they continue to pay their credit card payments for life, trapping them in a prison of credit card debt forever.

These companies don’t WANT you to pay off your debt every month, because you aren’t making them any money. If everyone paid their credit cards off every month, these companies would go bankrupt.

Instead, they make us go bankrupt.

If someone is down on their luck, and unable to pay their bills in a timely fashion, why should Amazon/CHASE raise the APR on their card, making it even harder for someone to pay the card? They should be LOWERING the rates, so people at least pay their payments at a lower APR, vs. having to default on their loans, causing chargeoffs and Amazon/CHASE to lose any chance they might have had at getting paid what they are owed.

Silly industry, pure silliness.

Anyway, I told the supervisor that I was going to pay out the card in-full this month, cut up the card and let the account remain open but idle forever (NOTE: you should never close credit card accounts if you have them paid off in-full, it hurts your credit score in very bad ways), and I was going to advertise and publish their “methods”, as well as tell people NEVER to use their equities or properties again, ever.

And so now I have.

You have been warned.

This weekend was just as busy as the week at work. It’s Sunday afternoon, and I’m still going…

Reconstructing Maildir from Backups

Moments ago, I found that my archive of the Coldsync mailing list in Maildir format somehow became corrupt, so attempts to copy those messages to Gmail failed using my Thunderbird trick.

I found an older copy that was in mbox format, and used the “Perfect” mbox to Maildir converter script to convert it to Maildir format.

Now I’m back to populating Gmail with my email once again (8,427 in Gmail now, with about 112,000 left to go).

Calendaring Conundrum

Also this past week, I realized that my calendar in Outlook had somehow duplicated over 1,700 of my events. I’m sure it was the result of using things like PocketMirror and other sync tools for Palm with it. I’m going to be cleaning that up next. That requires manual, visual inspection of each event, to make sure I’m deleting the dupe and not the original (thus leaving the dupe copy on the calendar). Very odd.

Once that is done, I have to reinstall all of my Palm conduits on the Thinkpad X61s and get that all sync’d to my Treo. With everything on my Treo, I can then begin consolidating my various calendars and task lists into one clean interface.

Back to the Blog; 8 years of postings

I also cleaned up 8 years of blog postings, reformatted them all and cleaned up the broken HTML that was the result of importing the diary entries from Advogato. That was 353 separate posts to go through by hand and clean everything up. Now it looks as it should.

Going back and reading through those old diary posts was… interesting. I didn’t realize how much I’d done in those 8 years, all of the people I’d met, projects I’d completed, places I’d been. I might turn the whole blog into a set of memoirs for my daughter Seryn for when she’s old enough to understand all of the things her daddy did in his life.

Movies, movies, movies!

I managed to pack in watching 4 movies while I worked this weekend, but the two best ones were “Maxed Out” and “The Man from Earth“. Both of them were equally good, and worth watching. I highly recommend both of them.

“Maxed Out” was eye-opening, and depressing at points, because of the situation our country is in right now. People are literally killing themselves (3 cases are described in the movie) because of their debt. The industry specifically caters to those who can NOT pay their debts down, because those people are the cash-cow for them. These people pay their minimum payments for life and pass on their debts to their children. They don’t want people who pay their credit cards in-full every month to be their customers, there’s no profit in that. Watch the movie for the rest of the details.

“The Man from Earth” brings new ideas to our concepts of religion, biology, archeology and many other fields of traditional study. It reminded me somewhat of the information that was in the first 1/3 of “Zeitgeist: The Movie” (freely downloadable, or viewable online). The Man from Earth is a low-budget movie, but packs a punch in the back story. I won’t spoil it here, but definitely go rent it if you can.

AT&T WWAN in VMware

I managed to get my physical Windows machine (an HP machine I purchased at BestBuy a few years ago), virtualized and configured in VMware using VMware Converter. I had to hack into it to get the vm to recognize my legitimate Microsoft Product Key, but after that, it was a snap.



Then I installed and configured the AT&T Communication Manager software to talk to the physical SIM card inside my laptop, so I can go online with the laptop wherever there is valid GSM signal, at 3G speeds.

I didn’t think the vm would recognize the physical card in the machine, if Linux didn’t see it natively… but it does. It’s a bit slow, but at least I can function with one laptop connected to the WWAN on the train with a larger screen at 1920×1200 resolution, instead of the smaller laptop with the 12″ screen at 1024×768 resolution.

The next step is to get the second laptop networking across the connection that the first laptop provides. That should be interesting to solve, since one of these is Windows, and the other one is Linux.

One possible solution would be to take the SIM card that is physically inside the laptop, put it into an external 3G PC-Express card (as in the image here), and then put that into a WWAN router, and carry THAT with me on the train. It has to be portable, of course…

But if I use that approach, not only can I share the connection with both of my laptops, I can also provide “free” wireless to anyone on the train who wants to get online. Maybe I can solicit free beer or donations in the cafe car to help offset those costs.

New Web SEO

An old acquaintance from a Business Development Group in New London has recently contacted me asking me for help with his website. He pointed out that his site is losing customers to two other sites in his very narrow niche here in Connecticut.

I looked at the competition, and noted that they’re not doing anything special that would merit that increase in customers for them. But I also noted about a dozen problems with this person’s company website in question, that needs immediate attention. His site is ranking at PR0, and the other two sites are pushing PR1/PR2, with no real traffic.

So I think I might pick up a little side project to help him out, to bring his site up to where it should be, and up to the level of standards that I consider acceptable. It looks like it’ll be fun, and it should bring in more income to help fund some projects I’m working on in the background (mostly Seryn’s secret project :)

I make a decent amount of money with my websites now, and I think with the proper care and attention, he can too.

Wrestling MySQL

Speaking of websites, a few months back, I started making some MFA 2.0 websites of my own, based on WordPress that are populated with hundreds of public-domain articles on niche content.

To do that, I wrote some tools (in Perl, of course) to spider the remote sites, pull the articles, and stick them into my WordPress databases, complete with proper author in the Author field, original posting date and so on.

Here’s one example that took me less than an hour to create and populate with articles, using these tools.

This particular site has 220 articles in it, but if you look at them articles, they’re all linked to external quality citations and resources. This site, with zero marketing, pulls in an average of about 6,000 hits a month. I have a handful of others, with between 200 and 1,000 articles in each, all of similar high-quality and care behind their creation.

The hard part is that in order to have an article attributed to an author in WordPress, that person has to be in the User table. That gets complicated to get right, but I managed to figure it out.

But I needed a way to ensure that the site’s content didn’t remain “stale”, so I came up with another trick to do that:

UPDATE wp_posts SET post_date='1999-01-01' + interval rand() * 3391 day;
UPDATE wp_posts SET post_modified = post_date;

I put this in /etc/cron.weekly/, and now my articles are randomized on a weekly basis, so visitors or search engines coming to the site will constantly see “new” content.

So far, they’re doing well, and bringing in more than enough to cover the monthly costs of running the servers, bandwidth and power.

The Day Job

The day job continues to go well, and I’m picking up significant speed on those projects.

The more I work within the system, the more I see where some “optimization” can be put into place, including some automation to make things much easier for myself and others in my group. I need to carve out some time to do exactly that, within the limits of my time allotted for the critical tasks that have established deadlines.

The commute has worked itself out and my routine is fairly static now, so that is no longer an unknown. Now I just need to get my foot in the door and keep things moving forward at a cheetah’s pace.

Fun times, definitely fun times. The positive energy has come back in rushing waves, and the good luck has overwhelmed me.

When I’m traveling to and from the New York office, I see a lot of people using Blackberry devices. They treat them like a third arm, a new appendage, permanently attached to their outstretched hand.

These people constantly check their Blackberry devices every minute or two, no matter what they’re doing; eating, standing in line to buy tickets, walking down the road or waiting on the subway. They can’t STOP looking at it.

The problem is that they mistakenly believe that having a Blackberry automatically makes them

1. more important individuals,
2. busier than you could ever possibly be, and
3. more productive than anyone around them.

These people will even argue with you and tell you that you can’t possibly get more email than they get, you can’t possibly have more projects than they have, you can’t possibly KNOW what it’s like to be busy… because you don’t have a Blackberry.

Let me burst that bubble right here and now.

The fact that you own a Blackberry alone, shows that you are not in control of your own workload, your own tasks, your own projects. This is further reinforced by the fact that you can’t stop looking at it every 2 minutes.

How much focus can you possibly have when you’re working away on something and your Blackberry buzzes on your desk or at your side? What’s the first thing you do? You drop everything and attend to it, like it was a crying infant in need of food.

You just broke your focus. You fiddle with your device, and then you try to regroup with your previous task, only to be interrupted again minutes later by another Blackberry alert. Multiply this out hundreds of times a day, and you get nothing done. It’s no wonder you need to carry it 24×7 and work until 7pm at night. You’re not productive at all, you’re barely scraping by.

If you WERE on top of your projects and tasks, you wouldn’t need to check your email every 2 minutes because you’re already on top of it. You wouldn’t need to constantly attend to it, because everyone knows where your projects and tasks are, they don’t need to follow up with you on them, because everything is already on-track or ahead of schedule…. right?

The other thing I’ve noticed (and verified by talking to dozens of people who use these devices), is that people use them to READ their emails, but they never reply to them from the device. They read their emails, then go back to the office later and read those same emails again and reply to them from their office PC.

Riddle me this, Batman: What is the point of reading an email more than once? You read it, you reply right there if you have the time, or you write down some information in your notepad/PDA, and you delete it. The last step at the end of reading EVERY email is to delete it. If you’re going to read an email on your Blackberry, reply to it right there, take notes for later action items and delete it.

You should never have to read an email more than once… unless it’s some research email that is archived off into a folder specifically for that purpose.

There’s a tangent piece of productivity here as well, unrelated to Blackberry devices but definitely made more visible by those who use them. I see people at work who check their email 10 to 20 times an hour, every hour throughout their workday. I check my email 2-3 times a day, tops. I don’t need to check it more often than that (and yes, I get hundreds of important emails every day, 80% of which require my personal response and attention).

If someone needs something urgent from me, email is the wrong vehicle to use to get my attention. These people know to call me on the phone or find me in person to get to me. All of that time I’m NOT spending looking at email, is spent working through my projects and tasks. I would be nowhere as productive as I am if I constantly checked my email all day.

I have a Treo smartphone, and I get email on my device when I request it, not when it comes into my remote Inbox. I know I’ll respond to all of my emails at the end of every day, and my Inbox will be clean and tidy every night before 5pm. I don’t constantly check my PDA every 2 minutes because I don’t have to.

The more in control of your projects, tasks and your life as a whole, the less you need these things interrupting you from enjoying the rest of your non-work time.

In a previous post, I described two methods of moving almost 15 years of email into GMail.

I now have a newer, MUCH FASTER method, which eliminates having to go into Google and create labels and adjust the account preferences at each import… using nothing but Gmail and Thunderbird.

Here’s how:

1. Download yourself a shiny new copy of Mozilla Thunderbird and install it.
2. Download a copy of the Remove Duplicate Messages add-on and install that, then follow my instructions for removing duplicate emails.
3. Add your local IMAP account to Thunderbird, and verify that it works
4. Go through your folders and remove any of the duplicate email messages using the add-on above.
5. Add your GMail account to Thunderbird. Make sure your GMail account is IMAP enabled in your GMail settings (Settings -> Forwarding and POP/IMAP -> IMAP Access -> Enable IMAP). Verify that you can see your GMail email via Thunderbird.
6. Drag your local IMAP folders one-by-one into the Inbox of your GMail account in Thunderbird. Literally drag and drop the FOLDERS into your GMail Inbox. Don’t drag the messages, drag the entire folder.

What you should see, is that you will be creating folders under the Inbox in your GMail account in Thunderbird, and when you log into GMail, you will now see new labels for each IMAP folder you just dragged in there.

For example, if you dragged a folder called “Taxes” into your GMail folder, you will now have a label in GMail’s web interface called ‘INBOX/Taxes’. GMail applies the ‘INBOX’ prefix to the folder name and creates a label with that name.

From here, once you get all of your email into GMail, you can select ‘Edit Labels‘ in the left column of GMail and rename/delete those labels to suit your own personal preferences.

That’s it!

The benefits here are:

• MUCH faster than having Google fetch your email using pop3
• Auto-creates labels for you on the fly. Want a new label? Rename your local folder before you drag it over.
• No need to set up additional accounts inside Google to pull from
• Preserves dates and times of the original messages, including threading

I’ve imported hundreds of emails in the time it has taken me to write this post. Using the other methods, I would get 200 emails pulled over in about an hour. The older methods were S-L-O-W.