HOWTO: Run boot2docker in VMware Fusion and ESXi with Shipyard to Manage Your Containers

fbbb494a7eef5f9278c6967b6072ca3eThis took me awhile to piece together, and I had to go direct to the maintainers of several of these components to get clarity on why some things worked, while others did not, even following the explicit instructions. Here, I present the 100% working HOWTO:

I started with a post I found written by someone on the boot2docker project page, describing how to get this working in VMware. But he missed some crucial steps, and the syntax is wrong. Also, Shipyard has gone to a new version, and the method of starting the containers is very different from the steps posted.

Creating the boot2docker VM Instance

First, we need to create a VM inside VMware Fusion and/or ESXi. If you’re using VMware Workstation, the steps are roughly the same, but the screenshots may differ slightly.

You’ll create a new VM, and add two NICs and a single IDE HDD to the VM. Something like 10GB should be fine to hold all of your containers, build scripts and any other persistent data you might need. Follow the screenshots below for some specifics and details. There are a few subtle tweaks you’ll need to maximize your boot2docker VM.

Read the rest of this entry »

SOLVED: Installing VMware Tools in Ubuntu 5.04 under ESXi

It may seem odd, that it’s almost 2015, and I’m writing a post describing how to install VMware Tools inside Ubuntu 5.04, a Linux distribution that was released on April 8th, 2005 and went End of Life on October 31, 2006.

I run a home lab here with a number of test machines (all VMs now), which includes every single major Linux distribution and version. Yes, really. I’ve been going back trying to update all of them to current tools, VMHW versions, NIC drivers and tweaking other things. Among these, were a number of very old Linux distributions who did not support VMware drivers natively (eg: open-vm-tools didn’t exist at that time).

If you run the standard configuration (I run it non-interactive as below)…

# ./vmware-install.pl --clobber-kernel-modules=vmci     \
                      --clobber-kernel-modules=vmxnet3  \ 
                      --clobber-kernel-modules=pvscsi   \
                      --clobber-kernel-modules=vmmemctl \
                      --clobber-kernel-modules=vsock    \ 
                      -d

At some point after the X11 configuration, you’ll see this error displayed:

Creating a new initrd boot image for the kernel.

$Id: mkinitrd,v 1.201 2004/05/16 22:00:48 herbert Exp $

Usage: /usr/sbin/mkinitrd [OPTION]... <-o outfile> [version]

Options:
  -d confdir  Specify an alternative configuration directory.
  -k          Keep temporary directory used to make the image.
  -m command  Set the command to make an initrd image.
  -o outfile  Write to outfile.
  -r root     Override ROOT setting in mkinitrd.conf.

See mkinitrd(8) for further details.
ERROR: "/usr/sbin/mkinitrd" exited with non-zero status.

Your system currently may not have a functioning init image and may not boot
properly.  DO NOT REBOOT!  Please ensure that you have enough free space
available in your /boot directory and run this configuration script again.

Execution aborted.

Found VMware Tools CDROM mounted at /media/cdrom0. Ejecting device /dev/hdc ...

But there’s a one-line fix! Open up “vmware-tools-distrib/bin/vmware-config-tools.pl” in an editor and go down to line 7692, and you’ll see something that looks like this:

      $initmodfile = '/etc/sysconfig/kernel';
      if ($gSystem{'distribution'} eq 'suse') {
          $syscmd = join(' ', $binary, $kernelList, $initrdList);
      } else {
          $syscmd = $binary;
      }

You’ll want to modify that last “$syscmd” line to look as follows:

      $initmodfile = '/etc/sysconfig/kernel';
      if ($gSystem{'distribution'} eq 'suse') {
          $syscmd = join(' ', $binary, $kernelList, $initrdList);
      } else {
          $syscmd = join(' ', $binary, '-o', "/boot/initrd-$kernRel.img", $kernRel);
      }

That will now result in the following successful output:

Creating a new initrd boot image for the kernel.
   Checking acpi hot plug                                              done
Starting VMware Tools services in the virtual machine:
   Switching to guest configuration:                                   done
   Guest operating system daemon:                                      done
The configuration of VMware Tools 9.0.5 build-1065307 for Linux for this
running kernel completed successfully.

You must restart your X session before any mouse or graphics changes take
effect.

You can now run VMware Tools by invoking "/usr/bin/vmware-toolbox-cmd" from the
command line.

To enable advanced X features (e.g., guest resolution fit, drag and drop, and
file and text copy/paste), you will need to do one (or more) of the following:
1. Manually start /usr/bin/vmware-user
2. Log out and log back into your desktop session; and,
3. Restart your X session.

Enjoy,

--the VMware team

Voila! Now you have a successful build of the native VMware modules and can switch your ethernet module to VMXNET3 from E1000 to save a little memory on the ESXi side.

HOWTO: Remove the “Year in Review” Posts from your Facebook Wall

It’s annoying. It’s Facebook. We all learn to either love or hate it, but there are ways to make the annoying parts of it go away.

The most-recent annoyance is Facebook’s compulsion to add the “Year in Review” posts from people to your Facebook wall.

Thankfully I’ve never been asked to fill mine out, but I do see hundreds of these from other “Friends” of mine. It looks like this:

2014-12-26 Facebook - Year In Review

Here’s how to get rid of them:

  1. Log into Facebook and go to this page:

    https://www.facebook.com/settings?tab=blocking

  2. Once you get there, you’ll see a section near the bottom labeled “Block Pages”. Put “YearInReview” into that form and hit Enter.

  3. If you’ve done it right, you should see something like this:

    2014-12-26 Facebook- Manage Blocking

That’s it, you’re done!

If you want to get rid of more Facebook garbage, ads and other annoyances, you can install the “Social Fixer” browser extension in your browser of choice. There are versions for Safari, Chrome, Firefox and Microsoft Internet Explorer. Works great!

Quick tip for Users Hosting TurnKey Linux Images for Public Consumption

I host lots of internal infrastructure for friends, colleagues, the Open Source community, my LUG and others and much of the recent need I’ve seen has been with easy-to-contain images like those provided by the TurnKey Linux project.

These are very small, light, lean VMs and app containers you can deploy on your servers, baremetal, VHW or other environments such as Hyper-V, VMware, VirtualBox and more.

But they tend to advertise a bit too much inside each application they host, and for some people/clients, that’s not desirable. But there’s a quick and easy fix!

If your app, like Moodle for example, has a web interface, you’ll see a footer that advertises TurnKey Linux and will look like this:

2014-12-25 22_36_10-Moodle-TurnKey Linux

The easy way to remove this, is to SSH into your TurnKey Linux image (you did change that SSH port to something other than port 22, right?), and run the following commands:

 
$ sudo a2dismod substitute
Module substitute disabled.
To activate the new configuration, you need to run:
  service apache2 restart
$ sudo service apache2 restart
[....] Restarting web server: apache2apache2: 
 ... waiting apache2: ok

Once you’ve disabled the “substitute” apache module with a2dismod and restarted, you should now see something that looks like this:

2014-12-25 22_42_18-Moodle-TurnKey Linux a2dismod

If you want to modify that footer so it refers to something unique to your group, team, organization or otherwise, just re-enable the module as follows:

 
$ sudo a2enmod substitute
Enabling module substitute.
To activate the new configuration, you need to run:
  service apache2 restart

Now you’ll need to edit a small configuration file with a text editor before restarting Apache to make it visible. That file is “/etc/apache2/mods-enabled/substitute.conf”. Make your changes there, save and restart Apache as follows:

$ sudo service apache2 restart
[....] Restarting web server: apache2apache2: 
 ... waiting apache2: ok

That’s it, you’re done!

HOWTO: Quick 7-Zip Trick to Encrypt Your Files with Non-Interactive Mode

I have a lot of data that I archive away on a regular basis, both on my “PC” machines and my mobile devices OTA. I needed a secure, reproducible way to secure those data with a very strong, complex password using extremely tight compression.

Unfortunately, p7zip on Linux and 7-Zip for Windows don’t permit a non-interactive way to pass in a password. So I came up with a quick-and-dirty workaround!

  1. First, generate a complex password and put this in a file with one line. Make sure there are no leading or trailing characters in this file, including a trailing newline. There should be one and only one line in this file. Here’s are some examples of a password generated with the level of complexity you’ll want for secured archives (42 characters in length, 15 minutes of generated entropy from a hardware dongle):
    {Kt8}m.Gs7:g}=%-nfX[F_E(tKs[X,}GrN$mz^?m7^
    +S/>>@7DzKafmLkSkS:-"4-*@k)#u@mQ>"=7j_vvu,!
    <prX9WG#h,t-Ka`poA9rhJWc]H9M}`NA(8_93tD\hR
    
  2. Now with that password in a file, run the following loop to compress the data and pass in the password inline.

    You’ll notice that on Windows, I use Cygwin here, because I can create scripts and retain previous commandline history to reproduce this on a regular basis.

    I’m also using lzma2 here because it gives me slightly tighter compression for minimally more CPU time to generate the archives.

    On Linux

    cat your-pw-file | for i in *.xml; do 7z u -t7z -m0=lzma2 -mx=9 -mfb=64 -md=64m -ms=on $i.7z $i -p --; done;
    

    On Windows (using Cygwin)

    cat your-pw-file | for i in *.xml; do do /cygdrive/c/Program\ Files/7-Zip/7z.exe u -t7z -m0=lzma2 -mx=9 -mfb=64 -md=64m -ms=on $i.7z $i -p --; done;
    

  3. To verify that the files are properly encrypted and the right password works as expected, test as follows:

    On Linux

    cat your-pw-file | for i in *.7z; do 7z t $i -p --; done;
    

    On Windows (using Cygwin)

    cat your-pw-file | for i in *.7z; do do /cygdrive/c/Program\ Files/7-Zip/7z.exe t $i -p --; done;
    

  4. Now you can delete that password file from disk. I can’t stress this enough. Once you’ve used the password, and secured it in a managed password container, you’ll want to delete all traces of it that you do not need in plain sight on disk.

That’s it. Now when you want to decompress those archives, you’ll need to supply the password you generated before. Make sure you keep this password secured in a managed location. A password is only as secure as your ability to manage it.

Good luck!

Updating Legacy Fedora Linux Distributions to Use Archive Repositories

Fedora LinuxI run a VMware ESXi server here that hosts ~500 separate VMs, clones, templates and test builds of operating systems for testing, development, personal playground and other roles.

Some of these VMs are older Fedora, Ubuntu, Debian and various other Linux distributions. Since those distributions are no longer active, maintained by the community, the update URLs to install packages have gone away, or been moved to new locations.

Here’s how to update and fix those older versions of Fedora Linux so you can continue to install packages on them, past their “community expiration” date. I’ll post another entry for the same work for Ubuntu and Debian as well.

In your /etc/yum.repos.d/ directory are a number of configuration files specific to yum and repositories. It’s contents may look something like this:

[root@fedora-fc4 / yum.repos.d]# ls -lart
total 72
drwxr-xr-x  116 root root 12288 Nov 23 22:57 ..
-rw-r--r--    1 root root   344 Nov 24 14:42 fedora-updates-testing.repo
-rw-r--r--    1 root root   336 Nov 24 14:42 fedora-updates.repo
-rw-r--r--    1 root root   305 Nov 24 14:42 fedora-extras.repo
-rw-r--r--    1 root root   319 Nov 24 14:42 fedora-extras-devel.repo
-rw-r--r--    1 root root  1130 Nov 24 14:42 fedora-devel.repo
-rw-r--r--    1 root root   300 Nov 24 14:43 fedora.repo
drwxr-xr-x    2 root root  4096 Nov 24 14:43 .

In the case of Fedora Linux, you’ll want to change each of these so they reflect the new archive site, vs. the original download site, which no longer resolves and does not exist.

The original URL looks like this:

baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/$releasever/$basearch/os/

You’ll want to edit that to point to the following new URL (highlighted in red below):

baseurl=http://archive.fedoraproject.org/pub/archive/fedora/linux/core/$releasever/$basearch/os/

Once you make these edits to all of the repository files, you can run ‘yum update’ again and fetch all of the legacy update packages, install, remove and keep them as current as those distributions were at that time.

$ sudo yum install git-core
Setting up Install Process
Setting up repositories
updates-released          100% |=========================|  951 B    00:00
extras                    100% |=========================| 1.1 kB    00:00
base                      100% |=========================| 1.1 kB    00:00
Reading repository metadata in from local files
Parsing package install arguments
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for git-core to pack into transaction set.
git-core-1.4.4.2-2.fc4.x8 100% |=========================|  67 kB    00:00
---> Package git-core.x86_64 0:1.4.4.2-2.fc4 set to be updated
--> Running transaction check

Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 git-core                x86_64     1.4.4.2-2.fc4    extras            2.9 M

Transaction Summary
=============================================================================
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)
Total download size: 2.9 M
Is this ok [y/N]: y
Downloading Packages:
(1/1): git-core-1.4.4.2-2 100% |=========================| 2.9 MB    00:05
warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID 1ac70ce6
public key not available for git-core-1.4.4.2-2.fc4.x86_64.rpm
Retrieving GPG key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-extras
Importing GPG key 0x1AC70CE6 "Fedora Pre Extras Release "
Is this ok [y/N]: y
Key imported successfully
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: git-core                     ######################### [1/1]

Installed: git-core.x86_64 0:1.4.4.2-2.fc4
Complete!

Now it works. Good luck!

SOLVED: Fixing Perl Plug-ins for XChat Azure in OS X Mavericks 10.9

I’m a heavy user of IRC and have been since the late 90’s. I use X-Chat for IRC on almost all machines, including Mac and Windows, not just the Linux environment that it was created upon.

When I recently upgraded my OS X Lion (10.7.5) machine to OS X Mavericks (10.9), I started seeing a significant amount of core OS bugs. One of them affected how I use XChat to get to Freenode and other networks via SASL, SSL and Tor.

If you’re using XChat Azure 1.16.0 or the latest preview of 1.16.1, you’ll see the following errors when trying to load Perl plugins from Window -> Plugins and Scripts inside XChat:

Invalid Mac OS X bundle or required Mac OS X version is not satisfied

You’ll see this when you try to select the version of Perl showing in the Plugins window. The Python plugin works fine, just not Perl.

Here’s the quick-and-dirty fix, until the XChat Azure team can come out with a Mavericks-compatible release that uses the latest Perl:

Change to the XChat Azure PlugIns directory:

cd /Applications/XChat Azure.app/Contents/PlugIns/

Duplicate the contents of the perl-lion Perl bundle into one for Mavericks

sudo rsync -avP perl-lion.bundle/. perl-mavericks.bundle

Update the Info.plist file to match the versioning:

sudo defaults write ./perl-mavericks.bundle/Contents/Info XChatAquaMacOSVersionBranch 10.9

There’s one last step I found when I ran that last command to update Info.plist, and that is that you have to hand-modify the version in the .plist file to match OS X Mavericks 10.9 versioning:

sudo vi perl-mavericks.bundle/Contents/Info.plist

Down near the bottom of the file, change the string value of XChatAquaMacOSVersionBranch from 10.6 to 10.9, like this:

        <key>XChatAquaMacOSVersionBranch</key>
        <string>10.9</string>

Now if you go quit and restart XChat Azure, go to Window -> Plugins and select your Perl interpreter (the one with 10.9), it will work, and you’ll see your plugins loaded as expected:

[16:27]   Python interface loaded
[16:27]   Perl interface loaded
[16:27]   SASL: auth loaded from /Users/$USER/Library/Containers/org.3rddev.xchatazure/Data/Library/Application Support/XChat Azure/sasl.auth

I wrote some HOWTO documents describing how to configure Tor + SASL + SSL to connect to IRC, you can find them here:

That’s it! Good luck and happy IRC’ing!

How Many Java Versions is Enough for Mavericks, Apple?

Apple OSX Mavericks logoA lot of software outright fails to work on Apple OS X Mavericks.

It’s a disaster. Almost nothing works right.

Not only is the entire OS noticeably slower, by several orders of magnitude over the previous Lion (10.7.5) was running until a few days ago on my 11″ MacBook Air, but there are dozens and dozens of glaringly-obvious bugs that make me want to go back to my Linux laptop full-time.

Here are some obvious ones:

  • The trackpad randomly disables two-finger scrolling and the only way to get it back is to either log out and back in, or restart the machine entirely.
  • The direction of the trackpad scrolling was reversed after the upgrade. Dragging fingers down, used to pull the page down, now it pulls the page up. You can flip the toggle to reverse it, but why was it changed at all from the default?
  • The audio up/down buttons are about 1-2 seconds behind the actual button press, which is a bit disjointed when you’re trying to determine how far down or up to change the volume for a video or song.
  • USB Ethernet used to work plug-and-play, but now if your OS X machine is booted and you connect a USB Ethernet dongle, it will not be recognized, until you reboot the machine with the dongle plugged in. Every time. This feels like Windows to me. I never had to do this with Lion previously.
  • There’s a cut-off/echo with the voices in OS X Mavericks. When I have the clock set to announce the time every 15 minutes, instead of “It’s three-fifteen” or “It’s eleven o’clock”, I hear “…ee fifteen” or “…ven o’clock”, the first 1-2 syllables are completely missing, cut off.

There are dozens more that I’ve tripped on (and reported), but they still hamper productive use of the machine.

I also run several apps that depend on Java, including XCode, XMind, The Hit List and others. Most of these just flat-out fail to function. I was so frustrated at the amateurish quality of this major “greatest ever” OS update, that I started investigating myself.

Apple, a plea… how many Java versions, incorrect, non-current Java versions is enough? On this upgraded version of OS X (Lion -> Mavericks), I count 6+ distinct installs!

# OpenJDK Runtime Environment (build 1.7.0-internal-root_2012_07_25_17_59-b00)
./Applications/Xcode.app/Contents/Applications/Application Loader.app/Contents/MacOS/itms/java/bin/java

# Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
./Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java

# Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
./Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home/bin/java

# Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
./Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home/jre/bin/java

# Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
./System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/java

# Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
./System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java

[...]

The only one that is clean and current, is the one I installed:

# "./Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java" -version
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

Of course, you don’t use it anywhere, no apps are referring to it, and instead you refer to the other versions which crash, break or fail to correctly launch any applications that use these Java interpreters.

Please, don’t tout your OS as being the “greatest work ever”, while providing a slow, buggy, de-evolved experience from the previous versions.

Fix it, or allow us to roll back to the previous version of the OS, which did work.

UPDATE: After much testing, I determined that the short-term “solution” was to rm the symlink to ‘java’ in ‘/usr/bin/’ and point it to the version of Java I installed from Oracle, as follows:

$ sudo ls -l /usr/bin/java
lrwxr-xr-x  1 root  wheel  74 Oct 27 15:55 /usr/bin/java -> /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java
$ sudo rm /usr/bin/java
$ sudo ln -s /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java /usr/bin/java

After doing this, my Java-based OS X apps started working as expected. This is not a fix, it’s a temporary hack and workaround, but it gets me back up and running on apps that were crashing and failing before.

Apple, please fix this.

SOLVED: Remove annoying duplicate users in your WordPress blog

WordPress logoThis happens quite a bit, and I’m surprised the default WordPress install doesn’t have this fixed, but here’s a fix for cleaning up the duplicate users that will end up in your _users table in WordPress.

In my multi-site installation, that table is called ‘wpmu_users’. If you didn’t change the defaults for your site, it’s probably called ‘wp_users’.

Change this accordingly below for your site and table name.

 
# Create a temporary table and populate it with the unique users
# from the original wpmu_users table
CREATE TABLE wpmu_users_X AS 
       SELECT * FROM wpmu_users WHERE 1 GROUP BY user_login;

# Drop the original table that contains the duplicate users
DROP TABLE wpmu_users; 

# Rename the temporary table to the original table name. This
# effectively 'moves' the temporary table to the original table
# name
RENAME TABLE wpmu_users_X TO wpmu_users; 

# Alter the table attributes to set 'user_login' as a unique key
# so attempts to create a username that already exists will be
# denied
ALTER IGNORE TABLE wpmu_users ADD UNIQUE (user_login); 

Here’s what that looked like at the end of the process:

mysql> ALTER IGNORE TABLE wpmu_users ADD UNIQUE (user_login);
Query OK, 157348 rows affected (0.55 sec)
Records: 157348  Duplicates: 0  Warnings: 0

If you want to identify and kill off inactive users, spammers and such, look into the Inactive User Deleter plugin. I’ve used this with great success on my own blog, and it works great to kill off spam users that manage to sneak past the registration mechanisms, but post nothing.

That’s it, you’re done!

SOLVED: Enabling transparent desktop icons on XFCE desktop

Xfce logoAnother quick and dirty XFCE desktop hack, this one to add transparent icon backgrounds to your desktop icons.

If you’re tired of looking at ugly icon label backgrounds, and really want transparent backgrounds, the fix is simple!

Xfce4 transparent desktop icons Xfce4 transparent desktop icons

Here’s the code:

style "xfdesktop-icon-view" {
  ## opacity of text background (0 - 255, 0 = transparent)
  XfdesktopIconView::label-alpha = 0
  XfdesktopIconView::selected-label-alpha = 100
 
  ## text background colors
  base[NORMAL]    = "#edeceb"
  base[ACTIVE]    = shade (0.8, "#86abd9")
  base[SELECTED]  = "#86abd9"
 
  ## text foreground colors
  fg[NORMAL]      = shade (0.9, "#ffffff")
  fg[ACTIVE]      = shade (0.8, "#ffffff")
  fg[SELECTED]    = "#ffffff"
 
}
widget_class "*XfdesktopIconView*" style "xfdesktop-icon-view"

Just add this snippet to your “~/.gtkrc-2.0” file, kill xfdesktop (or log out and back in), and you’re all set!

Bad Behavior has blocked 877 access attempts in the last 7 days.