Tags: Debian, Encryption, linux, LUKS, Ubuntu
My daughter recently dropped my computer flat on the floor from a standing height (she’s only 5), catastrophically crashing both hard drives in the laptop. I limped along for a few days because I had to function with it for work, and only just today, decided to rip it apart, back up the data that could be read, put in two new 500GB/7200 drives and reinstall everything from scratch.
The painful part, was that my original configuration was a dm-crypted LUKS volume inside an encrypted LVM container, and mounting the volume without booting to it, is not straightforward.
The first piece was to back up the data on it, as best as I could. That was a bit trickier than I’d originally assumed, because I had to be sure I wasn’t going to “fix” the area of the drive that contained the encryption key that unlocked the encrypted LVM container.
After booting in single-user mode and putting in my passphrase to unlock the volume, I ran:
# tune2fs -l /dev/mapper/galaeus-root
It showed that the filesystem was shut down unclean, and had to be checked:
Filesystem state: not clean with errors Errors behavior: Continue
To get it to continue to check the fs until the end, regardless of state (to mark it as 100% checked and “clean”), I had to change the error state with the following:
# tune2fs -e continue /dev/mapper/galaeus-root
This makes sure that even in the face of errors, it will continue to normal execution if/when I need to boot the volume.
Next, I ran a simple
e2fsck on the volume, forcing it to fix anything that it found wrong:
# e2fsck -C0 -f -y /dev/mapper/galaeus-root
That took several hours, as it churned through the bad sectors of the drive, to mark the drive as 100% clean (with lots of damage).
Once that was done, I shut down and rebooted back into single-user mode, and ran the following:
# e2fsck -c -k -p -f /dev/mapper/galaeus-root
This now goes through and attempts to mark the bad blocks as bad, unreadable, and sets them in the bad-blocks inode. It also attempts to repair the fs (“preen”), with no user intervention (“-f”).
Now I had enough where I could go and back up the data on the volume to an external drive. I tried to boot back into the OS on the drive itself, but the fsck damaged enough of the booting userland that it wasn’t possible, so I had to back the data up with the drive plugged into another machine, externally.
I plugged the drive into an external, USB SATA bridge on another Linux machine here is where the trickery begins. Because the drive contained an encrypted LVM, with an encrypted LUKS volume inside that, with my actual partitions inside that, my host computer could not correctly identify the volumes and the encrypted volumes they contained, so even trying to decrypt it through the Debian/Ubuntu popup dialog, failed:
I had to find out which drive it was with
# fdisk -l /dev/sd? ... Disk /dev/sde: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xc1317447 Device Boot Start End Blocks Id System /dev/sde1 * 1 60770 488134993+ 83 Linux /dev/sde2 60771 60801 249007+ 5 Extended /dev/sde5 60771 60801 248976 83 Linux
To mount it, I need to use
# /sbin/cryptsetup luksOpen /dev/sde1 external Enter passphrase for /dev/sde1: device-mapper: remove ioctl failed: Device or resource busy Key slot 0 unlocked.
Now the actual device is accessible under
# stat /dev/mapper/external File: `/dev/mapper/external' Size: 0 Blocks: 0 IO Block: 4096 block special file Device: 5h/5d Inode: 224998 Links: 1 Device type: fc,6 Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) Access: 2010-01-17 13:33:49.077659138 -0500 Modify: 2010-01-17 13:33:49.077659138 -0500 Change: 2010-01-17 13:33:49.077659138 -0500
Let’s create a place to mount it, such as
# mkdir -p /mnt/external/
And now let’s mount that there:
# mount /dev/mapper/external /mnt/external/ mount: unknown filesystem type 'LVM2_member'
Oops! So the problem here, is that the encrypted LUKS volume contains an encrypted LVM, which we have to handle next. First, let’s make sure we have the appropriate LVM support and kernel module loaded:
# dpkg -l lvm2 ||/ Name Version Description +++-==============-==============-============================================ ii lvm2 2.02.39-0ubunt The Linux Logical Volume Manager
$ sudo /sbin/modprobe dm-mod $ /sbin/lsmod | grep dm dm_crypt 13035 5
Now let’s scan for LVM volumes and choose the right volume group name that we’re looking for:
$ sudo vgscan Reading all physical volumes. This may take a while... Found volume group "galaeus" using metadata type lvm2
We know the volume name on the laptop drive plugged into the external SATA bridge is “
galaeus“, so let’s activate that volume:
$ sudo vgchange -ay galaeus 2 logical volume(s) in volume group "galaeus" now active
Now we see two volumes, but we’re only really concerned with the one that has our data on it. Let’s list them to be sure:
$ sudo lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert root galaeus -wi-a- 443.65G swap_1 galaeus -wi-a- 21.87G
Now we can mount it:
$ sudo mount /dev/galaeus/root /mnt/external/
And now our data is visible once again! But that’s only the system data (
/usr, and so on), I still need to get into my own home directory. Since I couldn’t start X to log in as my normal username, my home directory was still not mounted, because it is encrypted.
Let’s look at the situation (as ‘root’ this time):
# cd /mnt/external/home/desrod/
In here, you’ll see some template data, but no user data. We need to make sure the
.ecryptfs symlink points to the mounted directory, and not the local one on the host system that the drive is plugged into for recovery. Remember, we’re still in
# rm .ecryptfs && ln -s /mnt/external/var/lib/ecryptfs/desrod .ecryptfs
We need to have a mountpoint, so let’s put that in
/mnt/ on the host also:
# mkdir /mnt/priv
And now let’s attempt to mount it. This will prompt you for several questions, all of which I’ve included below (with my own answers in bold; yours may differ):
# mount -t ecryptfs /mnt/external/home/desrod/.Private /mnt/priv Passphrase: Select cipher: 1) aes: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded) 2) blowfish: blocksize = 16; min keysize = 16; max keysize = 56 (not loaded) 3) des3_ede: blocksize = 8; min keysize = 24; max keysize = 24 (not loaded) 4) twofish: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded) 5) cast6: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded) 6) cast5: blocksize = 8; min keysize = 5; max keysize = 16 (not loaded) Selection [aes]: 1 Select key bytes: 1) 16 2) 32 3) 24 Selection : 3 Enable plaintext passthrough (y/n) [n]: n Enable filename encryption (y/n) [n]: n Attempting to mount with the following options: ecryptfs_unlink_sigs ecryptfs_key_bytes=24 ecryptfs_cipher=aes ecryptfs_sig=6536228a683536a4 WARNING: Based on the contents of [/root/.ecryptfs/sig-cache.txt], it looks like you have never mounted with this key before. This could mean that you have typed your passphrase wrong. Would you like to proceed with the mount (yes/no)? : yes Would you like to append sig [6536228a683536a4] to [/root/.ecryptfs/sig-cache.txt] in order to avoid this warning in the future (yes/no)? : no Not adding sig to user sig cache file; continuing with mount.
And that’s it! Now my encrypted home directory is mounted under
/mnt/priv, with the remainder of the encrypted backup drive mounted under
/mnt/external. From here, I can rsync all of the data off to another external drive plugged into another port, and away I go!
When I’m done, I can unmount those mountpoints, and detach the crypted device. That’s the opposite of the
luksOpen used before. You simply supply the device name to
cryptsetup, and close it:
$ sudo cryptsetup luksClose external
p.s. I’m a bit of a cleanliness nit, so when I’m done and have unmounted those mountpoints, I get rid of
/mnt/priv to keep things tidy on the system. I don’t like temporary files and directories littering the system. I could have mounted this to
/tmp and played with the data there, but
/tmp/ is world-readable, and would have allowed any other users to access it during the backup phase, so I didn’t.