Converting a Physical Disk to a VMWare Virtual Disk
[Update: there is an alternative method here. It has better success and is more reliable for partitions that reside in the second or later primary partition positions. It is also a more user friendly.]
I have a Windows XP SP2 installation that I use for work, while my current personal OS of choice is Windows Vista. I only work weekends, so rebooting wasn’t too bothersome initially. Over time it has become more and more of a chore, as I can’t easily switch from one development environment to another without rebooting. I decided it was time to make my work installation a virtual machine.
I had some additional complications that made the process a little non-standard. I originally tried using VMWare’s converter tool, but it would fail at 97% of the creation of the disk. I then tried using a Windows port of the Unix utility ‘dd’ to create a raw image of the disk, but because the Windows volume manager was accessing the disk, dd would give me access errors. Additionally, the VMWare converter doesn’t support converting from a raw image (…grr…), so Qemu’s qemu-img tool would have to be used to convert the raw image to a disk in VMWare’s vmdk format.
The steps to reach our goal aren’t too complicated, and can be replicated by others easily. To do it, I used:
- VMWare Workstation
- A Linux installation or LiveCD (I used my existing Debian installation, but something like Knoppix would work fine)
- Qemu
- NTFS-3G (if you plan on writing out to an NTFS partition from Linux, as I did)
I started by booting into my Linux install. Linux only mounts the disks it uses (hint hint, Microsoft) so we can access all sectors of the partition to make a dump of the disk with dd. I first had to mount the partition where I wanted the output file to reside, which uses NTFS:
ntfs-3g /dev/sdb1 /mnt/external
Next, create the image. I did this with the following (substitute your device/partition and output file):
dd if=/dev/hdc3 of=/mnt/external/diskImage/XPSP2.img bs=1024
When that finishes, the file specified with the ‘of’ option in dd will contain a block-by-block exact copy of your partition. However, it is in a raw format - we need it in a format VMWare can read. This is where Qemu comes in. Qemu is distributed with qemu-img, a tool used for creating, manipulating, and converting images. Specifically, our goal is to use qemu-img’s convert functionality to convert from a raw image format to the vmdk format. This is accomplished with:
qemu-img convert -f raw /mnt/external/diskImage/XPSP2.img -O vmdk /mnt/external/diskImage/XPSP2.vmdk
Be prepared to wait. For a 40gb image, this process took roughly 12 hours. Since qemu-img provides no status as to how far it has come, I kept tabs on it just by monitoring the filesize of the output image. This is entirely unnecessary, but if you want to do the same, just open a new terminal and type the following:
while [1 -gt 0 ]; do du -hs /mnt/external/diskImage/XPSP2.vmdk; sleep 10; clear; done
This will just print out the size of the file on your screen so you can watch it grow. Alternatively, Roberte provided a tip in the comments that suggested using the “watch” command. Either will work:
watch ‘ls -lh /mnt/external/diskImage/XPSP.vmdk’
When the process is completed, boot back into Windows (or if you are using Linux as the host, stay put) and create a virtual machine around your new disk image. Don’t forget to remove the original img created with dd, it is a huge waste of disk space :)
[Notes]
- This process is really only feasible if you have a lot of disk space. At worst, the disk requirements are greater than 2*P, where P is the partition size of the virtual machine you wish to create. However, qemu-img only writes out actual data, not empty sectors, so your output image will be the size of the used space in the input image. For my conversion, I used over 40gb (input)+15gb (output) of disk space, which was reclaimed with the deletion of the output of dd, and resizing another partition to use the old physical installation’s space.
- qemu-img doesn’t support stream input, which is why we can’t pipe dd’s output directly into qemu-img convert. This would have reduced the disk requirements to only the size of the vmdk image, and sped up the process substantially. Bug the Qemu developers to implement this feature :)
You may be able to skip the dd command. Try this command:
qemu-img convert -f raw /dev/hdc3 -O vmdk boot.vmdk
I think this should work just fine. dd is just reading from the disk and qemu-img would not know the difference.
I tested on /dev/hda1 (/boot) and it worked just fine.
good luck.
Grant
Feb 24th, 2008 at 5:30 am
Further to my previous post - I think you may have stumbled on the single most simple way of converting the system from physical to virtual.
I think the following procedure would work, however I have not had time to test it.
on the machine to be converted (physical machine) nfs mount vmware host drive where you will be putting the virtual disks. Then run the qemu-img convertion from /dev/hda putting the output on the nfs mounted partition. This will be a one step conversion directly to the final location. It might be best to shut down as many processes as possible on the source host.
I’ll give this a go and report back.
Feb 24th, 2008 at 5:50 am
Thanks for sharing, Grant! I haven’t had a need to revisit this technique for a while now, but I’m really looking forward to seeing how your second suggestion works out.
Thanks!
Feb 24th, 2008 at 1:16 pm
Could you please explain: “create a virtual machine around your new disk image.”
I tried importing the vmdk image into vmware-server under ubuntu with no success. It only accepts vmx files.
Any hints?
Feb 28th, 2008 at 5:08 am
I tried creating a custom VM under vmware-server and told it to use an existing vmware image (the qemu converted one) but vmware crahes with no errors after I select the image…
Feb 28th, 2008 at 5:15 am
Solved: I created a .vmx file (copied from another vm with the same guest os) and changed it accordingly.
Then I put .vmx and .vmdk files into a directory under
/var/lib/vmware-server/Virtual Machines and
Feb 28th, 2008 at 7:10 am
I followed this guide and then created a virtual machine (in vmware workstation 6.x linux) using the created .vmdk (from /dev/hda3, containing win xp sp2 c:-drive) as the virtual disk. But it doesn’t want to boot. All I get is: “A disk read error occurred. Press ctrl+alt+del to restart.”
I tried booting the VM with the winxp install-cd and run fixmbr and fixboot (not on the same .vmdk), but that didn’t help. When booting the “fixmbr”-image, I get some error that the partition table has been damaged. When booting the “fixboot”-image, I get the same error as without the “fix”.
How do I fix this?
Feb 29th, 2008 at 3:08 pm
@istvan:
I ran into the same problems you did. VMWare apparently *really* doesn’t like using images created from individual partitions. The reason isn’t documented, but I believe it is because the partition table for the disk is located at the beginning of the disk, which isn’t being copied. So you are left with a single partition that VMWare believes is the whole disk, and no information as to the partition’s geometry. Additionally, there is no room for an MBR at the beginning of the partition, because it jumps straight into the data section. My suggestion is to use LiveView, which I wrote about here:
http://www.robertpeaslee.com/index.php/raw-image-to-vmdk-the-easy-way/
If that doesn’t work, then… well, I have ideas, but they aren’t fun :) Let me know if LiveView works for you. Good luck mate!
Mar 1st, 2008 at 5:07 pm
Great article, just want to add, most distros include a very useful utility called ‘watch’ you can also do:
# watch ‘ls -l *.vmdk’
it will update every 2 seconds with it’s default settings.
Converting a 40GB XP image now. Hope it works. :)
May 3rd, 2008 at 7:49 pm
@Roberte:
Thanks for the tip! I didn’t know about that command, but I’m updating a couple of my scripts to incorporate it. I’m also including it in my post, I appreciate it :)
I don’t know if you caught this, but if your conversion doesn’t work, there is an easier way to do it that is more reliable. I wrote about it here:
http://www.robertpeaslee.com/index.php/raw-image-to-vmdk-the-easy-way/
Good luck!
May 4th, 2008 at 7:38 pm
The watch command is nice, but only exists in Linux best I can tell.
May 14th, 2008 at 7:33 pm
qemu-img conversion is only that slow when target and destination files are on the same disk. Nice tips though!
May 29th, 2008 at 3:18 pm
Thanks to your article I was able to overcome a client’s predicament when their Windows 98 machine on which their custom legacy application was running lost its motherboard. I was not able to obtain a Win98 compatible replacement motherboard, so now they are running their grotty old 98 machine in a window on a spanking new dual core system with Mandriva 2008.1 as the host for VMPlayer.
The only trick was to dd the entire physical volume and not just the partition so that the MBR and partition table was retrieved, and to use two different disks, one for the source dd image and one for the destination vmdk image, in order to speed things up.
Jul 30th, 2008 at 11:21 am
@Duncan:
Glad you found the article helpful! And thanks for leaving the tips too, I’m sure others will appreciate them.
Jul 30th, 2008 at 5:00 pm
Thanks for the article! It was a lot of help. I actually used this to convert two old Windows 98 machines into VMs on a new multi-core system. The interesting part was the speed at which Qemu-img converted from img to vmdk. The first drive was 8Gb, and I was working on two separate HDs to increase performance, and it took ~1.5-2hrs to convert. The second drive was 13Gb, same setup, and it took ~24hrs to convert. My only guess as to the vastly different process times is perhaps disk fragmentation, or data density (not easily archived content), or both. Anyone else experience this kind of disparity?
Aug 4th, 2008 at 2:30 pm