QEMU Mac Image Converter: tool for converting disk images

[CharlesS]

So, if you're like me, you might have some old disk images that you were using with SheepShaver, Basilisk II, Mini vMac, or another Mac emulator that contained things you wanted to test on OS 9 in QEMU, only to find that as a more faithful emulation of the old Mac hardware, QEMU is finicky about disks having driver partitions that are set up just so, and will refuse to mount disks without built-in drivers, leaving your data inaccessible despite those very drivers it wants being built into the operating system itself, thus causing much hair-pulling, weeping, and gnashing of teeth—just like a real OS 9 machine!

The solution: I've made a quick-and-dirty little conversion utility that should, in theory, take a random disk image and convert it into something that OS 9 should find palatable. Should work with standard raw files as well as old-school NDIF images, vMac .dsk images, and anything that hdiutil can mount (including compressed images), as long as said image is formatted as either HFS or HFS+ with the Apple Partition Map scheme (GPT won't work). Straight HFS(+) partitions with no partition map should work too. Operation is simple; just drop a disk image on it, and save the new image somewhere else. Don't try to replace the source image or you'll get a sarcastic error message.

If anything doesn't work that should, send me an e-mail! support (squiggly thing that looks like an "a") charlessoft (sentence ender thingy) com.

Requires Yosemite or higher.

http://www.charlessoft.com/QEMU_Mac_Image_Converter.zip

[adespoton]

Thanks, Charles! I've been doing this as a two-step process in Terminal, but this beats searching my bash history to see how I did it the last time.

Are the images all still original-emulator safe?

--I'm hoping that someone will add sparse bundle support to qemu, but as that's platform specific, I don't have too much hope.

[CharlesS]

I'm curious how you were doing it in the Terminal with just two steps. My first instinct was to try that with hdiutil and its "UNIVERSAL HD" layout, but it didn't work. It made the driver partitions, and put things in them, but OS 9 wasn't automounting the disk on the Desktop. The fact that "UNIVERSAL HD" was removed from the documentation ages ago seems somewhat telling. Anyway, I ended up deciding that figuring out what hdiutil was doing wrong and fixing it was not substantially easier than just building my own partition layout from scratch, so I ended up going with the latter.

All the resulting images should still be original-emulator safe. The only real change is that the converted images will now have the driver partitions that OS 9 throws a royal hissy fit if it doesn't see. Other than that, they're still just ordinary raw disk images. The HFS partition is copied byte by byte, so it should be completely identical, and I think that's the only part that the existing emulators are looking at anyway. I still wouldn't toss the original image before testing this pretty well, though.

[adespoton]

What I was doing was taking a properly formatted existing image, and copying the driver partition to slack space in the image I wanted to work. Sometimes this required an hdiutil resize of the existing partition first, which was annoying, and didn't always work very well. In those instances, sometimes I needed to create a new blank image and copy the driver partition from the "good" image and the hfs partition from the target image. This can all be done in two lines of bash, but depending on the condition of the original image, one of the two methods works better.

Your method sounds both safer and more sane Any chance you could do a command line version that would work on Linux and Windows as well?

[CharlesS]

What I was doing was taking a properly formatted existing image, and copying the driver partition to slack space in the image I wanted to work. Sometimes this required an hdiutil resize of the existing partition first, which was annoying, and didn't always work very well. In those instances, sometimes I needed to create a new blank image and copy the driver partition from the "good" image and the hfs partition from the target image. This can all be done in two lines of bash, but depending on the condition of the original image, one of the two methods works better.

Ah, yeah, that does sound like a bit of a pain.

Your method sounds both safer and more sane Any chance you could do a command line version that would work on Linux and Windows as well?

I did everything from scratch, so probably. It's all written in Swift 3, though, so support for other platforms would probably have to wait on the new stuff being incorporated into the Linux version of the Swift compiler (if it hasn't already). As for Windows, I still don't see a Windows version of the compiler at swift.org, so that'd probably need to wait even longer.

[adespoton]

8| There seems to be some hidden irony in there that software is being written in Swift 3 to create properly partitioned disk images that are compatible with Mac OS 9, when hdiutil can't get it right.

Thinking about all this, I think I just realized that there's not much likelihood of qemu supporting sparsebundles any time soon. It requires the same sort of pre-interpretation that would go into handling HFS partitions directly.

[CharlesS]

8| There seems to be some hidden irony in there that software is being written in Swift 3 to create properly partitioned disk images that are compatible with Mac OS 9, when hdiutil can't get it right.

I'm going with the general policy that all new projects I make for the Mac are in Swift 3. More future proof, and also I grow my experience in the language while doing it. I'm not going to go get an old copy of CodeWarrior to write it in, if that's what you were wondering.

What I really probably ought to do sometime is to merge this and HFS Disk Maker into one app, given the amount of overlap (and the fact that HFS Disk Maker could use its functionality; its current behavior of making raw HFS partitions with no partition map is starting to look kind of silly).

Thinking about all this, I think I just realized that there's not much likelihood of qemu supporting sparsebundles any time soon. It requires the same sort of pre-interpretation that would go into handling HFS partitions directly.

Sparsebundles with partitions seem to work when I try it, so I wouldn't say that it would require the same sort of pre-interpretation.

Actually, it might not require any at all. If you're willing to piggyback off hdiutil, you can actually use sparsebundles with QEMU right now. Just make sure the sparsebundle has all the needed partitions, then mount it with hdiutil attach -nomount, and point QEMU at the raw device in /dev/rdiskX (replace X with the appropriate digit). I just tried this, and it seems to boot.

[adespoton]

Doh! I should have thought of that. I'll have to script it and then roll it into all my clickable Macs Well, maybe not all of them; Mini vMac really isn't going to gain anything here, but I could even use this on my Wineskinned PearPC instances. The tricky bit will be unmounting the disks when I'm done.

This will allow me to free up ~40GB of space on my SSD, and LOTS of network traffic for my Time Machine archives :D

[CharlesS]

You could probably use this trick to install OS 9 straight from the CD, too, without having to image it first.

[adespoton]

A bit offtopic, but guess what I did?

Code: Select all

#!/bin/sh

mypath=`dirname "$0"`
rootpath=`dirname "$mypath"`
respath="$rootpath/Resources"
cd "$respath"
MOUNTPATH=(`hdiutil attach -nomount "OSX10.0.4.sparsebundle"`)
./"Mac OS X 10.0" -bios ./openbios-qemu-vga.elf -boot c -drive file="$MOUNTPATH",format=raw,media=disk -drive file=realtek-drivers.iso,format=raw,media=cdrom -M mac99 -m 256 -cpu G3 -netdev user,id=network0 -device rtl8139,netdev=network0 -prom-env 'auto-boot?=true' -g 1280x720x32 -name "- Mac OS X 10.0.4" & EMUPID=$!
wait $EMUPID
hdiutil detach "$MOUNTPATH"

This successfully attaches the sparse bundle, loads it into my embedded qemu, and then detaches it when qemu terminates :D

I also made a droppable app that does the folllowing:

Code: Select all

#!/bin/sh

echo $1
hdiutil convert "$1" -format UDSB -tgtimagekey sparse-band-size=2048 -o "$1-1Mb.sparsebundle"

This will take any DMG-compatible image and convert it to a sparsebundle with 1MB files -- the smaller files keep fragmentation bloat down. You can then use hdiutil compress to clean it up every once in a while, or just use the script again to quickly shrink down a bloated sparsebundle.

So now I can use your converter to add the partition map, then convert to sparsebundle, then mount the sparsebundle :D

[CharlesS]

You'll probably get slightly better performance connecting to the block device instead of the character device, though. This is what I put in my qemu.command file:

Code: Select all

cd "$(dirname "$0")"

DEVNUM=`hdiutil attach -nomount mac_hd.sparsebundle | perl -ne'if ($_ =~ /^\/dev\/disk([0-9]+)\s+Apple_partition_scheme/) { print($1); exit(0) }'`

./qemu-system-ppc-28pre-vga -bios ./openbios-qemu-vga.elf -boot c -M mac99 -m 256 -prom-env 'auto-boot?=true' -prom-env 'boot-args=-v' -net user -net nic,model=rtl8139 -drive file="/dev/rdisk$DEVNUM",format=raw,media=disk

hdiutil detach "disk$DEVNUM"

Additionally, I like this better because it only sends the device name to hdiutil detach, instead of all the other garbage that hdiutil attach might return.

[adespoton]

Good idea; I was going with simplicity on forcing array output and taking the first token, but perl parsing the block num will be safer, just in case hdiutil spits out something odd. I wanted to avoid spitting out and parsing the XML they recommend, although I think the XML output includes the blockref as well as the character device name.

So far I've saved 20 GB across 4 qemu instances and my Time Machine has moved from 80GB backups to 40MB backups... time to start converting my PearPC images over :D

[CharlesS]

If you don't mind your images being read-only, you could compress them and use the same trick to save even more space.

[adespoton]

I considered using compressed images with a shadow copy, but then the shadow copy ends up being a single slice, so I'd be back to the problem of ever-growing Time Machine backups. I might try one that way just to see how the two methods compare.