QemuPPC disk performance
Moderators: Cat_7, Ronald P. Regensburg
-
- Mac Mechanic
- Posts: 178
- Joined: Sun Feb 07, 2016 4:40 pm
QemuPPC disk performance
This is yet another area that QemuPPC falls on it's face, a disk image is faster than a Ram Disk.
Re: QemuPPC disk performance
Right, that's because QEMU doesn't (yet?) have a paravirtualised driver and instead relies on the much slower device emulation - see Cat_7's comment at http://www.emaculation.com/forum/viewto ... 725#p56308.darthnvader wrote:This is yet another area that QemuPPC falls on it's face, a disk image is faster than a Ram Disk.
If you've got any MacOS development experience (unlike me) then this would be a great project to work on that would give a significant improvement in emulation speed.
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
That said, I found that playing with caching enabled me to speed up the device emulation significantly from the default. I believe enabling writethrough to a block device on an SSD was what gave me the best performance.
Re: QemuPPC disk performance
That's strange because writethrough effectively means executing a sync or equivalent for each individual transfer to ensure it hits the disk before returning rather than relying on the OS to commit batches of transfers asynchronously. Can you provide the command lines and benchmark results (including information about the host OS) to demonstrate the differences you are seeing?adespoton wrote:That said, I found that playing with caching enabled me to speed up the device emulation significantly from the default. I believe enabling writethrough to a block device on an SSD was what gave me the best performance.
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
I'm going to run some tests in writethrough, writeback, none and unsafe modes; I think it's possible that I meant writeback when I said writethrough, and I set things back to writethrough after the fact to be safe. But nothing beats hard numbers.
Host: MacBook Pro (Retina, 15-inch, Mid 2015) 2.5 GHz Intel Core i7 16 GB 1600 MHz DDR3
Guest: qemu-ppc QEMU emulator version 2.6.91 (v2.0.0-15634-g4247773) (It's the August 2016 build)
Mac OS 9.2.2
MacBench 5 Disk score: 78%
Writeback: 80% (looks like it was writethrough; these numbers sound familiar to me)
None: 135%
Unsafe: 77% (I haven't tested unsafe before, as it seemed... unsafe)
Odd set of results, but this is what I got. I suppose I could try it with a different image; this was mounting a custom-sized sparsebundle and accessing it as a block device. Reading directly from disk images seems to be slower.
[edit] from IBM's Knowledge Center:
Host: MacBook Pro (Retina, 15-inch, Mid 2015) 2.5 GHz Intel Core i7 16 GB 1600 MHz DDR3
Guest: qemu-ppc QEMU emulator version 2.6.91 (v2.0.0-15634-g4247773) (It's the August 2016 build)
Mac OS 9.2.2
Code: Select all
-bios "openbios-qemu-wip-13082016.elf" \
-boot c \
-drive file="/dev/rdisk$DEVNUM",format=raw,media=disk,cache=writethrough \
-M mac99 \
-m 256 \
-cpu G3 \
-netdev user,id=vlan0 \
-device sungem,netdev=vlan0 \
-prom-env 'auto-boot?=true' \
-g 1280x720x32 \
Writeback: 80% (looks like it was writethrough; these numbers sound familiar to me)
None: 135%
Unsafe: 77% (I haven't tested unsafe before, as it seemed... unsafe)
Odd set of results, but this is what I got. I suppose I could try it with a different image; this was mounting a custom-sized sparsebundle and accessing it as a block device. Reading directly from disk images seems to be slower.
[edit] from IBM's Knowledge Center:
In my environment, the host device is a local block device on an SSD, so the host-side operations are ludicrously faster than the guest-side operations. That said, direct I/O is taking approximately twice as long as synchronous I/O for me for some reason.For local or direct-attached storage, it is recommended that you use writethrough mode, as it ensures data integrity and has acceptable I/O performance for applications running in the guest, especially for read operations. However, caching mode none is recommended for remote NFS storage, because direct I/O operations (O_DIRECT) perform better than synchronous I/O operations (with O_SYNC). Caching mode none effectively turns all guest I/O operations into direct I/O operations on the host, which is the NFS client in this environment.
Re: QemuPPC disk performance
Eeek that is really old. There have been so many optimisations since then (including switch to proper async macio emulation, reduced locking, TCG performance improvements etc.) I'd try repeating one of your tests with 2.9 and seeing how that compares. So much happens in a short space of time in QEMUadespoton wrote:Guest: qemu-ppc QEMU emulator version 2.6.91 (v2.0.0-15634-g4247773) (It's the August 2016 build)
Re: QemuPPC disk performance
Yes,
But that build is the first that had vga, sungem and pmu support
Best,
Cat_7
But that build is the first that had vga, sungem and pmu support
Best,
Cat_7
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
Exactly Is there a 2.9 build currently that supports VGA, Sungem and PMU? I use all three
Re: QemuPPC disk performance
Nope,
But the vga driver is now very close to being included in the main code.
I'll create new builds when that has happened.
Best,
Cat_7
But the vga driver is now very close to being included in the main code.
I'll create new builds when that has happened.
Best,
Cat_7
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
Well, I'll try testing 2.9 tonight without vga, if I can get it working with 9.2.2. Ran out of time last night to get it working.
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
Hmm... anyone know how to get 9.2.2 to boot on qemu 2.9 vanilla builds? It doesn't seem to like my VGA patch, but doesn't boot at all without it.
No way for me to test OS 9 disk I/O without being able to boot to it; the August build with VGA is the latest I've been able to get working for that.
No way for me to test OS 9 disk I/O without being able to boot to it; the August build with VGA is the latest I've been able to get working for that.
Re: QemuPPC disk performance
Hi,
No idea what could be going on at your end. I just compiled from current source and I can boot both 9.22 CD and hard disk installation.
Best,
Cat_7
No idea what could be going on at your end. I just compiled from current source and I can boot both 9.22 CD and hard disk installation.
Best,
Cat_7
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
Hmm; I'll re-compile by hand then; I used the brew install for a quick check.
Re: QemuPPC disk performance
Right, but presumably you were running the old release with -M mac99p to enable VGA/sungem which is a different QEMU machine (read: different virtual hardware) and so the drivers from your *mac99p* install probably won't match the hardware from the default *mac99* machine.adespoton wrote: qemu-system-ppc -boot c -M mac99 -cpu G3 -m 256 -g 1280x720x32 -drive file="/dev/rdisk5",format=raw,media=disk,cache=writethrough
This is my standard image that works just fine with the same flags using the August 2016 build instead of 2.90.
I suspect you'll have to do a fresh install from the ISO in order to run a benchmark, however last time I did a full install of OS 9.2 it was just under 10 mins on 5 year old hardware so it won't be too onerous.
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
Thanks! I knew I was missing something. I forgot about using mac99p on those. I'll get back to configuration/speed tests once I link to a different image (I think I've got one set up already somewhere that should work)
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
New results with 2.90:
Standard RW disk image:
Writethrough: 167%.
Writeback: 168%
None: 164%
Unsafe: 168%
Optimized SparseBundle read as mounted block device:
freezes at happy mac.
Standard RW disk image:
Writethrough: 167%.
Writeback: 168%
None: 164%
Unsafe: 168%
Optimized SparseBundle read as mounted block device:
freezes at happy mac.
Re: QemuPPC disk performance
Thanks for the testing, that's a lot closer to what I was expecting
According to the latest patches on the mailing list, there should be more goodies in the next release with reducing locking on both disk I/O and display updates in the pipeline which will enhance performance even further.
According to the latest patches on the mailing list, there should be more goodies in the next release with reducing locking on both disk I/O and display updates in the pipeline which will enhance performance even further.
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
Any idea why the move from 2.8-ish to 2.9 doubled my read/write times and prevented block mounts completely?
Re: QemuPPC disk performance
In terms of doubling the read speed from disk, there are many hundreds of patches that could have improved this. If you need to know which particular patch caused the improvement you could do a rather large bisect, although is there a particular reason you need to know which one caused the improvement?
Regarding the inability to mount block devices that's not something I've seen here, so it could be a bug specific to Mac. Can you post a reproducer so that someone else with a Mac can confirm the issue? With a test case it's also possible to raise the issue upstream if required.
Regarding the inability to mount block devices that's not something I've seen here, so it could be a bug specific to Mac. Can you post a reproducer so that someone else with a Mac can confirm the issue? With a test case it's also possible to raise the issue upstream if required.
- Ronald P. Regensburg
- Expert User
- Posts: 7822
- Joined: Thu Feb 09, 2006 10:24 pm
- Location: Amsterdam, Netherlands
Re: QemuPPC disk performance
Hm, didn't adespoton mean the opposite? He wrote that read/write times were doubled, not read/write speed.
Re: QemuPPC disk performance
I'm not sure. However in every benchmark tool I've ever used, 100% represents the speed of what you are comparing against so 160% means an increase in performance of 60% etc. which should match with the recent improvments in QEMU. Or is this tool different somehow?
- adespoton
- Forum All-Star
- Posts: 4227
- Joined: Fri Nov 27, 2009 5:11 am
- Location: Emaculation.com
- Contact:
Re: QemuPPC disk performance
In MacBench, I thought that the disk percentages had to do with write times (smaller being better) and then that all got applied against the overall benchmark score where larger was better. However, reviewing the screens, it appears this may not be the case.
In which case, it's kind of odd that writeback performance is higher than straight passthrough isn't it? Or is the caching actually helping things with small reads and writes somehow?
In which case, it's kind of odd that writeback performance is higher than straight passthrough isn't it? Or is the caching actually helping things with small reads and writes somehow?
Re: QemuPPC disk performance
Good question. Generally the write* options will get passed to the OS as option flags when opening the backing file. Perhaps using an SSD helps narrow the gap here because writes are fast? I'm also not sure what effect sparse bundles vs. a raw partition would have either.
Effectively the option flags for the write caching are passed through to the kernel so you're at the mercy of the Darwin kernel as to how exactly it interprets those flags (sadly I don't believe Apple publish their kernel sources anymore).
The reports I've heard suggest that writing to qcow2 images is faster than raw images since QEMU's disk layer can cache the writes for subsequent reads itself and sync asynchronously, although I should add this is second-hand information and not something I've directly tested myself.
Effectively the option flags for the write caching are passed through to the kernel so you're at the mercy of the Darwin kernel as to how exactly it interprets those flags (sadly I don't believe Apple publish their kernel sources anymore).
The reports I've heard suggest that writing to qcow2 images is faster than raw images since QEMU's disk layer can cache the writes for subsequent reads itself and sync asynchronously, although I should add this is second-hand information and not something I've directly tested myself.
Re: QemuPPC disk performance
If the information over here is true, the kernel sources can be found at:mcayland wrote:… sadly I don't believe Apple publish their kernel sources anymore
https://opensource.apple.com/source/xnu/
EDIT:
Actually, if you go to https://opensource.apple.com/ and click on the desired macOS version, you can scroll to the bottom and find the corresponding XNU (Darwin) version. You can also access it by clicking on it there.