Login  •  Register


The time is now: Sat May 27, 2017 12:39 am

Emaculation wiki  •  Delete all board cookies



Post new topic  Reply to topic Page 1 of 1 [ 25 posts ]
Print view Previous topic  |  Next topic
Author Message
 Post subject: QemuPPC disk performance
PostPosted: Mon May 01, 2017 4:01 pm 
Offline
Tinkerer

Joined: Sun Feb 07, 2016 4:40 pm
Posts: 99
This is yet another area that QemuPPC falls on it's face, a disk image is faster than a Ram Disk.

Image


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Mon May 01, 2017 8:59 pm 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
darthnvader wrote:
This is yet another area that QemuPPC falls on it's face, a disk image is faster than a Ram Disk.

Right, that's because QEMU doesn't (yet?) have a paravirtualised driver and instead relies on the much slower device emulation - see Cat_7's comment at viewtopic.php?f=34&t=7047&start=1725#p56308.

If you've got any MacOS development experience (unlike me) then this would be a great project to work on that would give a significant improvement in emulation speed.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Mon May 01, 2017 9:56 pm 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
That said, I found that playing with caching enabled me to speed up the device emulation significantly from the default. I believe enabling writethrough to a block device on an SSD was what gave me the best performance.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Mon May 01, 2017 10:52 pm 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
adespoton wrote:
That said, I found that playing with caching enabled me to speed up the device emulation significantly from the default. I believe enabling writethrough to a block device on an SSD was what gave me the best performance.


That's strange because writethrough effectively means executing a sync or equivalent for each individual transfer to ensure it hits the disk before returning rather than relying on the OS to commit batches of transfers asynchronously. Can you provide the command lines and benchmark results (including information about the host OS) to demonstrate the differences you are seeing?


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Tue May 02, 2017 3:24 am 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
I'm going to run some tests in writethrough, writeback, none and unsafe modes; I think it's possible that I meant writeback when I said writethrough, and I set things back to writethrough after the fact to be safe. But nothing beats hard numbers.

Host: MacBook Pro (Retina, 15-inch, Mid 2015) 2.5 GHz Intel Core i7 16 GB 1600 MHz DDR3
Guest: qemu-ppc QEMU emulator version 2.6.91 (v2.0.0-15634-g4247773) (It's the August 2016 build)
Mac OS 9.2.2
Code:
  -bios "openbios-qemu-wip-13082016.elf" \
  -boot c \
  -drive file="/dev/rdisk$DEVNUM",format=raw,media=disk,cache=writethrough \
  -M mac99 \
  -m 256 \
  -cpu G3 \
  -netdev user,id=vlan0 \
  -device sungem,netdev=vlan0 \
  -prom-env 'auto-boot?=true' \
  -g 1280x720x32 \

MacBench 5 Disk score: 78%

Writeback: 80% (looks like it was writethrough; these numbers sound familiar to me)

None: 135%

Unsafe: 77% (I haven't tested unsafe before, as it seemed... unsafe)

Odd set of results, but this is what I got. I suppose I could try it with a different image; this was mounting a custom-sized sparsebundle and accessing it as a block device. Reading directly from disk images seems to be slower.

[edit] from IBM's Knowledge Center:
Quote:
For local or direct-attached storage, it is recommended that you use writethrough mode, as it ensures data integrity and has acceptable I/O performance for applications running in the guest, especially for read operations. However, caching mode none is recommended for remote NFS storage, because direct I/O operations (O_DIRECT) perform better than synchronous I/O operations (with O_SYNC). Caching mode none effectively turns all guest I/O operations into direct I/O operations on the host, which is the NFS client in this environment.


In my environment, the host device is a local block device on an SSD, so the host-side operations are ludicrously faster than the guest-side operations. That said, direct I/O is taking approximately twice as long as synchronous I/O for me for some reason.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Tue May 02, 2017 4:39 pm 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
adespoton wrote:
Guest: qemu-ppc QEMU emulator version 2.6.91 (v2.0.0-15634-g4247773) (It's the August 2016 build)


Eeek that is really old. There have been so many optimisations since then (including switch to proper async macio emulation, reduced locking, TCG performance improvements etc.) I'd try repeating one of your tests with 2.9 and seeing how that compares. So much happens in a short space of time in QEMU :)


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Tue May 02, 2017 4:49 pm 
Offline
Expert User
User avatar

Joined: Fri Feb 13, 2004 8:59 am
Posts: 4088
Location: Sittard, The Netherlands
Yes,

But that build is the first that had vga, sungem and pmu support ;-)

Best,
Cat_7


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Tue May 02, 2017 9:14 pm 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
Exactly :) Is there a 2.9 build currently that supports VGA, Sungem and PMU? I use all three :)


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Wed May 03, 2017 7:05 am 
Offline
Expert User
User avatar

Joined: Fri Feb 13, 2004 8:59 am
Posts: 4088
Location: Sittard, The Netherlands
Nope,

But the vga driver is now very close to being included in the main code.
I'll create new builds when that has happened.

Best,
Cat_7


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Wed May 03, 2017 4:51 pm 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
Well, I'll try testing 2.9 tonight without vga, if I can get it working with 9.2.2. Ran out of time last night to get it working.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Sat May 06, 2017 3:52 am 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
Hmm... anyone know how to get 9.2.2 to boot on qemu 2.9 vanilla builds? It doesn't seem to like my VGA patch, but doesn't boot at all without it.

No way for me to test OS 9 disk I/O without being able to boot to it; the August build with VGA is the latest I've been able to get working for that.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Sat May 06, 2017 7:57 am 
Offline
Expert User
User avatar

Joined: Fri Feb 13, 2004 8:59 am
Posts: 4088
Location: Sittard, The Netherlands
Hi,

No idea what could be going on at your end. I just compiled from current source and I can boot both 9.22 CD and hard disk installation.

Best,
Cat_7


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Sun May 07, 2017 3:30 am 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
Hmm; I'll re-compile by hand then; I used the brew install for a quick check.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Wed May 10, 2017 4:18 am 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
This is just odd...
Image

qemu-system-ppc -boot c -M mac99 -cpu G3 -m 256 -g 1280x720x32 -drive file="/dev/rdisk5",format=raw,media=disk,cache=writethrough

This is my standard image that works just fine with the same flags using the August 2016 build instead of 2.90.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Wed May 10, 2017 1:03 pm 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
adespoton wrote:
qemu-system-ppc -boot c -M mac99 -cpu G3 -m 256 -g 1280x720x32 -drive file="/dev/rdisk5",format=raw,media=disk,cache=writethrough

This is my standard image that works just fine with the same flags using the August 2016 build instead of 2.90.


Right, but presumably you were running the old release with -M mac99p to enable VGA/sungem which is a different QEMU machine (read: different virtual hardware) and so the drivers from your *mac99p* install probably won't match the hardware from the default *mac99* machine.

I suspect you'll have to do a fresh install from the ISO in order to run a benchmark, however last time I did a full install of OS 9.2 it was just under 10 mins on 5 year old hardware so it won't be too onerous.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Wed May 10, 2017 4:44 pm 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
Thanks! I knew I was missing something. I forgot about using mac99p on those. I'll get back to configuration/speed tests once I link to a different image (I think I've got one set up already somewhere that should work)


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Fri May 12, 2017 5:02 am 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
New results with 2.90:
Standard RW disk image:
Writethrough: 167%.
Writeback: 168%
None: 164%
Unsafe: 168%

Optimized SparseBundle read as mounted block device:
freezes at happy mac.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Mon May 15, 2017 7:06 am 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
Thanks for the testing, that's a lot closer to what I was expecting :)

According to the latest patches on the mailing list, there should be more goodies in the next release with reducing locking on both disk I/O and display updates in the pipeline which will enhance performance even further.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Mon May 15, 2017 6:08 pm 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
Any idea why the move from 2.8-ish to 2.9 doubled my read/write times and prevented block mounts completely?


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Mon May 15, 2017 8:41 pm 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
In terms of doubling the read speed from disk, there are many hundreds of patches that could have improved this. If you need to know which particular patch caused the improvement you could do a rather large bisect, although is there a particular reason you need to know which one caused the improvement?

Regarding the inability to mount block devices that's not something I've seen here, so it could be a bug specific to Mac. Can you post a reproducer so that someone else with a Mac can confirm the issue? With a test case it's also possible to raise the issue upstream if required.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Tue May 16, 2017 7:35 am 
Offline
Expert User
User avatar

Joined: Thu Feb 09, 2006 10:24 pm
Posts: 4720
Location: Amsterdam, Netherlands
Hm, didn't adespoton mean the opposite? He wrote that read/write times were doubled, not read/write speed.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Tue May 16, 2017 8:01 am 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
I'm not sure. However in every benchmark tool I've ever used, 100% represents the speed of what you are comparing against so 160% means an increase in performance of 60% etc. which should match with the recent improvments in QEMU. Or is this tool different somehow?


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Tue May 16, 2017 8:02 pm 
Offline
Forum All-Star
User avatar

Joined: Fri Nov 27, 2009 5:11 am
Posts: 1599
In MacBench, I thought that the disk percentages had to do with write times (smaller being better) and then that all got applied against the overall benchmark score where larger was better. However, reviewing the screens, it appears this may not be the case.

In which case, it's kind of odd that writeback performance is higher than straight passthrough isn't it? Or is the caching actually helping things with small reads and writes somehow?


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Fri May 19, 2017 8:02 pm 
Offline
Granny Smith

Joined: Sun Nov 01, 2015 10:33 pm
Posts: 102
Good question. Generally the write* options will get passed to the OS as option flags when opening the backing file. Perhaps using an SSD helps narrow the gap here because writes are fast? I'm also not sure what effect sparse bundles vs. a raw partition would have either.

Effectively the option flags for the write caching are passed through to the kernel so you're at the mercy of the Darwin kernel as to how exactly it interprets those flags (sadly I don't believe Apple publish their kernel sources anymore).

The reports I've heard suggest that writing to qcow2 images is faster than raw images since QEMU's disk layer can cache the writes for subsequent reads itself and sync asynchronously, although I should add this is second-hand information and not something I've directly tested myself.


Top
 Profile  
Reply with quote Post a reply  
PostPosted: Fri May 19, 2017 10:23 pm 
Online
Granny Smith

Joined: Wed Apr 10, 2013 9:32 am
Posts: 128
mcayland wrote:
… sadly I don't believe Apple publish their kernel sources anymore

If the information over here is true, the kernel sources can be found at:
https://opensource.apple.com/source/xnu/

EDIT:
Actually, if you go to https://opensource.apple.com/ and click on the desired macOS version, you can scroll to the bottom and find the corresponding XNU (Darwin) version. You can also access it by clicking on it there.


Top
 Profile  
Reply with quote Post a reply  
Display posts from previous:  Sort by  
Post new topic  Reply to topic Page 1 of 1 [ 25 posts ]


Who is online

Users browsing this forum: mabam and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
 

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group