GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

About Qemu-system-ppc, a PPC Mac emulator for Windows, macOS and Linux that can run Mac OS 9.0 up to Mac OS X 10.5

Moderators: Cat_7, Ronald P. Regensburg

User avatar
adespoton
Forum All-Star
Posts: 4208
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by adespoton »

Alternatively, if you have access to Ida, it will happily detect PPC assembly and output it in readable form. I believe there are demo versions of Ida for Mac available as well from the Ida website.

Thanks for getting this pushed out, Programmingkid! I have limited time over the next few months, but I'll eventually have my G4s hooked up to the same network as my emulation Mac, and will run everything through its paces.

Out of curiosity, what is your current list of buggy instructions? We could compare this against the ppc asm of an audio program (or Quicktime) and see where the overlap is to prioritize reimplementation.
Programmingkid
Apple Corer
Posts: 243
Joined: Sun Jan 31, 2016 6:01 pm

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by Programmingkid »

adespoton wrote:Alternatively, if you have access to Ida, it will happily detect PPC assembly and output it in readable form. I believe there are demo versions of Ida for Mac available as well from the Ida website.
I couldn't find a free demo of IDA that would disassemble PowerPC code. If you find it, please share the link with us. It does look like a great tool.
adespoton wrote: Thanks for getting this pushed out, Programmingkid! I have limited time over the next few months, but I'll eventually have my G4s hooked up to the same network as my emulation Mac, and will run everything through its paces.

Out of curiosity, what is your current list of buggy instructions? We could compare this against the ppc asm of an audio program (or Quicktime) and see where the overlap is to prioritize reimplementation.
We definitely need someone to look at the G4 emulation. I hope to do more thorough testing later in the week, but here is what I found so far as problem instructions:

- fdivs
- fctiw
- fdiv
- divw
- fctiwz
- fsub

The emulated FPSCR appears to be the major problem. It is a register that stores settings and indicates the status of certain floating point operations. I plan on making some C code that will print all out of the settings of the FPSCR. When I do I will post it here for others to use to help debug QEMU.

To everyone who wants to help, risu is easy enough for most QEMU users to handle. You don't have to be a programmer to use it. The complete listing of supported PowerPC instructions can be found in the ppc.risu file. It can help you decide which instructions you wish to test.
darthnvader
Mac Mechanic
Posts: 178
Joined: Sun Feb 07, 2016 4:40 pm

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by darthnvader »

Programmingkid wrote:
adespoton wrote:Alternatively, if you have access to Ida, it will happily detect PPC assembly and output it in readable form. I believe there are demo versions of Ida for Mac available as well from the Ida website.
I couldn't find a free demo of IDA that would disassemble PowerPC code. If you find it, please share the link with us. It does look like a great tool.
adespoton wrote: Thanks for getting this pushed out, Programmingkid! I have limited time over the next few months, but I'll eventually have my G4s hooked up to the same network as my emulation Mac, and will run everything through its paces.

Out of curiosity, what is your current list of buggy instructions? We could compare this against the ppc asm of an audio program (or Quicktime) and see where the overlap is to prioritize reimplementation.
We definitely need someone to look at the G4 emulation. I hope to do more thorough testing later in the week, but here is what I found so far as problem instructions:

- fdivs
- fctiw
- fdiv
- divw
- fctiwz
- fsub

The emulated FPSCR appears to be the major problem. It is a register that stores settings and indicates the status of certain floating point operations. I plan on making some C code that will print all out of the settings of the FPSCR. When I do I will post it here for others to use to help debug QEMU.

To everyone who wants to help, risu is easy enough for most QEMU users to handle. You don't have to be a programmer to use it. The complete listing of supported PowerPC instructions can be found in the ppc.risu file. It can help you decide which instructions you wish to test.
Good work, Qemu's PPC emulation is about 1/2 to 1/3 of SS on my system and Altivec is about 1/10 of SS, and SS is nowhere near the performance of Apple's Rosetta.

If we could get CPU emulation up near what Rosetta can do, that would be nice, tho I don't think QemuPPC has a JIT.

I've put my Voodoo emulation on hold, until I get a motherboard that supports IOMMU( VT-d ). I'm having trouble getting the 3dfx drivers to load, under OS 9, so I need to plug in a real Voodoo cards and figure out what I'm doing wrong.
Programmingkid
Apple Corer
Posts: 243
Joined: Sun Jan 31, 2016 6:01 pm

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by Programmingkid »

darthnvader wrote:Good work, Qemu's PPC emulation is about 1/2 to 1/3 of SS on my system and Altivec is about 1/10 of SS, and SS is nowhere near the performance of Apple's Rosetta.
Sheepshaver sound like the superior technology. I think it has a two stage translation system: PPC -> x86. QEMU has a three stage system: PPC -> TCG -> x86. The extra stage is definitely going to slow down things.
darthnvader wrote: If we could get CPU emulation up near what Rosetta can do, that would be nice, tho I don't think QemuPPC has a JIT.
It does. It is called Tiny Code Generator (TCG). I do wish someone would replace it with something that translates PPC to x86 without any extra stages.
darthnvader wrote: I've put my Voodoo emulation on hold, until I get a motherboard that supports IOMMU( VT-d ). I'm having trouble getting the 3dfx drivers to load, under OS 9, so I need to plug in a real Voodoo cards and figure out what I'm doing wrong.
I wonder now if there is an easier way. A large amount of games support OpenGL. What if we implemented a video card that just used OpenGL? We would need to make guest OS drivers for it. I think with all the documentation and code we would have access to, making an OpenGL card might be much easier than implementing a real proprietary card. We would be spared the pain of reverse engineering hardware and closed source drivers. Does this sound feasible?
darthnvader
Mac Mechanic
Posts: 178
Joined: Sun Feb 07, 2016 4:40 pm

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by darthnvader »

Programmingkid wrote:
darthnvader wrote:Good work, Qemu's PPC emulation is about 1/2 to 1/3 of SS on my system and Altivec is about 1/10 of SS, and SS is nowhere near the performance of Apple's Rosetta.
Sheepshaver sound like the superior technology. I think it has a two stage translation system: PPC -> x86. QEMU has a three stage system: PPC -> TCG -> x86. The extra stage is definitely going to slow down things.
darthnvader wrote: If we could get CPU emulation up near what Rosetta can do, that would be nice, tho I don't think QemuPPC has a JIT.
It does. It is called Tiny Code Generator (TCG). I do wish someone would replace it with something that translates PPC to x86 without any extra stages.
darthnvader wrote: I've put my Voodoo emulation on hold, until I get a motherboard that supports IOMMU( VT-d ). I'm having trouble getting the 3dfx drivers to load, under OS 9, so I need to plug in a real Voodoo cards and figure out what I'm doing wrong.
I wonder now if there is an easier way. A large amount of games support OpenGL. What if we implemented a video card that just used OpenGL? We would need to make guest OS drivers for it. I think with all the documentation and code we would have access to, making an OpenGL card might be much easier than implementing a real proprietary card. We would be spared the pain of reverse engineering hardware and closed source drivers. Does this sound feasible?
There's the Gallium3D project, and Virgil 3d project that already works with Qemu. The trouble is I don't know how to write drivers for OS 9, I've read through the old documentation, but it's all over my head. I've ported a few Linux pci device drivers to OS X, and even wrote my own driver for nVidia cards, to change the clock rates, control the TV Out, and report some values of the pci registers.
CharlesS
Granny Smith
Posts: 107
Joined: Thu Aug 14, 2008 9:05 am

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by CharlesS »

Programmingkid wrote:
darthnvader wrote:Good work, Qemu's PPC emulation is about 1/2 to 1/3 of SS on my system and Altivec is about 1/10 of SS, and SS is nowhere near the performance of Apple's Rosetta.
Sheepshaver sound like the superior technology. I think it has a two stage translation system: PPC -> x86. QEMU has a three stage system: PPC -> TCG -> x86. The extra stage is definitely going to slow down things.
Well, that depends on your definition of "superior." SheepShaver has more bugs than a summer's day in Minnesota, and is more crash-prone than a Formula One racer on meth. But it is fast.

I for one am content to have a slower emulator, if it is reliable.
There's no earthly way of knowing, which direction we are going, for the rowers keep on rowing, and they're certainly not showing any signs that they are slowing.
User avatar
sentient06
Mac Mechanic
Posts: 188
Joined: Tue Mar 29, 2011 8:57 pm
Location: London, UK

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by sentient06 »

CharlesS wrote: Well, that depends on your definition of "superior." SheepShaver has more bugs than a summer's day in Minnesota, and is more crash-prone than a Formula One racer on meth. But it is fast.

I for one am content to have a slower emulator, if it is reliable.
Great metaphors/puns! :mrgreen:
User avatar
mabam
Master Emulator
Posts: 497
Joined: Wed Apr 10, 2013 9:32 am

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by mabam »

I don't like all these rating possibilities on the net. But for comments like CharlesS' we should introduce the possibility to up vote a post :-)
User avatar
adespoton
Forum All-Star
Posts: 4208
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by adespoton »

Indeed: SheepShaver is the experimental track-only emulator, where qemu is more like a tank: slow, but it can go pretty much everywhere. Neither are quite ready for a regular commute.
User avatar
Cat_7
Expert User
Posts: 6121
Joined: Fri Feb 13, 2004 8:59 am
Location: Sittard, The Netherlands

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by Cat_7 »

It does. It is called Tiny Code Generator (TCG). I do wish someone would replace it with something that translates PPC to x86 without any extra stages.
That tiny code generator is the successor of the now problematic jit that is included in SheepShaver. Qemu used to use the same jit. That's why at one point I contacted one of the tgc developers to get some insights into how to adapt sheepshaver to the tgc. But the answer was way over my head...

A better speed comparison could be made by looking at SheepShaver performance with the jit disabled.

It looks like enabling a multi threaded tcg for 32 bit ppc is considered for the future. The 64 bit ppc version is currently available.

We'll see what speed improvements that brings.

Best,
Cat_7
CharlesS
Granny Smith
Posts: 107
Joined: Thu Aug 14, 2008 9:05 am

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by CharlesS »

adespoton wrote:Indeed: SheepShaver is the experimental track-only emulator, where qemu is more like a tank: slow, but it can go pretty much everywhere. Neither are quite ready for a regular commute.
Actually, SheepShaver is more like a skateboard with a rocket from the ACME corporation strapped to it. If it works, you'll go really fast, but it's more likely to either blow up, go the wrong direction, slam you into a rock wall, swerve into a lake, cause you to fall off a cliff, or fail in some other crazy way that only Chuck Jones could come up with.
There's no earthly way of knowing, which direction we are going, for the rowers keep on rowing, and they're certainly not showing any signs that they are slowing.
User avatar
adespoton
Forum All-Star
Posts: 4208
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by adespoton »

Ah; so it's the hoverboard of emulators ;)
Programmingkid
Apple Corer
Posts: 243
Joined: Sun Jan 31, 2016 6:01 pm

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by Programmingkid »

I made a program that tests various floating point instructions. Just double clicking on the "fpscr test" program is all it takes to make it work. The program indicated that most of the floating point instructions in qemu-system-ppc are not correctly implemented. If you want to try testing it I would suggest using Mac OS 10.4 as a guest operating system. The source code is included.

http://www.mediafire.com/file/6j9tqubvk ... rogram.zip

When I ran the program I saw 18 failures. :sad:
andreas_g
Student Driver
Posts: 20
Joined: Thu Dec 02, 2010 5:50 pm
Location: Austria

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by andreas_g »

Floating-point math is probably the most tricky part of CPU/FPU emulation. I have spent lots of time improving 68k FPU emulation to a point where one expect to get "99% correct results" recently. I don't know how QEMU emulates floats (native or software floating point library?). Using any native floats will lead to different results on different platforms, which is definitely bad behavior for an emulator. I decided to use SoftFloat as a base for FPU emulation. But I had to modify it heavily to generate the same results as a real 68k does (even though the 68k is IEEE compliant!).
The main problem is handling of edge cases and exceptions. I also had to extend SoftFloat with additional functions (FSCALE, FMOD, FSIN, FLOGN, ...), but I think that won't be an issue emulating a RISC processor.
I think (it is really just a guess) your problems might derive from the emulator not generating traps if certain exceptions occur. The RISC design might make use of those exceptions to implement missing features, like handling of NaN and infinity, in software (for example the i860 RISC processor does that).
Programmingkid
Apple Corer
Posts: 243
Joined: Sun Jan 31, 2016 6:01 pm

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by Programmingkid »

andreas_g wrote:Floating-point math is probably the most tricky part of CPU/FPU emulation. I have spent lots of time improving 68k FPU emulation to a point where one expect to get "99% correct results" recently. I don't know how QEMU emulates floats (native or software floating point library?). Using any native floats will lead to different results on different platforms, which is definitely bad behavior for an emulator. I decided to use SoftFloat as a base for FPU emulation. But I had to modify it heavily to generate the same results as a real 68k does (even though the 68k is IEEE compliant!).
The main problem is handling of edge cases and exceptions. I also had to extend SoftFloat with additional functions (FSCALE, FMOD, FSIN, FLOGN, ...), but I think that won't be an issue emulating a RISC processor.
I think (it is really just a guess) your problems might derive from the emulator not generating traps if certain exceptions occur. The RISC design might make use of those exceptions to implement missing features, like handling of NaN and infinity, in software (for example the i860 RISC processor does that).
Your work sounds impressive. Which emulator do you work on?

QEMU uses a softfloat implementation. I have dreamed about implementing an x87 (x86 floating point unit) backend.

Interesting fact Rosetta actually has an even worse PowerPC floating point implementation than QEMU. 23 tests failed for Rosetta compared to QEMU's 18 failures.

The testing has shown that the PowerPC floating point status and control register is not correctly implemented. All of the calculations done in the tests are correct.
andreas_g
Student Driver
Posts: 20
Joined: Thu Dec 02, 2010 5:50 pm
Location: Austria

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by andreas_g »

I am working on Previous (NeXT Computer emulator). My FPU improvements have only been possible with the help of Toni Wilen (WinUAE), who tested lots of instructions on his machines. Documentation of the 68k is not clear about many details.
If QEMU uses SoftFloat then there is a good chance to be able to implement the missing things. The only problem will be that SoftFloat is also used to emulate other platforms which is likely to cause conflicts when one changes SoftFloat's behavior to be PPC specific. That is the cause why a dedicated emulator is always superior to an emulation system like QEMU or MESS when it comes to compatibility and precision (and maintainability).
It is certainly possible to get the information necessary for correct status register implementation out of SoftFloat. Btw, I quickly checked PPC 601 data sheet and it seems to handle special cases, like NaN and infinity, all in hardware. But one would need to extend SoftFloat with some instructions, like fused multiply-add, if not already done.
User avatar
Cat_7
Expert User
Posts: 6121
Joined: Fri Feb 13, 2004 8:59 am
Location: Sittard, The Netherlands

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by Cat_7 »

Here is some discussion about the future of softfloat in Qemu.

http://lists.nongnu.org/archive/html/qe ... 01730.html

Best,
Cat_7
andreas_g
Student Driver
Posts: 20
Joined: Thu Dec 02, 2010 5:50 pm
Location: Austria

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by andreas_g »

Interesting discussion, thank you for the link. If I were them and if the only issue they have with SoftFloat 2 is missing half precision, then I'd recommend to just write an own implementation of 16 bit floats. Using single precision as a template it should not be too difficult. If they need it for ARM emulation they have to customize it anyway (obviously ARM uses no or a different NaN and infinity encoding). Porting everything to SoftFloat 3 is definitely no easy task and much harder then just implementing half precision.
CharlesS
Granny Smith
Posts: 107
Joined: Thu Aug 14, 2008 9:05 am

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by CharlesS »

andreas_g wrote:If QEMU uses SoftFloat then there is a good chance to be able to implement the missing things. The only problem will be that SoftFloat is also used to emulate other platforms which is likely to cause conflicts when one changes SoftFloat's behavior to be PPC specific. That is the cause why a dedicated emulator is always superior to an emulation system like QEMU or MESS when it comes to compatibility and precision (and maintainability).
Don't you think that's a strong statement to make, given that every dedicated emulator we have for every Mac machine other than the original 68000-based models is far inferior to QEMU and MESS in all three of those categories, despite most of them being far more mature?
There's no earthly way of knowing, which direction we are going, for the rowers keep on rowing, and they're certainly not showing any signs that they are slowing.
andreas_g
Student Driver
Posts: 20
Joined: Thu Dec 02, 2010 5:50 pm
Location: Austria

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by andreas_g »

CharlesS wrote:Don't you think that's a strong statement to make, given that every dedicated emulator we have for every Mac machine other than the original 68000-based models is far inferior to QEMU and MESS in all three of those categories, despite most of them being far more mature?
Explaining this is simple. There has been no development on PearPC or SheepShaver for years, while QEMU is improving continuously.
CharlesS
Granny Smith
Posts: 107
Joined: Thu Aug 14, 2008 9:05 am

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by CharlesS »

SheepShaver and Basilisk were worked on for many, many years longer than QEMU's Mac guest has been, and it's not as if it suddenly became less accurate when work stopped on it. Part of why the work did stop, in addition, was because SheepShaver was/is a maintainability nightmare. QEMU has very quickly surpassed it in a very short amount of time, which speaks to its quality IMO.
There's no earthly way of knowing, which direction we are going, for the rowers keep on rowing, and they're certainly not showing any signs that they are slowing.
User avatar
adespoton
Forum All-Star
Posts: 4208
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by adespoton »

The architecture of qemu (and MESS for that matter) also forces developers to use best practices, as changes they make to make one thing work could affect something else as well.

So SheepShaver has a lot of single-product custom performance tweaks (kind of like snes9x) to make it run fast in situations people have commonly encountered, and there are a lot of logic patches to make it behave the way people want. The down side of that is that it's a mess from an architectual and maintenance perspective, and a single change in host hardware or compiler can cause the entire structure to fail in a manner that is difficult to discover.

Qemu on the other hand has a general core that is shared between multiple sets of emulation logic, so no shortcuts can be taken in one set of code to make some unrelated feature work properly/quickly. This means that changes to the code are much more thoroughly understood and vetted, resulting in a higher quality product.

The down-side of course is that there are specific structures that must be adhered to, which prevents quick hack features and performance optimizations, which can make it seem slower or less feature-complete compared to other software, for a limited set of host configurations.
andreas_g
Student Driver
Posts: 20
Joined: Thu Dec 02, 2010 5:50 pm
Location: Austria

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by andreas_g »

That SheepShaver is a mess is not caused by it being a dedicated emulator. One could also make MESS a mess and it still would work and one could also make SheepShaver a best practice programming example. Connecting bad coding style and dedicated emulators is illegitimate.

An example: Let's say we want to implement 68k emulation into QEMU. At some point we need to add correct FPU emulation. We then notice that the extended precision floating-point format of the 68k does not match the one in SoftFloat, which is identical to x87. So we need to change that ... lots of quite complicated changes and pre-processor switches in almost every function necessary. Some functions need to be duplicated and totally re-written. Some functions need to be added because they are missing. Then at some point we notice that for emulating arithmetic exceptions we need to get internal intermediate values from SoftFloat. Therefore we need to customize SoftFloat even further to get those intermediate values as an output. Again more pre-processor switches for not breaking other platforms causing more mess. This requires to be extremely careful to not break other platforms. So finally we are done. But then ... the lead of the project comes to the conclusion that everything needs to be switched from SoftFloat 2 to SoftFloat 3, because it better suits some more common platform. Back to the start.

I don't want to spam this thread with this off-topic discussion any further. But I hope the example explains my opinion.
CharlesS
Granny Smith
Posts: 107
Joined: Thu Aug 14, 2008 9:05 am

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by CharlesS »

andreas_g wrote:That SheepShaver is a mess is not caused by it being a dedicated emulator. One could also make MESS a mess and it still would work and one could also make SheepShaver a best practice programming example. Connecting bad coding style and dedicated emulators is illegitimate.
I guess the main thing I was objecting to was the statement that a dedicated emulator is *always* superior, when there are other factors which are far more important, IMO, than whether an emulator is dedicated or not (especially when, as adespoton points out, shortcuts of the sort used by SheepShaver and Basilisk are far less likely to be practical in a system like MESS).

Anyway, I'm extremely grateful for the amazing work being done on QEMU-PPC right now, which seems to be working toward something far better than any dedicated emulator we have. Hats off to all of the developers that have been working on this. :)
There's no earthly way of knowing, which direction we are going, for the rowers keep on rowing, and they're certainly not showing any signs that they are slowing.
mcayland
Mac Mechanic
Posts: 150
Joined: Sun Nov 01, 2015 10:33 pm

Re: GSOC qemu Boot Mac OS >= 8.5 on PowerPC system

Post by mcayland »

andreas_g wrote:An example: Let's say we want to implement 68k emulation into QEMU
I should probably point out that this isn't a particularly good example since QEMU already supports the 68K target, and Laurent even has a test branch which attempts to emulate a Quadra! At least recently there were patches applied that fixed up the emulation enough to able to boot a Linux 68K kernel.

However I should just point out that you're making the mistake here of confusing simulation and emulation. QEMU is an emulator and therefore is not concerned with 100% accuracy, the trade-offs being that timings will be different to real hardware and floating point maths will have rounding errors and some FP status may not match for certain edge cases. But for 99.9% of use cases that's good enough for everyone to run their old software.

FWIW QEMU has struggled in the past because it's SF2 library was forked fairly early on which led to a number of differences. If the decision ends up being that the project moves to SF3 I suspect this time around there will be a greater effort to get any QEMU-specific patches upstream so there is no need to maintain an internal fork.

In terms of emulation you are right in that QEMU will likely never reach the speed of SS, however SS works by live-patching the OS which is not really the focus of QEMU which aims to model real hardware. I should also add as a QEMU maintainer one of the benefits of working on QEMU is that there are a large number of developers who work on independent subsystems such as the JIT, disk IO etc. and so all I need to do is focus on the PPC mac hardware emulation routines.

This contrasts sharply with SS which as a bespoke emulator requires any developer to have complete in-depth knowledge of Mac and PPC hardware. The reason I think that QEMU has a brighter future compared to a lot of the other emulators is that the number of developers with the time, knowledge and skill to work on SS to emulate an entire PPC-based Mac is very few, while at the moment with QEMU I can focus the time I have available on just the PPC Mac-specific hardware emulation. And as as major benefit, as other people improve other QEMU subsystems over time then QEMU's PPC Mac emulation gains from these enhancements automatically :)
Post Reply