Sheepshaver development and supported platforms

About SheepShaver, a PPC Mac emulator for Windows, MacOS X, and Linux that can run System 7.5.3 to MacOS 9.0.4.

Moderators: Cat_7, Ronald P. Regensburg, ClockWise

Post Reply
Jagmn
Student Driver
Posts: 10
Joined: Fri Feb 19, 2021 7:37 am

Sheepshaver development and supported platforms

Post by Jagmn »

Hi Folks,

I've been getting an Apple Silicon build of SS up to scratch. Lots of stuff to learn and I've been having to use tools such as thread sanitizer to help me bottom out some of the race conditions and other multi-threading issues present for weakly-ordered memory platforms. This has led to me using some C++11 features to quickly solve some of the synchronisation issues I've found. So my question is:

What platforms (or OS/CPU combos) does SheepShaver need to run on for its present user base?
My current understanding:
  • Recent MacOS X platforms: 86_64 and arm64 based
  • Recent and embedded Linux platforms: 86_64, arm & arm64 based
  • Recent Windows platforms: 86_64 based - though I have no idea which versions of Windows is desirable
Platforms that (I believe) used to be supported but now are not:
  • MacOS X on PPC
  • Beos
Anything else?

And as for programming features; I assume that, if the above is true, C++11 feature usage would be acceptable?

Thanks!
User avatar
adespoton
Forum All-Star
Posts: 3312
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: Sheepshaver development and supported platforms

Post by adespoton »

While doing this, I'd recommend syncing up with Kanjitalk755 and ensuring your changes work for BII as well as SS.

As for platforms, cebix SDL1.2 BII has been integrated into LibRetro now. It might be a useful thing to check against the libretro framework to ensure your changes are compatible for compiling BII and SS cores for there.
Jagmn
Student Driver
Posts: 10
Joined: Fri Feb 19, 2021 7:37 am

Re: Sheepshaver development and supported platforms

Post by Jagmn »

Thanks for the heads up - I'll fire up a BII build to see how that goes (not touched it as yet). And LibRetro looks interesting - first time I've come across it.

What is the best way to collaborate with Kanjitalk755? I'm happy to open some gradual pull requests once I'm satisfied things are reasonably clean. :smile:
User avatar
adespoton
Forum All-Star
Posts: 3312
Joined: Fri Nov 27, 2009 5:11 am
Location: Emaculation.com
Contact:

Re: Sheepshaver development and supported platforms

Post by adespoton »

Apart from pull requests off his repo, you can also message him through this site just to let him know what you're up to, in case he doesn't see this thread.
kanjitalk755
Mac Mechanic
Posts: 179
Joined: Thu Nov 09, 2017 12:06 pm

Re: Sheepshaver development and supported platforms

Post by kanjitalk755 »

Jagmn wrote: Fri Mar 26, 2021 7:55 amWhat platforms (or OS/CPU combos) does SheepShaver need to run on for its present user base?
Linux: x86 is also possible
Windows: x86 instead of x86_64, and Windows Vista and up
Jagmn wrote: Fri Mar 26, 2021 7:55 amAnd as for programming features; I assume that, if the above is true, C++11 feature usage would be acceptable?
C++11 has been supported since a long time ago gcc(4.8?), so I think it is acceptable.

I have two questions.
  • I think you replaced the JIT compiler with a new one for Apple Silicon, does the x86_64 JIT compiler work as well?
  • Are you planning to make the version under development a public repository?
Jagmn
Student Driver
Posts: 10
Joined: Fri Feb 19, 2021 7:37 am

Re: Sheepshaver development and supported platforms

Post by Jagmn »

Hi kanjitalk755, thanks for the heads up. What do you use for testing? VMs?

Q1) The JIT is the same as that which was originally extracted (from what I can tell) from Qemu. I've managed to get a GCC-10 brew build (and a fix-up script) to produce viable .o files and get dyngen.c updated enough (in a hacky way) to regenerate the JIT ops for both MacOS x86_64 and Arm64.
- The Apple Silicon / Arm64 JIT needed to have some changes to make the JIT cache swap between read-only-execute and read-write-no-execute as Apple enforce that you can't combine write-and-execute now.
- Some things I have changed for the XCode builds: My builds use NATMEM_OFFSET instead of the zero-page approach to move the Classic-Mac memory > 4G range - this has been working fine for me with regenerated JIT ops on both x86_64 and Arm64, and using shmat for the aliased kernel data area.

Q2) A public repo: Yes! Definitely looking to get this into github to share it. I have rabbit-holed a bit too much so I'm cleaning it up to compress the relevant patches before airing it :wink:
kanjitalk755
Mac Mechanic
Posts: 179
Joined: Thu Nov 09, 2017 12:06 pm

Re: Sheepshaver development and supported platforms

Post by kanjitalk755 »

I'm using VirtualBox for testing Linux and Windows version.

When I tackled the problem that 64-bit SheepShaver didn't work on macOS 10.10 and up,
I decided not to use shmat because it fails (unable to get shared memory) on macOS 10.13 at that time.
I'm interesting how you made it available and looking forward to the code being public.
Jagmn
Student Driver
Posts: 10
Joined: Fri Feb 19, 2021 7:37 am

Re: Sheepshaver development and supported platforms

Post by Jagmn »

Thanks for the heads up with shmat on 10.13 - I'll see if I can check that with my previous x86 macbook at some point. When I was prodding it I did find that almost anything within the first 4G didn't allocate but > 4G seemed to work. I should dive into it in more detail at some point to see if I could identify the root cause.

The following is the very basic patch for building Arm64 non-JIT on Apple Silicon (sorry for the inline-paste: I don't see an obvious attachment option). It doesn't contain fixes to do the NATMEM_OFFSET build for x86_64 or the (current set of) race conditions I've found but it shows you where I was heading before the rabbit holing began.

The SLIRP fix is from address-sanitizer and the gfxaccel.cpp fix is needed on Arm64 (otherwise random buffer overruns kill the emulator).

I'll do my best to get the remainder (all of the Dyngen/JIT + race condition fixes) cleaned up and in a public repo ASAP but it will take a while :smile:

Code: Select all

diff --git a/BasiliskII/src/CrossPlatform/vm_alloc.cpp b/BasiliskII/src/CrossPlatform/vm_alloc.cpp
index 3aa369dabd..e85a6766ee 100755
--- a/BasiliskII/src/CrossPlatform/vm_alloc.cpp
+++ b/BasiliskII/src/CrossPlatform/vm_alloc.cpp
@@ -229,6 +229,13 @@ void *vm_acquire_reserved(size_t size) {
 	return reserved_buf && size <= RESERVED_SIZE ? reserved_buf : VM_MAP_FAILED;
 }
 
+int vm_init_reserved(void *hostAddress) {
+	int result = vm_acquire_fixed(hostAddress, RESERVED_SIZE);
+	if (result >= 0)
+		reserved_buf = hostAddress;
+	return result;
+}
+
 /* Allocate zero-filled memory of SIZE bytes. The mapping is private
    and default protection bits are read / write. The return value
    is the actual mapping address chosen or VM_MAP_FAILED for errors.  */
diff --git a/BasiliskII/src/slirp/slirp.c b/BasiliskII/src/slirp/slirp.c
index 4aad7ba45a..45c97612c7 100755
--- a/BasiliskII/src/slirp/slirp.c
+++ b/BasiliskII/src/slirp/slirp.c
@@ -87,7 +87,7 @@ static int get_dns_addr(struct in_addr *pdns_addr)
 static int get_dns_addr(struct in_addr *pdns_addr)
 {
     char buff[512];
-    char buff2[256];
+    char buff2[257];
     FILE *f;
     int found = 0;
     struct in_addr tmp_addr;
diff --git a/SheepShaver/src/MacOSX/Info.plist.in b/SheepShaver/src/MacOSX/Info.plist.in
index dfe702403c..a70e1ec95a 100644
--- a/SheepShaver/src/MacOSX/Info.plist.in
+++ b/SheepShaver/src/MacOSX/Info.plist.in
@@ -39,10 +39,6 @@
 	<string>@PACKAGE_VERSION@</string>
 	<key>CSResourcesFileMapped</key>
 	<true/>
-	<key>LSArchitecturePriority</key>
-	<array>
-		<string>x86_64</string>
-	</array>
 	<key>LSMinimumSystemVersion</key>
 	<string>10.7.0</string>
 	<key>NSHighResolutionCapable</key>
diff --git a/SheepShaver/src/MacOSX/SheepShaver_Xcode8.xcodeproj/project.pbxproj b/SheepShaver/src/MacOSX/SheepShaver_Xcode8.xcodeproj/project.pbxproj
index 8002534c90..46958ce618 100755
--- a/SheepShaver/src/MacOSX/SheepShaver_Xcode8.xcodeproj/project.pbxproj
+++ b/SheepShaver/src/MacOSX/SheepShaver_Xcode8.xcodeproj/project.pbxproj
@@ -1210,7 +1210,7 @@
 				MACOSX_DEPLOYMENT_TARGET = 10.7;
 				OTHER_CPLUSPLUSFLAGS = "$(OTHER_CFLAGS)";
 				PRODUCT_NAME = kpx_cpu;
-				VALID_ARCHS = x86_64;
+				VALID_ARCHS = "x86_64 arm64";
 			};
 			name = Debug;
 		};
@@ -1249,7 +1249,7 @@
 				MACOSX_DEPLOYMENT_TARGET = 10.7;
 				OTHER_CPLUSPLUSFLAGS = "$(OTHER_CFLAGS)";
 				PRODUCT_NAME = kpx_cpu;
-				VALID_ARCHS = x86_64;
+				VALID_ARCHS = "x86_64 arm64";
 			};
 			name = Release;
 		};
@@ -1329,14 +1329,12 @@
 				OTHER_CFLAGS = "";
 				OTHER_CPLUSPLUSFLAGS = "$(OTHER_CFLAGS)";
 				OTHER_LDFLAGS = (
-					"-pagezero_size",
-					0x3000,
 					"-lkpx_cpu",
 				);
 				PRECOMPS_INCLUDE_HEADERS_FROM_BUILT_PRODUCTS_DIR = NO;
 				PRODUCT_BUNDLE_IDENTIFIER = net.cebix.sheepshaver;
 				PRODUCT_NAME = SheepShaver;
-				VALID_ARCHS = x86_64;
+				VALID_ARCHS = "x86_64 arm64";
 				WARNING_LDFLAGS = "";
 			};
 			name = Debug;
@@ -1395,14 +1393,12 @@
 				OTHER_CFLAGS = "";
 				OTHER_CPLUSPLUSFLAGS = "$(OTHER_CFLAGS)";
 				OTHER_LDFLAGS = (
-					"-pagezero_size",
-					0x3000,
 					"-lkpx_cpu",
 				);
 				PRECOMPS_INCLUDE_HEADERS_FROM_BUILT_PRODUCTS_DIR = NO;
 				PRODUCT_BUNDLE_IDENTIFIER = net.cebix.sheepshaver;
 				PRODUCT_NAME = SheepShaver;
-				VALID_ARCHS = x86_64;
+				VALID_ARCHS = "x86_64 arm64";
 			};
 			name = Release;
 		};
diff --git a/SheepShaver/src/MacOSX/config/config-macosx-aarch64.h b/SheepShaver/src/MacOSX/config/config-macosx-aarch64.h
index aa04b6a952..c904f67182 100644
--- a/SheepShaver/src/MacOSX/config/config-macosx-aarch64.h
+++ b/SheepShaver/src/MacOSX/config/config-macosx-aarch64.h
@@ -405,8 +405,8 @@
    ordering is the same as for multi-word integers. */
 /* #undef HOST_FLOAT_WORDS_BIG_ENDIAN */
 
-/* Define constant offset for Mac address translation */
-/* #undef NATMEM_OFFSET */
+/* Define constant offset for Mac address translation: macosx-aarch64 is always 64bit */
+#define NATMEM_OFFSET 0x400000000000
 
 /* Define to the address where bug reports for this package should be sent. */
 #define PACKAGE_BUGREPORT "Christian.Bauer@uni-mainz.de"
@@ -428,7 +428,7 @@
 
 /* Define if the __PAGEZERO Mach-O Low Memory Globals hack works on this
    system. */
-#define PAGEZERO_HACK 1
+/* #define PAGEZERO_HACK 1 */
 
 /* Define as the return type of signal handlers (`int' or `void'). */
 #define RETSIGTYPE void
diff --git a/SheepShaver/src/Unix/configure.ac b/SheepShaver/src/Unix/configure.ac
index 35f3495e63..3b3480ca46 100755
--- a/SheepShaver/src/Unix/configure.ac
+++ b/SheepShaver/src/Unix/configure.ac
@@ -1640,13 +1640,13 @@ if [[ "x$EMULATED_PPC" = "xyes" ]]; then
         esac
       fi
       if [[ "x$have_dyngen_gcc3" = "xyes" ]]; then
-        DYNGEN_OP_FLAGS="$DYNGEN_OP_FLAGS -fno-align-functions"
+        DYNGEN_OP_FLAGS="$DYNGEN_OP_FLAGS -fno-align-functions -fno-stack-protector"
       else
         DYNGEN_OP_FLAGS="$DYNGEN_OP_FLAGS -malign-functions=0"
       fi
       DYNGEN_OP_FLAGS="$DYNGEN_OP_FLAGS -finline-functions -finline-limit=10000 -fno-exceptions -g0"
       if [[ "x$have_dyngen_gcc3" = "xyes" ]]; then
-        DYNGEN_OP_FLAGS="$DYNGEN_OP_FLAGS -fno-reorder-blocks -fno-optimize-sibling-calls"
+        DYNGEN_OP_FLAGS="$DYNGEN_OP_FLAGS -fno-reorder-blocks -fno-optimize-sibling-calls -fno-reorder-blocks-and-partition"
       fi
       if [[ "x$DYNGEN_CC" != "x$CXX" ]]; then
         DYNGEN_CFLAGS="-O2 $CFLAGS"
diff --git a/SheepShaver/src/Unix/main_unix.cpp b/SheepShaver/src/Unix/main_unix.cpp
index 0c5b76a1e5..5a470e6cb7 100755
--- a/SheepShaver/src/Unix/main_unix.cpp
+++ b/SheepShaver/src/Unix/main_unix.cpp
@@ -2210,6 +2210,7 @@ rti:;
 }
 #endif
 
+extern int vm_init_reserved(void *hostAddress);
 
 /*
  *  Helpers to share 32-bit addressable data with MacOS
@@ -2221,7 +2222,15 @@ bool SheepMem::Init(void)
 	page_size = getpagesize();
 
 	// Allocate SheepShaver globals
+#ifdef NATMEM_OFFSET
+	if (vm_mac_acquire_fixed(ROM_BASE + ROM_AREA_SIZE + SIG_STACK_SIZE, size) < 0)
+		return false;
+	uint8 *adr = Mac2HostAddr(ROM_BASE + ROM_AREA_SIZE + SIG_STACK_SIZE);
+	if (vm_init_reserved(adr + size) < 0)
+		return false;
+#else
 	uint8 *adr = vm_mac_acquire(size);
+#endif
 	if (adr == VM_MAP_FAILED)
 		return false;
 	proc = base = Host2MacAddr(adr);
diff --git a/SheepShaver/src/gfxaccel.cpp b/SheepShaver/src/gfxaccel.cpp
index ed65e5d835..ec1e96a32c 100644
--- a/SheepShaver/src/gfxaccel.cpp
+++ b/SheepShaver/src/gfxaccel.cpp
@@ -85,7 +85,7 @@ static inline void do_invrect(uint8 *dest, uint32 length)
 	}
 
 	// Align on 32-bit boundaries
-	if (bpp < 32 && (((uintptr)dest) & 2)) {
+	if (bpp < 32 && (((uintptr)dest) & 2) && length >= 2) {
 		INVERT_2(dest, 0);
 		dest += 2; length -= 2;
 	}
@@ -198,7 +198,7 @@ static inline void do_fillrect(uint8 *dest, uint32 color, uint32 length)
 	}
 
 	// Align on 32-bit boundaries
-	if (bpp < 32 && (((uintptr)dest) & 2)) {
+	if (bpp < 32 && (((uintptr)dest) & 2) && length >= 2) {
 		FILL_2(dest, 0, color);
 		dest += 2; length -= 2;
 	}
diff --git a/SheepShaver/src/kpx_cpu/src/cpu/jit/dyngen.c b/SheepShaver/src/kpx_cpu/src/cpu/jit/dyngen.c
index b6ead79221..68be566eb1 100644
--- a/SheepShaver/src/kpx_cpu/src/cpu/jit/dyngen.c
+++ b/SheepShaver/src/kpx_cpu/src/cpu/jit/dyngen.c
@@ -2396,6 +2396,7 @@ void patch_relocations(FILE *outfile, const char *name, host_ulong size, host_ul
 				fprintf(outfile, "    *(uint32_t *)(code_ptr() + %d) = (int32_t)%s + %d;\n", slide, final_sym_name, addend);
 				break;
 			case R_X86_64_PC32:
+      case R_X86_64_PLT32:
 				fprintf(outfile, "    *(uint32_t *)(code_ptr() + %d) = %s - (long)(code_ptr() + %d) + %d;\n", 
 					slide, final_sym_name, slide, addend);
 				break;
diff --git a/SheepShaver/src/kpx_cpu/src/cpu/vm.hpp b/SheepShaver/src/kpx_cpu/src/cpu/vm.hpp
index 5c0b733076..c01dc6ac83 100755
--- a/SheepShaver/src/kpx_cpu/src/cpu/vm.hpp
+++ b/SheepShaver/src/kpx_cpu/src/cpu/vm.hpp
@@ -205,13 +205,13 @@ const uintptr VMBaseDiff = NATMEM_OFFSET;
 #if REAL_ADDRESSING || DIRECT_ADDRESSING
 static inline uint8 * vm_do_get_real_address(vm_addr_t addr)
 {
-	uintptr a = vm_wrap_address(VMBaseDiff + addr);
+	uintptr a = vm_wrap_address(addr);
 #if defined(__APPLE__) && defined(__x86_64__)
 	extern uint8 gZeroPage[0x3000], gKernelData[0x2000];
 	if (a < 0x3000) return &gZeroPage[a];
 	else if ((a & ~0x1fff) == 0x68ffe000 || (a & ~0x1fff) == 0x5fffe000) return &gKernelData[a & 0x1fff];
 #endif
-	return (uint8 *)a;
+	return (uint8 *)(VMBaseDiff + a);
 }
 static inline vm_addr_t vm_do_get_virtual_address(uint8 *addr)
 {
Post Reply