Page 1 of 1

How to merge data- and resource-fork back into one file?

Posted: Thu Oct 27, 2016 9:57 am
by mabam
I had a Synology DS-106j NAS drive running in my network until the HD broke. I didn't have a back-up so someone recovered the data from the broken HD onto a new external HD.

My question is: How can I copy the recovered data onto my Mac and have the data- and resource-fork merged back into one file? I could set up a Linux VM using Parallels. But I read Synology uses a customised Linux on their NAS drives. So what type/version of Linux should I use for the VM? Or does it depend on the version of netatalk (I suppose that's what they used)?

Apart from OS X, I have never worked with any unixoid system. Anyone who can help?

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Oct 27, 2016 8:33 pm
by mabam
In the mean time I have been told that I can use Ubuntu 12 to read the external HD. It is ext3 formatted, which is supported by the ext4 driver used by Ubuntu.

I have installed it as virtual machine and can access the recovery HD. But I figured that Synology has its very own way of storing the resource-fork. So I have to copy everything onto a new HD and plug it into the DS-106j in order to have my Mac data properly restored.

I was planning to plug an external HD into the NAS for back-up purposes. But as it will also get Mac files stored the Synology way onto it, I will have to find a way to do a network backup onto an external HD on my Mac. Does anyone have a recommendation of an appropriate back-up software?

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Oct 27, 2016 9:36 pm
by Cat_7

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Nov 02, 2016 12:08 am
by mabam
Thanks for the tip! I will look into that tool once I have a NAS set up again. But that is way more work than I thought: My Synology NAS is broken and was probably causing the original HD to break as well. A different HD doesn't help. I need a new NAS.

Maybe the bridged OpenVPN server (http://www.emaculation.com/doku.php/bri ... rver_setup) from the wiki could run on a NAS. But I will ask for that on the related thread (http://emaculation.com/forum/viewtopic. ... 735#p54735).

So back to the post subject: I have an external HD with the files rescued from the original HD from my NAS. There are a few files on there which are useless without their respective resource forks, which were stored on the HD as a separate file by the NAS. They are stored in folders called "@eaDir", which seem to be specific to Synology. How can I merge the resource forks to their respective data forks, now the NAS is broken and won't do that for me by presenting the files merged when accessed via AppleTalk?

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Nov 02, 2016 10:02 am
by 24bit
PC-MACLAN did store Mac files on Windows HDDs flawlessly.
Maybe look into the way files were stored there and sort things out accordingly.
After rearranging the files may be seen all right on your Backup from the Mac side.
If my memory is correct, PC-MACLAN did work with WIN7-32.
I was using it from WIN95 to W2K though.

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Nov 02, 2016 11:16 am
by mabam
I even used to have parts of PCMacLAN running on WinXP when I did my daily things on an XP machine. That was for printing purposes before I found out that it works much better to copy over some system files from Win2K into XP for that purpose.

I was hoping to have a different system reading the recovery HD and properly share the files via AFP. But due to the Synology-specific method of storing the AppleDouble files (the resource forks are called "SYNO_Resource" and placed in a folder corresponding with the name of the data fork) only a Synology system would be able to do that.
Thanks for the tip, but PCMacLAN won't do the job automatically either, I'm afraid. So I will have to spend some time doing this manually for a number of files.

I found this thread and will try the suggestions given there when I have some time.

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Nov 03, 2016 1:05 am
by mabam
mabam wrote:Maybe the bridged OpenVPN server [...] from the wiki could run on a NAS.
I just realised I have misunderstood things. I don't need the OpenVPN server.

I'm trying OMV and play around on a virtual machine with it, prior to buying any new hardware (there's an OMV version for Banana PI). Not sure whether I can manage to make it act as print server as well. Neither can DDP be activated through the web interface.

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Dec 14, 2016 1:44 am
by mabam
It looks like I'm almost done with sorting everything out I need for my AppleTalk file and print server. I will share more info here once it runs.

Just one question: Does anyone happen to know a USB Ethernet adapter capable of transporting AppleTalk via DDP? I bought one providing Gigabit Ethernet with an ASIX ax88179 chipset but it only works for AppleTalk over IP.

The MosChip MCS7830 should work as I got told, but it only provides Fast Ethernet. And none of the versions I found available online are extended by a cable with the USB plug on its end, but have the plug built into the housing instead – which I don't prefer.

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Dec 14, 2016 9:44 pm
by adespoton
What host machine are you using? USB to Ethernet adapters generally don't do full frame conversion but just do packetized serial data, which means they won't do DDP. Thunderbolt or Firewire to Ethernet adapters tend to do full frame conversion. There are some specialized USB to Ethernet adapters that will do DDP, but they don't conform to the pluggable Ethernet standard, and so need specific drivers -- which are generally only available under Windows, not on Linux or OS X.

As far as extended cables go, just by a USB extender cable, and plug the adapter into that, if you go this route.

You may also want to investigate this thread: https://mac68k.info/forums/message.jspa?messageID=1190 as it deals with LocalTalk bridging which runs into the same issues.

[edit] from that thread: "There's a bug in net/appletalk/ddp.c introduced somewhere in the kernel 2.6 era I presume. It seems that outgoing broadcast AppleTalk packets are being looped back onto the same machine...not totally sure why, but it does it on purpose. A change somewhere between Linux 2.4 and 3.8 changed the way the loopback worked, and now it's trying to prepend too many header bytes onto the loopback packet. I fixed it by hacking in a "+ 16" to a few places where the packet is created, but I'd like to eventually figure out the "correct" way to fix it and submit a patch. So yeah, that part was Linux's fault."

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Dec 15, 2016 12:53 am
by mabam
I use a Banana Pi. Its on-board Ethernet port doesn't work for DDP. In this thread in the Armbian forum someone made it work with a USB Ethernet adapter. In a PM he told me his adapter is using the MosChip MCS7830. I've now ordered one of those adapters and will use an extender cable.

The thread at mac68k is above me, I'm afraid.

There's no need for me to care about netatalk. I'll just install a2server and everything is settled. Ivan has done a great job fixing bugs and making everything work. Installation of the whole thing via his bash script is easier than with many GUI installers of other software.
The only thing is that the information on his website is outdated. Current version is 1.5 and running on Debian 8 Jessie.

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Dec 15, 2016 8:08 pm
by adespoton
How much did all the hardware cost for this setup? Seems like a nice little package for creating a network hub that truly supports the last 30 years of networking.

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Dec 15, 2016 10:12 pm
by mabam
If you just want a hub, use a Raspberry Pi. There's an image on Ivan's website you can just burn on an SD card and your raspberry is ready to go. And the RPi's on-board Ethernet connector seems to like DDP.

I've chosen for a Banana Pi mainly because I wanted to integrate an HDD. I use a 2.5 inch HDD that is powered by the Banana Pi. But also performance seems to be a little better than with the Raspberry Pi. And based on what I had found on the net it seemed that it should be possible to make it work with the Banana Pi. My biggest concern was how to manage compiling a kernel with AppleTalk. But using Armbian this is quite easy. With a bit more effort Bananian is doable as well. Spoken from my perspective.

Here is a guide I used regarding which hardware to buy. Next to the hardware listed there I needed a USB Ethernet adapter. But I didn't need to buy housings for the Banana Pi or the HDD as I use the one from my broken Synology NAS.

35.61 € – Banana Pi
8.99 € – Power Supply
67.63 € – HDD
8.99 € – SATA cable
12.71 € – SD Card
14.55 € – USB Fast Ethernet adapter
5.10 € for some screws
--
144.59 €

On top of that I spent 7.99 € as I didn't have an SD card reader yet, and another 12.99 € for the ASIX USB Gigabit Ethernet adapter I originally bought, which turned out not to work with DDP. But I might sell that one on ebay.
Furthermore I'll use an external hard disk for backup. But I still have one to use for that. I'll try to build the USB extension cable myself from some old cables. It'll be in the NAS housing anyway.

I'm planning to write a guide for building and installing the system. I have made many notes which are quite detailed. It's gonna be a beginner's guide. Not only because it is (supposedly) suited for beginners, but especially because it is written by a beginner :-)
I've never really done anything on terminal before, except for copy-pasting and throwing together a few commands I found on emaculation.com or elsewhere. And except for OS X I've never touched a unixoid system before. So during the last two and a half months or so I've spent many hours figuring out how to do it.
adespoton wrote:Seems like a nice little package for creating a network hub that truly supports the last 30 years of networking.
I consider buying another Bpi in case the first one breaks one day. That should cover me for the next 30 years of networking :-)

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Dec 15, 2016 10:36 pm
by adespoton
My own solution was an Airport Extreme, a USB hub, and setting up SMB networking via Thursby's DAVE on Mac OS 9. DAVE is really flaky though, under both SheepShaver and Qemu, so I may switch to FTP at some point; but my Airport Extreme doesn't support FTP, so I may introduce a Mac Mini G4 I've got that runs 10.5 Server. It has the benefit of being able to run OpenVPN as well, for remote access to my classic VMs.

Re: How to merge data- and resource-fork back into one file?

Posted: Thu Dec 15, 2016 10:46 pm
by mabam
Dave wasn't very reliable on my 7500/100 under Mac OS 8.1 either.

Since my NAS broke anyway, I was looking for a solution that could also share my LW 4/600 PS to the network. I didn't want something I had to start up every time I want to print. And a NAS (which I only use as simple fileserver anyway) is not switched off. A Banana Pi is not very power consuming and can run a2server with PAP activated.

Re: How to merge data- and resource-fork back into one file?

Posted: Mon Dec 19, 2016 2:25 pm
by mabam
mabam wrote:I bought [a USB Ethernet adapter] providing Gigabit Ethernet with an ASIX ax88179 chipset but it only works for AppleTalk over IP.

The MosChip MCS7830 should work as I got told, but it only provides Fast Ethernet.
The MCS7830 didn't want to work either, so yesterday I gave up for the moment. When starting the Banana Pi again today, some magic happened and everything worked. By entering 'nbplkup' I could see my printer and the AsantéTalk it is connected to. Also my old iMac, booted into Mac OS 9 and IP disabled, could see the Banana Pi as AFP server over DDP and connect to the shared volumes.

Then I exchanged the USB adapter and tried with the ax88179 again. Stopping netatalk and starting it again did not make DDP work. However, restarting did (these options are provided as simple commands by a2server).

As I have also connected an HDD to the BPi, I suspect the 2 amps power adapter I bought is insufficient and resulting in unexpected behavior of the system. I'll buy a 3 amps adapter and see if that solves my problems for good.

EDIT:
I had also tested both USB adapters via a Win XP VM with PC MacLAN running. I could access the Internet via Internet Explorer. But AppleTalk did not work (it did when I bridged the network to the internal Ethernet port of my Hackintosh). So after I had no luck using the adapters on Linux, that seemed to confirm they were not able to transport DDP. But maybe it's just the Win XP driver that doesn't support it. The adapters themselves do, as I know now.

About a week ago I had inquired at ASIX which ones of their chipsets work for DDP. This was the answer:
Thanks for your inquiry, as we know, the ASIX products are only handle PHY
and MAC layers, the AppleTalk Protocol is a transport layer protocol, So you
need make sure if your system can support AppleTalk or not. for your
questions, if you need USB 3.0 interface, you can use the AX88179 in your
application, if USB 2.0 interface, you can use the AX88772C in your
application. please provide us more details information about your target
application, forecast quantity, mass production schedule for more support,
thanks a lot.

Now I know it works I can inform ASIX about the required quantity and mass production schedule ;-)

I have also found a thread where 'mactjaap' makes a C.H.I.P. ready for use with DDP. There a Davicom DM96xx USB 10/100 Ethernet clone was used, connected to a powered USB hub.

So for anyone trying to build something like this: There are at least two more chipsets that work (providing the Linux driver for the AX88772C supports it).

Re: How to merge data- and resource-fork back into one file?

Posted: Mon Dec 19, 2016 6:20 pm
by adespoton
That explains a lot :). Have you also tried with a powered USB hub to connect the adapter and external storage?

Re: How to merge data- and resource-fork back into one file?

Posted: Mon Dec 19, 2016 8:37 pm
by mabam
No, I don't have one. I'd rather just use one power supply for everything, if possible.
Furthermore, the HDD is connected via SATA. That's one of the advantages of the Banana Pi, and the reason why I bought a 2.5" HDD. A 'regular' one wouldn't work with the 5V supplied by the Banana Pi.

I hope a 3 amp supply will fix it.

Re: How to merge data- and resource-fork back into one file?

Posted: Tue Dec 20, 2016 10:15 pm
by mabam
I have now built all the parts into the Synology housing. They fit pretty well.

Image Image

The power button of the housing is exactly pressing on the BPi one's. Right next to that I have sawed a slot for the SD card into the housing.
Everything is mounted onto the old circuit board of the Synology.

I have yet to install the PAP backend and share my LaserWriter 4/600 PS to the network, and to copy the rescued files from the old Synology HDD onto the new system.

Next to that I have to decide how I want to back up the system. Probably via the network on an external HDD on my Hackintosh.

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Dec 21, 2016 5:45 pm
by adespoton
It looks like you've got a spare USB port there; why not route it to the outside of the case so you can plug in an external HD?

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Dec 21, 2016 7:00 pm
by mabam
I thought about that as well. But then the external backup HDD is running the whole time, simply because the Banana Pi is placed behind a door I won't open frequently. I don't have loads of data, so backing up over the network isn't an issue. And I can back up onto an HFS+ formatted drive and don't have separate resource forks. But then again I have to manually start the backup each time.

Have to think about how I do it.

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Dec 21, 2016 9:52 pm
by adespoton
If you want to go the network route, you could just set up a scheduled rsync session; that's what I do.

Re: How to merge data- and resource-fork back into one file?

Posted: Tue Dec 27, 2016 12:56 am
by mabam
Thanks, that was a very good tip!

Using an AppleScript, I can now simply switch on an external HDD on the Mac, all relevant network volumes get mounted and the backup starts. Once it has finished, the HDD gets unmounted again.

All the data from my old NAS is now sorted and copied onto the new one. I plan to run the first backup tomorrow.

The only thing missing is to tell the Banana Pi to spin down the HDD attached to it if it's not used for a while.

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Apr 19, 2017 8:11 pm
by mabam
For anyone ending up in this thread for some reason: The guide mentioned in an earlier post in this thread is ready now:
http://www.emaculation.com/doku.php/app ... os_and_osx

If things are unclear or if there should still be an error in it – or if you know a way to do things in an easier way – please let me know. Any improvements are welcome!

At this point many thanks to Ronald P. Regensburg for his support, and to ClockWise for hosting the guide at emaculation.com!

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Apr 19, 2017 8:19 pm
by adespoton
One word of warning: OS X rsync copies resource forks, but Debian rsync doesn't. So be careful when restoring to an HFS+ volume.

[edit] Also, to answer the original thread question: on OS X, you can merge data and resource forks back into one file using named forks. In the terminal, you want to cp [resource fork of file] [filename]/..namedfork/rsrc -- that's all there is to it.

Unfortunately, this mechanism isn't quite the same as Microsoft's Alternate Data Streams in NTFS, even though it functions in a similar manner.

Would have been so easy for MS to transparently support files with resource forks using ADS, but they chose not to.

Re: How to merge data- and resource-fork back into one file?

Posted: Wed Apr 19, 2017 10:16 pm
by mabam
I only run rsync on an OS X machine for transfers between volumes from the Banana Pi connected to the OS X machine via AFP, to an HFS+ HDD connected to that same machine via USB.

It took me a while to understand how your entry for merging data and resource fork back into one file works. Good to know there's a quite easy way to achieve that!

I received my data on a recovery HDD from the rescue service last year. IIRC, I later on copied it onto the Banana Pi's HDD via Samba and then accessed it from OS X per AFP, from within networked SheepShaver per AFP, and – to clean up some stuff – from an XP VM via Samba. I then copied the files onto an HFS volume piece by piece.
Thus the warning in the guide :-)