SHARING AMERICA'S TECH NEWS FROM THE VALLEY TO THE ALLEY
by David Girard, courtesy ArsTechnica
With Apple’s licensing of Microsoft’s exFAT file system, it seems like we finally have a good option for OS X and Windows disk swapping. Dave Girard spent some time investigating the appeal, the limitations, and the alternatives to exFAT.
One of the more painful areas of cross-platform computing is data sharing. While networking between Mac OS X, Windows, and Linux has gotten a lot easier thanks to SAMBA, disk sharing still feels like it’s in its infancy thanks to proprietary file systems and the unique legacy needs of the respective operating systems they run on.
There are options for cross-platform file sharing—plenty actually—it’s just that each one presents its own limitations and appeals. With Apple’s licensing of Microsoft’s exFAT file system, it seemed like the main problem with FAT32—the 4GB file size limit—was put to rest, and many people are probably now using it to swap video libraries between their MacBooks and HTPCs or share downloads between OS X and Boot Camped Windows. But exFAT has its own issues and limitations that few people are probably aware of—and considering how few people even know about exFAT, we thought this was a good opportunity to cover it, along with the various alternatives.
First, a brief history of exFAT for those unfamiliar with it. exFAT is a proprietary Microsoft file system that was designed to bridge the gap between the NTFS file system and the more dated FAT32 file system. Its main advantages are that it can store files over 4GB since it is a 64-bit file system. Its max file size limitation is 16 EiB (Exbibyte) and its theoretical max capacity is 64 ZiB (Zebibyte), which our people in the lab have called “stupidly huge.” You won’t have any issues with hitting file size or capacity ceilings with exFAT. Since exFAT is a closed format, Apple had to license it to integrate it into OS X 10.6.5 and later. You can format a volume as exFAT within Disk Utility.
I’ve read about people having issues with exFAT disks that were formatted using Snow Leopard’s (10.6.x) Disk Utility showing up in Windows. Apparently, Apple’s block size was correct according to the standard, but different enough from Windows’ default to cause it not to be recognized on the Windows side. So Snow Leopard users were forced to use Windows to format the drive as exFAT, and then it would show up fine in OS X. This seems to be fixed in Mountain Lion (10.8) since I didn’t have any issues getting the Mountain Lion-formatted exFAT partitions to show up in Windows 7.
On the Windows side, native exFAT support is built into Windows as of Vista SP1, and exFAT drivers are available for older builds like XP. Since it’s a relatively new and proprietary format, Linux exFAT support is definitely lackluster, but there is an implementation of exFAT as a FUSE user-space level file system. I haven’t personally tried it.
Before you run out and format everything as exFAT, you should understand its limitations—and they aren’t insignificant. exFAT has no file system-level encryption or compression support, and, like FAT32 before it, there is no journaling built into the exFAT file system. This means it has a much higher probability of data loss than with NTFS or HFS+. Since FAT32 and exFAT are common USB stick file systems, TFAT and TexFAT are driver-level additions to FAT32 and exFAT volumes that address the lack of journaling in much the same way that Apple has with HFS+ in OS X. But those are currently only implemented in mobile OSes.
That’s not it. exFAT also isn’t supported by Time Machine in OS X, which requires an HFS+ volume. Another odd limitation of exFAT support in OS X is that you can’t create a software RAID array in exFAT format, but you can do it with the FAT32 format. It also has very limited permission and ACL support for those who need to isolate different users from certain files. If you’re just planning on sharing a video or music library, this isn’t a problem. Still, the lack of journaling could be problematic.
In the limited testing I did, bundled OS X applications launched fine from an exFAT volume. I wouldn’t recommend installing apps to a non-HFS+ volume, though, since the lack of permissions support could probably cause issues with larger applications.
For content developers and designers looking to share files between Macs and PCs, you should have no issues using exFAT as a bridge format. You should be careful when copying older Mac fonts to an exFAT-formatted disk, though. Older Mac PostScript font suitcases have resource fork data that is not retained when copied to non-HFS or HFS+ volumes (HFS is the older pre-OS X file system, for those wondering). So if you’re using anything other than HFS+ disks, it’s best practice to use the Finder to zip your fonts if you’re transferring older suitcase-based PostScript fonts. The OS X Finder’s zip supports resource forks—that’s one of the reasons it creates the __MACOSX folder in zip archives. Newer font formats like OpenType don’t use resource forks, so newly purchased fonts won’t be an issue on non-HFS+ volumes.
Since the NTFS file system has been around for a while, it is now well supported across all OSes with some free and commercial options. As a cross-platform file system, NTFS’s major appeal is that it supports journaling across all OSes—without any driver-level fixes or workarounds.
By default, OS X reads NTFS volumes but can’t write to them. You need additional software to get write access to these NTFS disks. The FUSE-based NTFS-3G was the most appealing option since it was free, but it hasn’t been updated since 2011 so I wouldn’t recommend it. Two popular Mac options are Tuxera NTFS, which is a FUSE-based implementation, and Paragon NTFS for Mac. Both packages offer control over volume caching, NTFS-specific options, and some handy additions for full Mac OS X compatibility:
Both Paragon and Tuxera let you format NTFS right from within Disk Utility, but Paragon is the only one that supports formatting of NTFS compressed volumes and full journaling of NTFS disks. At $20, it’s also currently $10 cheaper than Tuxera. After you see the benchmark results, it should be pretty clear which is the choice to go with.
Apple’s Boot Camp drivers install read-only support of HFS+ volumes by default so you can access those Mac partitions within Windows. Early versions were very unstable and caused frequent BSODs, but the latest version seems to work fine. The lack of write support is the real drawback, so you’ll need MacDrive for that. As the best-known HFS+ driver for Windows, Mediafour’s MacDrive has been around for a while now.
I’ve used MacDrive for years and have always liked its feature set. Apart from the basic read/write features, it lets you set volumes to read-only in case you don’t want Windows messing with your volumes. If you format a partition on a Mac OS X volume within Windows, MacDrive will protect the main partition info so that your other Mac volume data is unaffected. This frequently came in handy when formatting the test partition for the benchmarks in Windows 7 SP1.
The Pro version of MacDrive adds support for OS X HFS+ software RAID volumes, secure delete, and auto-defragmentation, which could be a boon to multimedia users who prefer to keep large media files from being fragmented.
MacDrive has partial support of journaling—if your machine lost power or the drive was removed while copying in OS X, it will read the journal entry and try to complete the copy. But if you’re in Windows, MacDrive does not edit the journal, so you get no journaling support for file system operations done within Windows. At $49.99 for MacDrive Standard and $69.99 for the Pro option, that makes MacDrive a harder sell given the slight handicap when compared to Paragon in OS X. Obviously, if you have an external HFS+ software RAID you need to share, MacDrive Pro is the only option. But if you are starting with disks from scratch, NTFS is the cheaper option.
For the benchmarks, I took a new Samsung 840 Pro SSD and partitioned it with one volume acting as the blank test partition. The disk was run inside the SATA3 MacBook Pro’s disk chassis and then swapped into the PC and connected to a SATA3 port of my ASRock Extreme4 X79 motherboard. I ran Intech’s cross-platform disk benchmarking tool Quickbench on the newly formatted partition and recorded the average of five runs. Test OSes were OS X 10.8.3 and Windows 7 SP1 64-bit. I was initially going to test the SSD in Boot Camp on the same machine, but Apple’s EFI bootloader forces Boot Camped Windows to run in IDE mode for a serious performance hit. Conspiracy theorists can put the pitchforks down—this was done for Windows installer compatibility, not to purposefully disadvantage Windows on Macs.
I read somewhere that the performance of FAT and exFAT volumes was part of their downsides. After reading some more and performing some benchmarking, it seems that this really isn’t an issue after all. I found out that the reverse was actually the case: file systems like NTFS and HFS+, which use journaling to ensure data integrity, are very slightly slower than exFAT in all scenarios:
Predictably, my newer and faster overclocked Ivy Bridge gaming rig is faster than the 2011 MacBook Pro—interestingly, with the exception of random writes. As for the inter-file system benchmark results, you can see that exFAT and FAT32 are actually quite fast. Except for random writes, NTFS on Windows and HFS+ are just slightly slower than FAT32 and exFAT for almost all tests. The speeds become almost exactly the same for large file sizes. If you’re using this to share large media files, it’s only the 1024KB that should interest you, which means that you can pick any file system except for Tuxera NTFS and be quite happy. Tuxera’s FUSE implementation of NTFS shows the limitations of a file system in user space instead of kernel space, and I’d guess that you’d see similar issues with FUSE-based exFAT in Linux.
The exFAT file system accomplishes much of what it set out to do: replace FAT32 as a decent cross-platform option. exFAT disks or partitions can be easily shared between OS X 10.7 and above and various Windows variants, provided you have installed the drivers for the more ancient versions of Redmond’s OS. The elephant in the room is Linux support—with only a FUSE-based exFAT option that is likely not as fast as kernel-level implementations. If you’re looking for a file system to play well across three OSes, NTFS is a good option since there are reliable kernel-level implementations of NTFS for Linux. For Mac HFS+ disk access, I’ve been using the kmod-hfs driver reliably with an external eSATA RAID in CentOS 6.3.
If you’re looking to share an external disk and don’t need Linux compatibility, exFAT will do fine. However, if you’re paranoid about data loss and need cross-platform support, I can’t recommend exFAT—you’re better off using NTFS and Paragon NTFS on the Mac side. Those represent the best way to prevent data loss when compared to HFS+, which doesn’t have a fully journaled implementation in Windows. Hopefully some day we’ll see the TexFAT extensions make their way to Apple’s exFAT support. Until then, this is best way to go—it also happens to be the cheapest.
Thank you, TiA