Howto: Linux on Compact Flash Cards

Earlier this year, I replaced my Linux desktop machine – which was a dual 500MHz Pentium II with 1GB RAM and a pair of SCSI drives on a RAID card – with a newer, more powerful 1.8GHz Mini-ITX machine in a tiny little case. It’s faster, it’s smaller, it’s quieter, and it uses a lot less energy – what’s not to like?

Well, for me, the fact that the case only has room for a single 2.5″ hard drive. I got spoiled by having RAID – and, arguably, I’d been spoiled by having some pretty nice 7200RPM SCSI drives, and I really wanted two drives in the new machine, as well. What was I to do?

Easy: Compact Flash cards on IDE adapters, that’s what.

I wasn’t trying to make a poor man’s SSD – I just wanted something that would fit where a laptop hard drive was supposed to go.

The results were absolutely abysmal. Completely unacceptable – we’re talking 170MHz SPARCStation territory here, while running KDE under Debian. Even dropping from software RAID to just using bare CF cards didn’t help matters significantly. Nor did spending extra on really “high-speed” compact flash cards, either.

I should have just used a regular old hard drive, but I persisted, in large part to prove it could be done – and learn how, along the way.

Several months of testing, fighting, and researching later, I’ve got the low-down on how to do Linux on Compact Flash with as few headaches as possible – and some of the recommendations might really surprise you.

The basic problem is this: Compact Flash cards were never, ever, meant to be used as anything other than storage for large files, and are fundamentally unsuited for any kind of file system that sees lots of small, reasonably random writes. (See this page for a brief discussion of why.)

This is exacerbated by the fact that Compact Flash cards have essentially no write cache (or write buffer, if you prefer that term): 1KB seems to be the norm. You’d be hard-pressed to find a hard drive these days with less than a 2MB buffer, and 8MB seems to be pretty much the norm.

What this means in practice is that Compact Flash cards are, under Linux, reasonably fast to write to – but you incur an enormous penalty in doing so, in CPU usage tied up in i/o wait. That’s almost acceptable if you’re transferring very large files around, but my experience is that your system can become unresponsive – loads in excess of 2.0! – simply from a very small series of very small random writes – like Iceweasel/Firefox writing to its disk cache (which, BTW, it does – under Debian, anyway – even when you’ve explicitly disabled the disk cache. WTF?) every time you load a page. And they don’t cope well with asynchronous i/o, either – trying to read and write on the same CF card at the same time makes everything even worse, in terms of speed and CPU overhead.

That sucks.

A journaling filesystem will make things rather worse still, but there’s little you can do to make things better. No amount of system tuning will provide what I consider an acceptable level of performance. (Note, I haven’t tried a dual-core machine – but I’m convinced you’d still be tying up an entire CPU every time you had a reasonably random write to the CF card, and you can’t tell me that’s acceptable…)

My months of testing suggest that the best option is in fact to use a microdrive in lieu of a Compact Flash card for any filesystems that are going to be written to often.

I know, microdrives can’t approach modern high-speed CF cards in raw performance – but speed isn’t really the problem, CPU usage because of iowait is. And microdrives have a big, big advantage – actually usable write caches! Hitachi microdrives – and most others – have 128KB caches, and Seagate microdrives have, rather amazingly, 2MB caches. The difference this – and a basic medium well able to cope with small, random writes – makes is relatively phenomenal…

That’s really the easy answer – if you want to use something in a Compact Flash form-factor to run Linux off of, get a microdrive or two.

If you are more of a power user, you can use Compact Flash cards and microdrives together and get the best of both worlds. That’s what I now do, and I have absolutely no complaints about the performance of my system now.

I use my system mainly for word-processing and internet browsing, with a little game-playing and music listening thrown in for good measure. All of the recommendations that follow assume that your Linux system is used for roughly similar purposes.

Under Debian, my desktop system (with KDE) uses about 4GB of disk space, roughly as follows:

/usr is 3.1GB, of which /usr/share is 2GB, and /usr/lib is 826MB.
/home is ~250MB at any given moment
/var is 245MB, 191MB of which is /var/lib
/lib is 118MB
/tmp is ~100MB at any given moment
/boot is 35MB
/sbin is 7MB
/bin is 4MB

Other systems will probably be fairly similar to that.

In use as a desktop system, most of these directories will almost never get written to, in practice, and there’s no reason not to put them on fast Compact Flash cards to make your system seem faster and more responsive.

What gets written to a lot? /tmp, and /home. Quick and easy answer – put those on a microdrive. (On my desktop system, at least, almost nothing gets written to /var/log.)

With an 8GB Compact Flash card, and a 2.5GB Seagate microdrive, you could then have a partitioning scheme something like this:

Compact Flash (mount everything noatime!):
50MB /boot
4GB /usr
everything else on /

Microdrive:
250MB swap
250MB /tmp
everything else (~2GB) /home

If you’re really a power user, and want to squeeze the absolute best performance out of these storage devices that were never meant for this kind of use, there is an alternative you can take:

Put /home on the CF card instead, as this will increase read performance of your files roughly threefold;
Make the free space on the microdrive a new, arbitrary mount point (/home2, or something);
Symlink directories that get written to often (/home/username/.Mozilla, for example) over onto the microdrive.

It’s easier than it sounds, but can require some trial-and-error. With this method, you get the advantage of having most of your files on /home/username load really fast – music files, OpenOffice documents, whatever – and you can move the stuff that gets written to a lot over onto a storage medium better able to cope with lots of small writes.

I know it’s strange to think that the microdrive outperforms CF cards in this application, but it’s true. Oh, raw performance speed – from hdparm or the like – doesn’t really favor the microdrive, but speed isn’t really the bottleneck – CPU usage from iowait is.

All of my experiences and recommendations come from using CF cards and microdrives on CF-to-IDE adapters, which are generally understood to provide the best overall performance. It’s possible that USB and PCMCIA adapters will give similar results, but I just don’t have the inclination to play with anything other than internal solutions.

I very strongly recommend a non-journaling filesystem on any flash medium, like a CF card, but you should feel free to use whatever filesystem you prefer on a microdrive – I’m using XFS and JFS on mine, and don’t notice any particular difference in performance compared to ext2.

If you can’t live with the relatively small amounts of space that CF cards and microdrives can provide, I’d suggest trying to shoehorn a 1.8″ laptop hard drive into your system – they’re only a little bigger than a CF card, don’t draw a lot of power, and their read and write performance will blow a CF card or microdrive out of the water. If you’re trying to use a CF card because you think it’s a lot like a high-end SSD, don’t. Only go the CF/microdrive route if you’re weird, are really cramped for space, or have some other reasonably compelling reason for doing so. I really don’t recommend it, when all is said and done – there is a reason that very few people do this, you know! – but if you are going to fly in the face of wisdom and try and use CF cards as system disks, well, you might as well squeeze the best performance you can out of them, that’s what I figure.

Published in: Geekiness, General | on November 9th, 2009| 6 Comments »

Both comments and pings are currently closed.

6 Comments

  1. On 11/11/2009 at 3:22 pm Nemo Said:

    For what it’s worth, you can boost the performance of both CF cards and microdrives slightly with hdparm, but only when they’re used with IDE adapters, so that’s yet another reason to skip USB or PCMCIA adapters.

    For microdrives, try:

    hdparm -c3 -m16 /dev/driveID

    and for CF cards, try:

    hdparm -c1 /dev/driveID

    …which sets the I/O mode for the devices to be 32bit, rather than 16bit, and enables multicount support for the microdrive, which seems to greatly reduce the CPU usage where writes are concerned.

  2. On 11/30/2011 at 2:38 am Guest Said:

    Should have bought a CF card that is in fixed disk mode and supports UDMA. Seems like your CF card was an older/cheaper model that was running in PIO mode.

  3. On 2/21/2012 at 4:02 pm Chris Said:

    Check out Russell’s blog and IBM X40 compact flash SSD swap searches in google. Quite a few ppl run Ubuntu no issues using a CF card and IDE as their SSD.

  4. On 6/17/2012 at 10:31 am A. Peon Said:

    Ancient news, but having made the same ‘mistake’ (actually willing to put up with it for cheap durability on ancient netbook-equivalent Pentium M laptops) I also find laptop-mode helps slightly here in 2012. Set to a modest delay, you have time to work interactively in between the CPU-clogging I/O bursts, however slow they may be. (I think I currently have it set to a conservative 60 seconds and should really up it to 2 or 5 minutes as is default.)

    It appears turning the disabled-by-default write cache on on my card (also _reports_ as 1kb but who knows?) also helps performance significantly – increase from around 500k/s to 2MB/s peak write in iostat when swapping (yes, swapping!) – so perhaps that permits some internal disk housekeeping to proceed in quicker fashion. I had it off (as it is by default due to IDE devices lying about sync, IIRC) for corruption concerns but perhaps that was misplaced.

    [My cards operate at UDMA5 and reads are snappy, but writes do kind of suck and the iowait load is indeed scary.]

  5. On 3/9/2013 at 4:14 pm smartd expert Said:

    I have used CF chips in a few Linux boxen since 2008 with good results. The key was to get a SATA2CF controller and a fast CF chip. I usually procure Syba SATA2CF controllers (@ Newegg for ~$16) and 4GB or 8GB 266X CF chips. I install the base system on the chip and put RW filesystems on another HDD. I also set option “noatime” on in /etc/fstab to prevent unnecessary writes and extend the life of the chip. SATA is so much faster that IDE

  6. On 1/27/2017 at 5:39 am Chuck Said:

    Welcome to 2017! I’m using 1066X (up to 160MB/s reads) CF cards in iBook G3 Clamshells from 1999 and 2000. The 300MHz models with a 33MHz bus speed run Debian 8.7.1 for PowerPC. The 366MHz with a 66MHz bus speed run Lubuntu 16.04 for PowerPC. Both have IDE connections on the logic board (motherboard). USB is 1.1. So I’m looking for speed over longevity from CF cards. The out of box installs were dramatically faster than the same exact installs on hard drives. Still have I/O wait, especially on the 300s. Of course I’d like to reduce the read/write cycles, but I have yet to try any tweaks (besides noatime) because almost all of the info is from 2006 to 2010. Some of the tweaks will break Debian at least. Has anyone knowledgeable in the field got any good suggestions or websites?