Compact Flash Cards

Page created: 28th July, 2000

Last Update: 1st August, 2003

This Compact Flash page is available as a downloadable EPOC Word file (inc. tables) here:-  CompactFlash.wrd  (34Kb)    (Updated: 18th July '03)

 Also, a Data file containing reported makes/sizes of CF cards working/not working in Psions:-  CF Database    (New: 1st August '03)

(If you can contribute to the database above then please email me at Anti-SPAM email image: just type it in as you read it.  :¬)


Latest Data Updates: 

I'm very grateful to Dalibor Jelinek who has kindly sent me the figures he has measured from his 96Mb Delkin Devices and 192Mb PQI (aka Hitachi) CF cards.  I've added both of these to each of the tables below.  Dalibor has also corrected a lot of my understanding of FAT formats and I've added a Technical Notes section at the bottom of the page which is essentially his contribution).   For other contributor acknowledgements, please see here.

Introduction: 

As I've gone through owning and using first a Series 5 and now a 5mx, I've gradually progressed to using larger and larger Compact Flash (CF) cards.  I started off with a 15Mb card, then a 22.5Mb card (odd size but it was a freebie!), then a 48Mb card, and just recently I've moved to a 128Mb card.  These days, the wealth of quality software available for the EPOC platform means that there's always one more program that you'd like to squeeze onto your machine and - whilst I'd survived quite happily for about 18 months with my 48Mb CF card - the temptation of TomeRaider proved just too much!  ;-)

CF Card Readers: 

One thing I can recommend straight away when you're looking to upgrade your CF card is a CF card reader for your PC.  These plug into your PC either via a USB port or via a daisy-chained parallel port connector (so that you can still use your parallel port for printing).  They're not very expensive (typically around UK£50) and they speed up the process of complete CF card transfers enormously.  A complete backup (i.e. from scratch) of my 48Mb card took about 1½ hours via the Psion's serial port + PsiWin.  A complete transfer direct from the CF card reader to or from the PC took about 5 minutes.  They're also extremely useful if you own a digital camera that uses Compact Flash (as opposed to the Smart Media format) for very rapid transfers of your photos to your PC's hard disk.

A typical CF card reader

I've noticed in a number of newsgroups and email digests that people have had problems with CF Card readers not coping with long file names (i.e. greater than the standard DOS 8+3 format).  Indeed I had exactly this problem with the Kingston 'PCREAD/CF CF Card Reader' that I own up until recently.  Specifically I noticed that if I backed up my CF card to my PC using the CF card reader and then restored the files back to the CF card, many of the file names would be truncated and/or in caps.  Thinking about it, I realised that the problem had to be with the CF card reader's driver for Windows rather than the hardware itself.  I hunted around on Kingston's website and found new drivers for Windows 98 (as well as drivers for Win 2000, etc.), installed them, and the problem now seems to have been fixed!  The moral of this story: always make sure that you have the most recent software drivers.  :-)

A typical Compact Flash adapter

An alternative option (especially if you own or use a laptop) is to use a Compact Flash Adapter that effectively converts your CF card into a PCMCIA card (aka. PC card).  This will then allow you to plug your CF card directly into one of the PCMCIA slots on your laptop (or desktop if you've got a PCMCIA card reader installed in it).  Typically these adapters retail for about UK£10 - £15.  In the case of both the CF card adapter and the CF card reader above, Windows will normally 'mount' the CF card as a removable drive under your 'My computer' tree and allow you to use it as though it were just another hard drive...

Alan Smith also kindly sent the following suggestion: "I got a Casio QV-300 Digital Camera, which has USB file transfer. If I put a Psion Compact Flash in the camera I can read it as a removable drive on my PC for file transfers, backups etc. I'm not sure if this works for all USB enabled cameras, but could save £50 on a CF reader, or be an extra £50 towards the next camera up in the range!"  Good idea - nice one Alan!

Formatting techniques: 

When I first got my 128Mb card, it was already formatted and so I just dumped the contents of my 48Mb card onto the PC using the CF card reader and then copied it back onto the 128Mb card.  It worked perfectly when I plugged it into my Psion.  However, after a while I realised that some applications (especially those that wrote data to d: drive) were very slow.  So slow in fact that I began to wonder whether the upgrade had been a good move at all...

The standard method (i.e. the only one that I know!) that most people use for measuring the speed of their CF cards in Psions is a small freeware program called DiskBench by Jan Rydval.  You can visit his site here for full details but basically the program works by writing a 1Kb file to the disk 10 times and then reading it back 10 times and measuring the times taken to do both to give measurements for drives c: and d:.  The absolute values measured are obviously fairly arbitrary - but they're useful for comparisons - and they do give a reasonable indication of how the card will perform in your Psion.  Just in case his site goes down for any reason, I've also put a copy of DiskBench v1.1 here.  I realised after a while however that the measurements that he has posted on his site are fairly meaningless as he's failed to record how the cards were formatted (although I assume that he's used the standard Psion format).  After some experimentation, it dawned on me that how you format the CF card has a very considerable effect on how the card will perform - both in terms of its speed and its formatted capacity.

Typical output from DiskBench v1.1...

Typical output from DiskBench v1.1...

It transpires that - for any given size of card - there is a pay-off between the speed of the card (as measured by DiskBench for example) and the card's formatted capacity.  It's important to clarify what I mean by 'formatted capacity' however.  A 16Mb CF card has 16Mb (Megabytes) of raw space for holding whatever you decide to put into it.  However, EPOC uses the standard MS-DOS formatting structures for CF cards (which is handy because it makes formatted cards interchangeable with PC card readers, most CF digital cameras, etc.).

Formatting CF cards is like formatting hard disks on your PC.  It establishes the framework inside the card which will hold the actual data by filling your card with pockets for holding the data (called 'clusters').  So it's not just the size (or the reported size) of the disk that's important; it's also the cluster size - as this can greatly affect their actual capacity.  A cluster is the smallest 'pocket' of data that your hard disk (or CF card) can store in one chunk and normally (i.e. by default) the cluster size increases as the size of the hard disk (or CF card) increases.

A 4Kb cluster containing only 1Kb of data...

Generally, cluster size can vary from between 512bytes (½Kb) on a small CF card to 128Kb on a large card.  This is important because if you've got (say) a dozen files each 1Kb in size that you want to store - but your cluster size is 4Kb - then you'll still need a dozen clusters to store them in = 12x4Kb = 48Kb.  That's 48Kb - 12Kb = 36Kb wasted space.  In other words, 75% of the space is being wasted!  

This may sound like an extreme case but it's not.   Consider that the Email program (Message Suite on ER3 machines) stores each individual email received, sent, etc. as individual small files (as well as the attachments) - as does the newsreader program RMRNews.  In my case, that's maybe 100 emails received and 50 sent at any given time - each being stored as small files in whatever the cluster size is.  I.e. there are literally hundreds of tiny files...

So you might assume that the obvious fix for increasing the effective formatted capacity of your CF card is to always have the card formatted with the smallest possible cluster size..?  Certainly this is fine for small (say <16Mb) cards.  However, as the size of the card increases, then the total number of clusters increases greatly if the minimum cluster size is always used.  On larger cards (say >20Mb) this begins to affect the cards performance in terms of speed of writing to the card and on the largest cards (say >48Mb), using the minimum possible cluster size become prohibitive in terms of the card's speed.  Hence there is a practical limit as to how small a cluster size you'd want to use on a larger card - as well as to how large a cluster size you'd want to use on smaller cards... 

Atelier's 'Essential Disk Utilities':

A 96Mb CF card from SanDisk...

"This is all very well", you might say, "but my Psion doesn't give me any options as to how I format my CF card".  Very true.  However, a 3rd party company called Atelier (see my Commercial Links page for their website) have produced a suite of disk utilities for EPOC called Essential Disk utilities (EDU).  One of these utilities - called SmartFormat - allows you to vary the format parameters of the card.  The documentation that comes with the program is fairly poor insofar as there's no description of what the various options actually mean (on the other hand, this means that you won't lose much if you wanted to buy the software online and download it).  However, the 2 most important parameters that you can vary are 'Sectors per cluster' (i.e. the cluster size) and 'File system'.  File system allows you to chose between the Psion's default FAT12 system and the more common MSDOS (aka Windows!) FAT16. 

Atelier's SmartFormat program - Advanced options

Atelier's SmartFormat program - Advanced options

Is there an alternative to EDU?:

I've been told that some versions of PowerQuest's PartitionMagic program will let you format removable drives on the PC - and specify the cluster size, FAT type, etc.  I've not been able to confirm this unfortunately,

However, you can also use the DOS format command to play with the number of sectors per cluster.  It has the hidden option /Z:n where n is the sectors per cluster.  Eg. Format E: /Z:32  Where E: is the drive in question. Make sure that you the correct drive letter however!!  If you manage to seriously mess up the CF card using Windows (such that it cannot be formatted on the Psion), then FDisk will allow you to remove the problem partition & put a new blank one it it's place. I would not recommend using FDisk unless you are very experienced however; as you could easily wipe your Windows hard disk :¬(

'Real world' results:

"Okay, so I know how to change the formatting of the CF card - but what can I expect and what's 'good'?"  Well, the only way of finding this out is really just to carry out a bunch of tests on various sizes of cards.  Since I've collected a number of these over the years, I thought I'd report the results here.  DiskBench v1.1 reports both the Read time and the Write time results and so I've created 4 tables that show the Write time results for both the FAT16 and FAT12 formats and their corresponding Read time results.  Where I've quoted a brand of CF card with another brand name in brackets, this is to show the the actual manufacturer's name (in the brackets) v's the brand that it's sold under.  E.g. the Kodak branded cards that I have are all actually manufactured by SanDisk.  NB: 64Mb SanDisk figures kindly contributed by Gavin Probyn, 32Mb Simple Technology figures kindly contributed by Rob Mattholie, 128Mb SanDisk figures kindly contributed by Michi Oshima, 64Mb Lexar / 80Mb Lexar / 256Mb Simple Technology figures kindly contributed by Mike McLellan, 96Mb Delkin Devices / 192Mb PCI figures kindly contributed by Dalibor Jelinek.

The first table shows the Write times for FAT16 using various cluster sizes.  As can be seen, the write speed of each card varies quite considerably with the cluster size - and generally decreases with the size of the card as you would expect.  In my experience, I've found that a write speed in the region of 5 - 6 seconds (as measured by DiskBench) is reasonable both is terms of 'real world' speed of day-to-day use and in terms of useable formatted capacity.  Hence, I've highlighted the values that I'd recommend for each of the cards below in Orange.

NB: Please note that all the figures quoted below are measured on a 5mx/5mxPro.  Whilst the actual figures measured on a Series 7, netPad, etc. are likely to be different, the same principle applies.

FAT16 Write Sectors/Cluster

256Mb Simple Tech. 192Mb PCI (Hitachi) 128Mb SanDisk

128Mb PCI (Hitachi)

96Mb Delkin Devices 80Mb Lexar 64Mb Lexar 64Mb SanDisk

48Mb Kingston

32Mb  Simple Tech.

20Mb SanDisk

16Mb Kodak (SanDisk)

8Mb Kodak (SanDisk)

4Mb Kodak (SanDisk)

1

10.06 X 10.77

10.94

X X X 12.65

10.15

11.34

9.11

7.35

5.31

6.14

4

10.07 X 10.80

10.98

7.65 9.47 9.28 8.83

6.92

4.96

5.73

4.24

3.82

5.17

8

10.18 8.28 6.99

7.46

5.04 7.12 7.35 6.80

5.29

3.88

5.20

3.84

3.60

4.90

16

6.50 5.63 5.15

5.64

3.66 5.92 6.49 5.82

4.94

3.34

4.88

3.63

3.46

4.95

32

4.95 4.30 4.07

4.85

2.26 5.28 5.92 5.34

4.02

3.09

4.71

3.51

3.38

X

64

3.03 3.01 3.10

4.10

2.02 3.22 3.72 4.69

3.50

2.74

4.22

3.15

X

X

FAT16 Write values measured for various cluster sizes 2

You might ask why I've chosen this range of speeds as opposed to faster ones.  Well, I've found that using the larger cluster sizes on these cards greatly affected the usable capacity of the card.  There's no standard program for measuring the effect a CF card's cluster size has on usable capacity so I've taken the fairly ad hoc method of simply seeing how mach space my Psion reported as having left on the card once I'd loaded all my programs and data back onto the card once it had been formatted.1

  1. NB: I don't mean just look to see what your Psion reports as the size of the CF card.  I mean check to see how much space it says you've got left on your CF card once you've loaded all your data back onto it.  Obviously, this will vary completely from person to person but - so long as you use the same data for each test - it will give you a reliable and realistic indication of how much use that format method is to you.

  2. The cells marked 'X' in the tables are where SmartFormat refused to format the card.  My assumption here is that there is a limit as to how big a cluster size you can have in small CF cards.

This was a fairly subjective and non-scientific way of measuring it's usable capacity - but I think that it was a pretty good real-world measurement as I use both Email and RMRNews on my machine - both of which generate a large number of small files on my CF card.  Obviously this test takes quite a bit longer to carry out and so I've only carried it out on my 128Mb card and on the results I've highlighted in Pink in the tables. For the FAT16 / 16 sectors/cluster result above, my Psion reported that I had 40.3Mb remaining on my card.  However, this decreased to 17.2Mb for the same data on the card when formatted for FAT16 / 32 sectors/cluster.  Hence a big increase in usable capacity for a relatively small decrease in speed.

I then performed the same tests using the FAT12 format and the results are below:-

FAT12 Write Sectors/Cluster

256Mb Simple Tech. 192Mb PCI (Hitachi) 128Mb SanDisk

128Mb PCI (Hitachi)

96Mb Delkin Devices 80Mb Lexar 64Mb Lexar

64Mb SanDisk

48Mb Kingston

32Mb  Simple Tech.

20Mb SanDisk

16Mb Kodak (SanDisk)

8Mb Kodak (SanDisk)

4Mb Kodak (SanDisk)

1

8.30 X 8.92

9.18

X X 11.34 10.72

8.57

9.24

7.79

6.45

4.90

6.11

4

8.19 X 10.74

10.94

6.34 9.23 9.32 8.85

6.87

4.43

5.77

4.20

3.73

4.94

8

10.06 X 6.02

6.74

4.38 6.50 6.91 6.25

4.90

3.65

4.97

3.68

3.53

5.01

16

5.66 X 4.58

5.18

3.36 5.85 6.16 5.58

4.23

3.22

4.77

3.48

3.45

4.93

32

4.23 X 3.87

4.53

X 5.52 5.97 5.19

3.90

3.02

4.74

3.25

3.47

X

64

2.90 2.52 3.10

3.97

X 3.30 3.63 4.51

3.47

2.72

4.24

3.09

X

X

FAT12 Write values measured for various cluster sizes

At first glance, the speeds would appear to be slightly faster than those using FAT16 (NB: see the Technical Notes section kindly contributed by Dalibor Jelinek below).  However, this is more than compensated for by a big loss in usable capacity.  I only carried out one 'usable capacity test' using the same data on the card as in the previous table (again, on the Pink value above).  My machine reported only 13.1Mb space remaining on the card v's the 40.3Mb available when formatted using FAT16 with the same cluster size - wow!  If I really wanted the better speed, I'd be better off using FAT16 + 32 sectors/cluster...

Interestingly, the default format that the Psion uses on its own is FAT12 + 64 sectors/cluster (shown above in blue) - pretty disastrous if you're using a large CF card!  (Ed's note: Not necessarily true - see the Technical Notes section below)

For completeness, I've also shown the Read speeds for FAT16 and FAT12 below.  Actually, these are quite interesting as they probably give a more accurate indication of the physical speed of the raw CF card (since they don't vary either with FAT format or the cluster size).  It also goes some way to explain why my 4Mb Kodak card appeared slower than its 8Mb neighbour in the Write tests above.  I suspect that it was manufactured using slower Flash chips than in the 8Mb card - could be an older card...

FAT16 Read Sectors/Cluster

256Mb Simple Tech. 192Mb PCI (Hitachi) 128Mb SanDisk

128Mb PCI (Hitachi)

96Mb Delkin Devices 80Mb Lexar 64Mb Lexar 64Mb SanDisk

48Mb Kingston

32Mb  Simple Tech.

20Mb SanDisk

16Mb Kodak (SanDisk)

8Mb Kodak (SanDisk)

4Mb Kodak (SanDisk)

1

0.81 X 0.82

0.82

X X X 0.86

0.94

0.98

0.82

0.87

0.89

1.03

4

0.81 X 0.82

0.82

0.81 0.83 0.86 0.85

0.94

0.97

0.81

0.86

0.87

1.02

8

0.81 0.81 0.82

0.81

0.81 0.83 0.86 0.84

0.94

0.96

0.80

0.85

0.87

1.01

16

0.82 0.81 0.83

0.81

0.81 0.85 0.86 0.84

0.94

0.96

0.80

0.86

0.87

1.02

32

0.81 0.81 0.82

0.81

0.81 0.83 0.86 0.84

0.94

0.96

0.80

0.86

0.87

X

64

0.81 0.81 0.81

0.81

0.81 0.83 0.85 0.83

0.94

0.96

0.80

0.86

X

X

FAT16 Read values measured for various cluster sizes

FAT12 Read Sectors/Cluster

256Mb Simple Tech. 192Mb PCI (Hitachi) 128Mb SanDisk

128Mb PCI (Hitachi)

96Mb Delkin Devices 80Mb Lexar 64Mb Lexar 64Mb SanDisk

48Mb Kingston

32Mb  Simple Tech.

20Mb SanDisk

16Mb Kodak (SanDisk)

8Mb Kodak (SanDisk)

4Mb Kodak (SanDisk)

1

0.81 X 0.83

0.82

X X 0.86 0.85

0.95

0.98

0.83

0.87

0.89

1.03

4

0.81 X 0.82

0.81

0.81 0.86 0.87 0.84

0.94

0.97

0.81

0.85

0.88

1.02

8

0.81 X 0.83

0.82

0.81 0.86 0.86 0.84

0.94

0.96

0.81

0.85

0.87

1.02

16

0.81 X 0.82

0.81

0.81 0.86 0.86 0.84

0.94

0.96

0.80

0.85

0.87

1.02

32

0.81 X 0.82

0.81

X 0.86 0.86 0.84

0.94

0.96

0.80

0.86

0.86

X

64

0.81 0.81 0.82

0.81

X 0.85 0.86 0.84

0.94

0.96

0.80

0.85

X

X

FAT12 Read values measured for various cluster sizes

 

The variation in Read speeds may also explain why I noticed the following:  When I first plugged my new 128Mb CF card into my Psion it appeared to be very slow - apart from when it was looking for a file/folder in a standard dialog box.  My assumption is that it was 'poorly' formatted (hence the apparent slow operation) but it's file/folder dialog box speed is dependent upon the cards Read speed - which was slightly faster than my previous 48Mb CF card.  However, it's also possible that this is related to the fragmentation of the disk and the order of directory entries on the disk...

 

Technical Notes:
(NB: This section contributed by Dalibor Jelinek)

The fact is that FAT12 is always faster and more efficient that FAT16 with the same number of sectors per cluster. Technically the only difference between FAT12 and FAT16 is that FAT12 uses 12-bit integers for numbering clusters and FAT16...  This means that the FAT chain for FAT12 uses 25% less space than a FAT16 chain.  That means that you can fit 33% more information in one sector and that's in my opinion the reason for why the 'writes' on a FAT12 formatted CF card are faster.

FAT12 by definition is ALWAYS more space-efficient than FAT16 but ONLY compared to FAT16 with the SAME cluster size.  However there are two other disadvantages with FAT12:-

  1. First is that numbers are 1½ bytes long and that complicates the FAT handling a little bit (but this is only an issue for the software programmer and not the user).

 

  1. The second disadvantage is more important and it relates to the total number of clusters addressable bit FAT12.  When using FAT12 the valid cluster numbers are 0x001 - 0xFF6.  This gives you a maximum of 4079 clusters per disk.  Depending of your sector/cluster value (and I suppose a normal 512 bytes sector) the maximum sizes for a disk formatted as FAT12 are:-
FAT 12 Sectors/Cluster Maximum addressable memory
1 1.9Mb
2 3.9Mb
4 7.9Mb
8 15.9Mb
16 31.9Mb
32 63.8Mb
64 127.7Mb
128 255.4Mb

This is the reason why FAT12 appears to waste so much space.  Because in fact if you use (say) 16 sectors/cluster on a 128Mb CF card, it means that you have only 31.9 MB of usable space. The strange thing is that SmartFormat doesn't tell you anything about this and is quite happy to format the flash 'illegally'.

My suggestion therefore is:  If you have already decided that 16 sectors/cluster is the best compromise between speed and wasted space AND it's sensible to use FAT12 formatting for your disk (considering the maximum addressable size of a FAT12 disk with that given sector/cluster value) then ALWAYS choose FAT12.  But ALWAYS first base your decision on speed v's wasted space for for the number of sectors/cluster and then choose FAT12 or FAT16 depending upon whether FAT12 will allow you to address all the memory.

Just for completeness there are the maximum sizes of disks using FAT16 (max 0xFFF6 = 65526 clusters):-

There are more problems with SmartFormat than just this one. 

  • One strange thing is that it allows you to format the disk with sectors/cluster values which are not power of 2 (1,2,4,8,...) - and this then renders the disk unusable.
  • The next problem I've found is that every time I have formatted the CF with
    SmartFormat and then put it in my PC CF reader and run Chkdsk on it there was one lost cluster on the CF.  So I'd recommend that people immediately run the CheckDisk utility after SmartFormatting their flash card.
FAT 16 Sectors/Cluster Maximum addressable memory
1 31.9Mb
2 63.9Mb
4 127.9Mb
8 255.9Mb
16 511.9Mb
32 1023.8Mb
64 2047.7Mb
128 4095.4Mb

Dalibor's Additional Suggestions:

 

Thanks Dalibor.  Very interesting reading - and food for thought.  :¬)

So that's it.  If you've any thoughts, comments, or ideas on any of the above (or perhaps a set of results to fill in any of the CF card gaps), then please email me at Anti-SPAM email image: just type it in as you read it.  :¬) and let me know.

Martin

Click here to go to the Pscience5 homepage