I can reformat a brand new 2TB WD Passport drive to exFAT, with choice of many “Allocation Unit Size”:
128kb 256kb 512kb 1024kb 4096kb 16384kb 32768kb
which one is best if this drive is mainly used for recording HDTV programs using Media Center on Windows 7? thanks.
You should first understand what “Allocation Unit Size” means.
It is the smallest data block on the disk. Your actual data will be seperated to those units while saving to the disk. For example, if you have a file sized 512KB and you have 128KB allocation unit size, your file will be saved in 4 units in the disk (512KB/128KB). If your file’s size is 500KB and you have 128KB A.U.S, your file still be saved in 4 units in the disk because as mentioned above 128KB is the smallest size of an allocation unit (384KB will be allocated in 3 units and remaining 116KB will be allocated in another unit. You can observe this behaviour on file properties screen on Windows, what is your file size and how much space this file actually covers on the disk). And the operating system reads only that A.U.Sd much data at a low level disk read operation.
Those being said, using large A.U.S significantly reduces the free space utilization due to not using the last allocation unit completely. And as a side effect, the number of files to store on the disk is reduces due to same problem, last a.u not being used fully. But, here’s the trade-off, using A.U.S, significantly again, improves the disk reading performance. The O.S can read more data at one read. Imagine, O.S makes couple of disk reads to completely read a GB sized file!..
Using small A.U.S improves the free space utilization but reduces the disk read performance. Think using large A.U.S in reverse, same category problems and improvements, but in reverse…
So, what is the conclusion here? If you will store large, I mean “large!”, files on the disk, higher A.U.S will give you appreciable read performance while reducing the file count and free space…
Which A.U.S you should use? This depends on how much your average file size is. Also you can compute the free space utilization accourding to your file sizes.
Very lucid breakdown. But does each cluster have any inherent storage overhead (e.g. indices or the cluster equivalent of sector headers)? And are there any interactions with physical/emulated sector sizes or cache sizes? Lastly, do larger cluster sizes negatively affect random access performance? 4KB sector HDDs seem to have lower random access performance even though they have higher throughput than 512byte HDDs.
There are no significant storage overhead at high levels. Besides there is enough hrdw overhead since the actual physical sector size is 512Bytes… There is a part of file system formatting that records the cluster information, from how many sector this cluster is created, to the partition structure. The sector size emulation is a job of disk driver. O.S. file system server should deal with logical organization (NTFS, FAT etc) at high level O.S ops, smallest unit reads/writes at low level O.S ops and disk driver itself must work back to back with controller(hardware) for low level hardware access which contains the emulation. And caching is not a job of O.S. It is done by hardware itself. O.S asks for certain data, disk decides wheter look on cache or platter itself for it… Random access performance should actually not be a general performance criteria when having parameters like A.U.S.. Think it this way:
N sized units, M number units, N*M capacity disk, “what is the probability of hitting this unit?” and remember, disk has to be more precise in locating the beginnings of the units.. So, Random access performance is something bound with M^2/N.. 4K units, 8 units, 32K capacity disk. R.A bound with 64/4. 8K units, 4 units, same capacity, same disk. R.A becomes 16/8. You wouldn’t find an article about this kind of calculation, but believe me :) It is more job to “randomly” locate a data using large unit sizes over small sizes.