Saturday, November 26, 2016

Same files have different "size on disk" on new drive



I recently bought a 8TB HDD to replace my 1TB drive for storing my backups. However after moving the files to the new drive I noticed they take up a substantially larger amount of space on the disk, even though the true file size is the same. Both drives are formatted to be exFAT so I am not sure why the reported size is so different. Why is substantially more space being used on the new drive?



File size comparison (Left is old, right is new)



Answer



The default cluster size for exFAT is extremely large. According to this answer it's like this



Volume size   Cluster size
500 GiB 128 KiB
1000 GiB 256 KiB
1500 GiB 512 KiB
2000 GiB 512 KiB
3000 GiB 1 MiB
4000 GiB 1 MiB

5000 GiB 2 MiB
8000 GiB 2 MiB
10000 GiB 4 MiB
20000 GiB 8 MiB


Your files are 753674490827/726245 ≈
0.9897 MB on average, so on the 1 TB drive with 256 KB cluster, about half of the files take 4 clusters (and waste almost nothing) and roughly the remaining half need 5 clusters (and waste 1 cluster)



The average wastage is 0.5 clusters = 128 KB per file or 12.5%. The total expected size on disk is 753674490827 × 112.5% ≈ 789.7 GB which is extremely close to what's in your screenshot. However on the 8 TB disk, the clusters are 2 MB, so you're wasting ~1 MB per file, which is why the new size on disk is almost twice the total size and the previous size on disk




See





It's also the same reason as this question Why is there such a big difference between "Size" and "Size on disk"?



On NTFS there are many other reasons. If you're interested read Why does "Size on disk" vary when "Size" does not with the same set of files?



In short, exFAT is bad for storing small files. Either use NTFS, or reformat exFAT with a smaller cluster size (4 KB just like almost everyone else including NTFS)



No comments:

Post a Comment

hard drive - Leaving bad sectors in unformatted partition?

Laptop was acting really weird, and copy and seek times were really slow, so I decided to scan the hard drive surface. I have a couple hundr...