.net - LRU file cache and the cost of finding a file in a Windows directory -
i have application download , cache, @ minimum, 250,000 8kb* files totaling 2gb. need remove least used file when updating cache. *these tiny files span 2 4kb sectors.
what relative cost of obtaining file handle name type of file in directory on ntfs-formatted 5400 rpm drive? if store 200k files in 1 directory merely getting file handle take more few milliseconds? can bucket files different directories.
windows 7 disables last access time files default, , don't want require administrator enable feature. should maintain separate list of file access times in memory (serialized disk when app exits?)
should consider storing these files in 1 large flat file? memory mapping might difficult if use older .net 4.0
opening 250,000 files -- if that's mean -- take more few milliseconds, yes. size of directory less interesting fact you're going through entire file system stack 250,000 times (everything ntfs, kernel, , grandmother's favorite anti-virus filter have chance play).
and last access time isn't rock-solid in case.
Comments
Post a Comment