SuperSpy wrote:Also keep in mind that on modern file systems adding a new file to a directory incurs a pretty big hit (at least relative to the effort needed to actually copy a small file) as the directories index has to be updated.
Yup. On the other hand, the indexing also makes meta-data lookups more efficient, so if you have directories with lots of files in them it is a net win (e.g. when copying a file, the system can figure out very quickly whether it already exists, to decide whether it needs to remove the old copy first). Modern file systems also do more caching, which helps. On the other OTHER hand, modern file systems are also journaled
, which improves reliability but introduces additional complications when meta-data gets updated.
People tend to take file systems for granted, but it's a pretty involved topic, and an area where research is still ongoing...