I will say there's a lot out there in the world of distributed and cluster filesystems (and, frankly, most of the free stuff is half-baked). One problem you'll have, though, is that most of the good quality solutions out there are not made to work with replication across wide area networks. They are generally designed for local area replication, because WANs usually have quite limited bandwidth and high latency. If you really want to do WAN replication with a distributed filesystem, you might try looking at GlusterFS, which does not require shared disks and can replicate (Lustre doesn't require shared disks but doesn't do replication itself -- they suggest hardware RAID or block-device replication like DRBD, which would be awful over a WAN). I don't know how well it'll perform, though. Like some others have mentioned, an above-FS replication mechanism might ultimately work better with your particular use-case.*
As notfred mentioned, Coda has nice features, but it's kind of a dead end these days. Coda and AFS were created in the 80s and early 90s at CMU as distributed filesystem research projects. Now that the research is done, Coda is very lightly maintained and it never really had a large user base to begin with. Typically with research software, the research goals are not really in line with creating and maintaining production-ready software.
shank15217 wrote:You could try OCFS2 from Oracle. Its free and is a hell lot easier to setup than coda.
OCFS2 is a shared disk filesystem. The developers point this out in the intro to the OCFS2 1.4 manual: "OCFS2 is a symmetric shared disk cluster file system. ... As the name suggests, such file systems require a shared disk (SAN). All server nodes in the cluster must be able to perform I/O directly and concurrently to the disk. A cluster interconnect that provides a low latency network transport is used for communication in between the server nodes." If file servers are around the world, they're not going to be on a common, low-latency SAN.
* Actually, I've worked as a developer on a high-performance parallel cluster filesystem from one of the large commercial vendors, and one of the "value-added" storage solutions they are developing is essentially designed to deal with use cases similar to yours. Basically, the system provides multi-site WAN caching and reconciliation to support "follow the Sun
" development where you have multiple sites all over the world that all want a window on to a coherent data set, and you have updates coming in from different places (but cross-site concurrent write sharing is rare). I'm pretty sure it's not released yet, though.