@modulux The thing that scares me about nfs is that it’s an ancient protocol, and authentication and encryption feel like a bolted on mess. Especially because I haven’t set up Pam or whatever to share accounts between the systems. So I’m confused about what happens when nfs tries to sync permissions.

@quanin One server has all the storage, and the other has all the cpu and ram. The first server is just packed with 20 tb spinning discs, and enough cpu and ram to serve files.

Sounds like an NFS share is what you want.

This.

@quanin@tripplehelix So I guess the first thing I need to do is guarantee the users and groups on both machines are identical for NFS, right? Because unlike SFTP I'm not logging in as some particular user.

Nope, it'll automatically convert the user account file permissions. All you need to do is install the server and client and setup /etc/exports on the server. Then test mount on the client then add it to /etc/fstab.

@tripplehelix@quanin Oh, got it! For some reason the nfs documentation makes that look more complicated.

@tripplehelix@quanin Thanks! I'll have to read through the options carefully, because I also want to know what happens if the network connection goes down for some reason. Does it silently unmount and then the ram server starts writing to its local disc and filling it up? Or do writes fail? Or do they get cached? Or what? What happens when I fix the network connection? Does it remount? Or stay broken? Or sync some cache somewhere? I now understand how this works with rclone and I don't like it LOL.

@yo The issue with syncthing is that one machine has 200 tb of storage, and the other has 2 tb. So syncing isn't an option.

@dashdsrdash I need one machine to be able to do file locking and random read/writes on the filesystem on the other machine.

drbd or ceph, then. Probably drbd.

@dashdsrdash So it looks like DRBD might be more than I need. NFS over a wireguard tunnel looks like it will do 99 percent of what I need. I just have to figure out what recovery looks like in a network failure scenario.

@jscholes They need to be shared. The machine that needs to act on the files doesn't have enough space to store the files.

I'd probably still be looking at a rotating sync. Grab some files, work on them, upload the results, delete and repeat.

@jscholes Right, but we don't know what files we're going to need until we need them. And we need to know what files are available at all times. So now we're into keeping an updated list in sync between two machines, and doing caching and batch syncs, and this is all starting to sound like a filesystem.

Fair enough. Not knowing what you're trying to achieve, I can't offer much more.

@jscholes NFS is pretty much what I want. In short, I'm managing my dozens of Linux ISO's and hours of public domain video, but the box that actually downloads the ISO's is on a different network from the one that requests, serves, and indexes them.