Published the 2025-05-23 on Willow's site

My perfect music synchronization solution

I'm buying DRM free music for a long time, and I synchronized those music files across my devices using Syncthing. But my library weight growth, now weighting 60G… Some of my devices can't afford this! I had to find an alternative.

Years ago I switched to NFS and FS-cache. NFS is a pretty common folder net mounting. We just have to write a line in "/etc/fstab" to mount a remote folder locally. FS-cache is an additional layer to keep cache on the client machines. It will automatically store and invalidate cache, and will prune the less used blocks before the filesystem start to suffocate. This will considerably reduce the bandwidth usage for clients when you listen the same albums by example. But in the same time, you still need an active internet connection to access the files.

The whole problem is how to authenticate and secure NFS connections? For a long while I hosted my own OpenVPN server, so that all my machines can communicate securely. It was a bit of a pain to configure every new machines, because I had sign and move certificates...

Today NFSv4 can be easily connected to more secure mechanisms, the most legit one named Kerberos. So I spend some time to configure everything. Now that I have a great setup, I'd like to share everything here, because I think it is very cheap to self-host, and not that hard to get everything working. Just a bit overwhelming at first…

In my case the NFS server, Kerberos kdc server, and kadmin service are hosted on the same machines: Your mileage may vary ~

I will not go into too many details, so my main sources are:

https://web.mit.edu/Kerberos/krb5-latest/doc/index.html

https://wiki.gentoo.org/wiki/Nfs-utils#Encryption

https://wiki.alpinelinux.org/wiki/Setting_up_an_NFS_server#Kerberos_authentication

https://wiki.archlinux.org/title/Kerberos

Server config #

Let's start with the server parts. Of course replace the "willowbarraco.fr" domain and realm by your own.

$ apk add nfs-utils krb5-server

DNS config #

$ dig music.willowbarraco.fr
music.willowbarraco.fr.	2703	IN	A	82.66.55.247
$ dig kerberos.willowbarraco.fr
kerberos.willowbarraco.fr. 2691	IN	CNAME	willowbarraco.fr.
willowbarraco.fr.	2691	IN	A	82.66.55.247

"music.willowbarraco.fr" must be a "A", not a "CNAME" here. That's because Kerberos will reverse DNS domains, and a CNAME would behave differently.

Firewall config #

There are very few required ports. You have to configure your NAT to redirect those if needed.

# /etc/nftables.d/access.nft
#!/usr/sbin/nft -f

table inet filter {
	chain input {
		udp dport 88 accept comment "accept Kerberos v5"
		tcp dport 88 accept comment "accept Kerberos v5"
		udp dport 749 accept comment "accept kadmin"
		tcp dport 749 accept comment "accept kadmin"
		tcp dport 2049 accept comment "accept NFSv4"
	}
}

Kerberos config #

Add the "default_realm" and dedicated "realms" part.

# /etc/krb5.conf
[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = WILLOWBARRACO.FR

[realms]
 WILLOWBARRACO.FR = {
  kdc = kerberos.willowbarraco.fr
  admin_server = kerberos.willowbarraco.fr
 }

Then you can start the Kerberos services.

$ rc-service krb5kdc start
$ rc-service krb5kadmind start
$ rc-update add krb5kdc
$ rc-update add krb5kadmind

Create Kerberos admin principal for remote admin #

This admin principal is used to remotely administrate your Kerberos services. Use a strong password here.

$ kadmin.local
addprinc admin/admin

Write this file.

# /var/lib/krb5kdc/kadm5.acl
admin/admin	x

NFS config #

Add those exports.

# /etc/exports
/home/willow/music  192.168.1.0/24(rw,sec=krb5p,no_subtree_check,sync)
/home/willow/music  *(ro,sec=krb5p,no_subtree_check,sync)

Here I export with read-write for intern IPs, and read-only for outside IPs. Meaning I can only add albums to my library from inside my local network. "krb5p" is to Authenticate, and secure communications.

Edit this.

# /etc/conf.d/nfs
NFS_NEEDED_SERVICES="rpc.svcgssd rpc.idmapd"
OPTS_RPC_NFSD="8 -V 4 --lease-time 30"

"-V 4" to only use the NFSv4 version. And "--lease-time 30" to reduce the client re-connection time when they switch network interface. It is considered not safe to use a lower value.

idmap config #

Idmap is used to map your client machine users UID and GID to the NFS server users ones. So that you got the required ACLs to read and write files.

Edit those values.

# /etc/idmapd.conf
[General]
Domain = blue-balloon
Local-Realms = WILLOWBARRACO.FR,BLUE-BALLOON
[Translation]
Method = static,nsswitch
[Static]
stacy@WILLOWBARRACO.FR = willow

"blue-balloon" is my system hostname here. I use "static,nsswitch", and this static rule, because on some machine I still got a "stacy" username. My NFS folder is owned by "willow", so I have to map this manually to allow the "stacy" users to read/write. Every client "willow" users would get mapped to the machine "willow" user by the nsswitch, and the "Local-Realms" value.

Create principal for the NFS server #

The "nfs/..." format is important. "ktadd" store keys on the "/etc/krb5.keytab" file.

$ kadmin.local
add_principal -randkey nfs/music.willowbarraco.fr
ktadd nfs/music.willowbarraco.fr

Create end users. Those are the usernames you use on your client machines. Ideally you use the same usernames as you NFS folder, but you can map them manually, as seen previously when configuring idmap.

$ kadmin.local
add_principal willow
add_principal stacy

(some of my machine still use "stacy" as username, seen previoulsy).

If you plan to automatically generate user tickets, using pam by example, you have to use the same password as you machine user ones.

If you reached this, everything seems done for the server part.

$ rc-service nfs start
$ rc-update add nfs

Client config

Those are the only steps you have to run on your client machines.

$ apk add nfs-utils krb5

Add this.

# /etc/krb5.conf
[libdefaults]
 default_realm = WILLOWBARRACO.FR
[realms]
 WILLOWBARRACO.FR = {
  kdc = kerberos.willowbarraco.fr
  admin_server = kerberos.willowbarraco.fr
 }

Create and store this host machine principal. Replace "$HOSTNAME" with your machine hostname. The "host/..." format is important.

$ doas kadmin -p admin/admin
add_principal -randkey host/$HOSTNAME
ktadd host/$HOSTNAME

Add this.

# /etc/fstab
music.willowbarraco.fr:/home/willow/music	/home/willow/music	nfs4	rw,fsc	0	0

You can optionnally configure fs-cache editing "/etc/cachefilesd.conf" if needed.

Make the "nfsmount" service depending on "rpc.gssd" by adding.

# /etc/conf.d/nfsmount
rc_need="rpc.gssd"

Then everything should be fine for the client services.

$ doas rc-service cachefilesd start
$ doas rc-service nfsmount start
$ doas rc-update add cachefilesd
$ doas rc-update add nfsmount

Your system user will still not be able to access the mounted directory. This is because you have to generate tickets for your user too.

$ kinit
Password for stacy@WILLOWBARRACO.FR

Or you can use pam for this. For example, you can generate tickets while unlocking your sessions with swaylock. Of course your session passwords must match your realms ones for that.

$ apk add pam-krb5

Add this.

# /etc/pam.d/swaylock
# Unlock krb5 session
-auth            sufficient      pam_krb5.so minimum_uid=1000

At the end, you expect to list those tickets.

$ ls /tmp/ | grep krb
krb5cc_0
krb5cc_1000
krb5ccmachine_WILLOWBARRACO.FR

The "machine" one is generated by "spc.gssd" when "nfs" mounts the folders. The "0" one by your "doas kadmin -p admin/admin". And the "1000" one by your pam trigger, or "kinit" command. Your user ticket expires after 24 hours, so you will have to regenerate one regularly.


It is possible I missed something. If you struggle while testing, or if found a problem, please contact me!

RSS feed

If this post inspired you, feels free to leave a comment!

Reach me