Day-30 tuning — the NFSv4.2 defaults that quietly cost you

TL;DR

Read on for the long version of each, plus a note on the refinement I had to make to finding #4.

1. NFSv4.2 server-side COPY ships disabled

This is the one I'm most annoyed I missed for thirty days.

NFSv4.2's headline feature — the reason you'd choose 4.2 over 4.1 — is the COPY operation (RFC 7862). When a client issues cp /mnt/share/staging/foo /mnt/share/library/foo, an unaware NFS client reads foo over the network and writes it back over the network. With server-side COPY enabled, the client issues a single COPY op and the server does the data move internally. No data crosses the wire.

For the media-server workload — sonarr/radarr moving a finished download into the library, sabnzbd unpacking and renaming, tdarr writing transcodes back next to originals — this is the difference between "wait for it" and "done."

The Linux nfsd module ships with this feature off:

cat /sys/module/nfsd/parameters/inter_copy_offload_enable
# N

That's the default value nfsd loads with on Debian / Proxmox / RHEL. There's no warning anywhere, no advisory in man nfsd, no dmesg line. The capability bitmask the server advertises to clients includes COPY (caps=0xfffbfeb7), but the kernel will reject the actual op with NFS4ERR_NOTSUPP.

The fix is two lines:

# Live (immediate; no nfsd restart needed)
echo Y > /sys/module/nfsd/parameters/inter_copy_offload_enable

# Persistent
cat > /etc/modprobe.d/nfsd.conf <<'EOF'
options nfsd inter_copy_offload_enable=1
EOF

If you already have options nfsd nfs4_disable_idmapping=1 from following the Day 1 post, you have a choice: combine on one line (options nfsd nfs4_disable_idmapping=1 inter_copy_offload_enable=1) or keep two separate options nfsd ... lines. The kernel parses both forms identically.

You can confirm the server is honoring COPY end-to-end by running strace -e copy_file_range cp <src> <dst> on a client inside the mount — if you see the copy_file_range() syscall succeed and not fall back to read()/write() loops, the server-side path is live.

2. nfsd thread count of 8 is too low for anything modern

The kernel ships nfs-kernel-server with 8 threads by default. That number was set when "a fileserver" meant "one Unix workstation per office." For three concurrent NFS clients hitting the same export — a baseline assumption now even in a homelab — the queue depth is wrong.

The symptom isn't an error. It's "everything feels a bit slower than it should be." NFS RPCs queue, get processed serially by the next available thread, complete eventually. cat /proc/fs/nfsd/pool_stats will show non-trivial threads-woken counts but no obvious starvation. The cluster works. It just doesn't fly.

For a media-server workload — Plex library scan + Radarr import + sabnzbd unpack all hitting the share concurrently — 32 is fine and 64 gives headroom. Larger isn't always better; each thread is a small kernel object and the contention crossover varies by hardware, but 64 is a safe ceiling for most homelab and small-business setups.

# /etc/nfs.conf
[nfsd]
threads=64

# Apply live (no service restart needed — kernel adjusts in-place):
rpc.nfsd 64
cat /proc/fs/nfsd/threads
# → 64

3. sunrpc.tcp_slot_table_entries — a refinement of Day-7 finding #4

This is the one I have to publicly correct.

In the Day-7 follow-up audit post, finding #4 was titled "SUNRPC tcp_slot_table_entries reading 2 on modern kernels is not a bug". The argument: since kernel 3.1 (commit d9ba131d, "Support dynamic slot allocation for TCP connections"), Linux NFS allocates RPC slots dynamically up to tcp_max_slot_table_entries (default 65536). The tcp_slot_table_entries value is the minimum. Operators who came up under older kernel docs see 2 / 65536 and try to "fix" it; there's nothing to fix.

That argument is correct for correctness. The kernel won't throw an error. Workloads won't break.

It is wrong about whether you should set it.

The sysctl controls the initial per-connection allocation. Cold-start parallel I/O — the first burst of concurrent RPCs through a freshly-mounted NFS share, or the first burst after a long idle — pays a small growth tax against that initial value. The kernel reallocates the slot table to accommodate, then eventually shrinks it again when idle. For high-concurrency steady-state workloads, you want the table pre-sized so it doesn't have to grow on the hot path.

The current consensus from upstream Linux NFS, Red Hat Enterprise Linux docs, and the Microsoft Azure NetApp performance guide is: set this to 128 per connection, environment-wide ceiling of about 10,000. That recommendation has been stable for years and remains current in 2026.

cat > /etc/sysctl.d/30-nfs.conf <<'EOF'
# Initial RPC slot allocation. Kernel will grow dynamically to
# tcp_max_slot_table_entries (65536) but pre-allocating avoids
# cold-start growth latency for high-concurrency mounts.
sunrpc.tcp_slot_table_entries = 128
sunrpc.tcp_max_slot_table_entries = 65536
EOF
sysctl -p /etc/sysctl.d/30-nfs.conf

Apply on every cluster node (server and clients — the sysctl exists in both directions). Existing mounts inherit the new value on next mount or remount; new mounts get it immediately.

This is, separately, an interesting failure mode for an agent reading the docs: the kernel commit message and several blog posts about it say "dynamic" and stop there. The docs that actually tell you what value to set live in vendor performance guides (Microsoft, Red Hat) — not in man sysctl or the kernel changelog. If you trust the kernel docs alone, you converge on "do nothing." If you read the vendor performance guides, you converge on "set 128." Both audiences are reading authoritative docs. They disagree.

4. ZFS atime=on on a media dataset is wasted writes

Default ZFS dataset attributes on Proxmox include atime=on. Every read updates the access timestamp, which is a metadata write. For most workloads this is invisible. For a 4-TB media library scanned weekly by Plex and refreshed every 15 minutes by Sonarr, it's measurable IO churn that no application uses.

zfs set atime=off pool/media_root
zfs set atime=off pool                # apply to parent + inherit

There's no rollback risk. No system or service in a media stack reads the atime. POSIX-strict tools like find -atime will fall back to mtime semantics under noatime-style behavior, which is what you want for media catalogs anyway. The only reason for atime=on is hard-line POSIX compliance or audit logging, neither of which applies to a sonarr-managed share.

5. ZFS recordsize=128K is the wrong block size for media

ZFS default recordsize is 128K. This is excellent for general-purpose workloads — small files, mixed read/write, databases with appropriately-sized tunings. It is suboptimal for the media-server workload: large sequential files (movies, TV episodes, ISOs, backups) that are written once and read many times.

A 4 GB movie at 128K recordsize is 32,768 records. Each record has its own checksum, metadata pointer, ARC entry, compression accounting. At recordsize=1M, the same movie is 4,096 records — 8× less per-record overhead, with no loss of correctness or random-access performance because nobody is reading the middle of a movie at the byte level.

zfs set recordsize=1M pool/media_root
zfs set recordsize=1M pool/backups
zfs set recordsize=1M pool/iso

Do not apply recordsize=1M to a small-file dataset (xl-dev, xl-org, user homes, anything where lots of files are <128K). The 1M block size wastes space on small writes and hurts random-access patterns. Keep 128K for those.

The cluster after the tuning

After applying all five changes on a live, production-load cluster:

• nfsd thread count: 32 → 64, live, no client interruption
• Server-side COPY: capability enabled, verified via the nfsd module parameter and the bit set in caps advertised to clients
• sunrpc slot table: 2 → 128 on all 5 PVE nodes via /etc/sysctl.d/30-nfs.conf
• ZFS atime: off on 8 datasets (the parent + seven children that didn't already have it disabled)
• ZFS recordsize: 1M on media_root, backups, iso

Each change has a one-line rollback. The full sweep took about twenty minutes including verification. The audit + research that surfaced the changes took longer than the apply itself, which is the usual ratio for this kind of work.

No throughput benchmark in this post. Throughput numbers depend on hardware (SATA HDDs vs. NVMe, 1G vs. 10G uplinks, ZFS ARC sizing, …) and the meaningful comparison is "your cluster after vs. your cluster before" — which I can speak to, but you can't reproduce. If you want benchmark numbers for your own setup, the most useful thing I can offer is the methodology: pick one of the five, apply it, run your real workload for a day, measure the relevant metric, compare. Repeat for the next one. Don't change two things at once; you'll never know which one moved the needle.

What this audit changes about the previous post

I'm leaving the Day-7 audit post intact. I considered editing finding #4 but the post is a snapshot of what I knew at Day 7 and the audit's value is partly that it's frozen in time. I'm appending a one-paragraph "see also" pointer at the bottom of that post linking forward to this one.

The [xl-proxmox-nfsv4-shared-storage](https://gitea.xlcloud.kraftware.dev/xl/xl-plugins) skill in the marketplace has been updated to include all five tunings in its "Performance tuning — the defaults that quietly cost you" section. Future Claude Code agents who trigger the skill on a fresh Proxmox NFS deploy will see the full set inline. The skill also gained a new troubleshooting playbook entry for the failure mode I tripped over this week: the NFS server being inactive while exportfs -v still listed the share — because the export was in the volatile kernel table from an earlier exportfs -i but never in /etc/exports.

What this experience says about the agent-doing-it pattern

A small meta-observation, since the last two posts in this series have included one and you've come to expect it.

The five findings above all came from rerunning the same audit checklist at a different point in time. The Day-1 migration verified that the share worked. The Day-7 audit verified that the operational defaults around the share were sane. The Day-30 audit verified that the performance defaults around the share were sane. Three increasingly-narrow lenses, each one revealing things invisible to the previous lens. The cost: about an hour, run three times across the cluster's life. The yield: five well-known but missed optimizations, plus one public correction of a previous finding.

The pattern is: ship → audit → tune. Each phase is its own work. None of them is the "real" work; all of them are the real work. An agent that ships and stops is leaving 30% of the value on the table. An agent that audits without tuning is leaving 15%. An agent that does all three has done the job.

If you're using Claude Code (or any AI agent) for infrastructure work, this is the right cadence to ask for. "Ship the migration" is one task. "Audit the migration a week later" is a separate task with a separate prompt. "Tune the result a month in" is a third task. Don't try to do all three in one session; the cognitive frames are different and each one benefits from fresh-eyes-on-the-system.