Fork: 0
Skyworks / skyworks-Nix-infra
Transfer to URL with SHA Find file
Newer
Older
skyworks-Nix-infra / hosts / skydick / DATAPOOL.md
@Dixiao-L Dixiao-L 17 days ago 17 KB skydick: document datapool user and admin workflow
Raw Blame History

Skydick Data Pool — User & Service Guide

Server: skydick (10.0.1.1), pool: dick

Network: 10GbE bonded (bond0), jumbo frames MTU 9000, subnet 10.0.0.0/16.

Available shares

Share Server path SMB name NFS mount Access
Public files /srv/public \\SKYDICK\public /public rw, all @storage users
Media library /srv/media/library \\SKYDICK\media /media/library ro, all @storage users
Personal files /srv/users/<you>/files \\SKYDICK\<you> /users/<you> rw, SMB owner-authenticated; NFS network-trusted all_squash to owner

NFS paths are relative to the NFSv4 pseudo-root (/srv on the server, exported with fsid=0).

The final storage layout is live on skydick:

  • dick/public mounted at /srv/public
  • dick/media mounted at /srv/media, with data/ and library/ directories in one hardlink domain
  • dick/users/<user>/{files,bt-state,vm} for per-user private data
  • dick/system/{backup,vm} for centrally managed system storage
  • dick/templates/vm for shared read-only VM base images

The old dick/{share,backup,torrent,vm} layout is no longer part of the design. Torrent payload now
lives under /srv/media/data, and organized media under /srv/media/library.

Identity and authentication

  • skydick resolves POSIX users and groups from LDAP at ldap://10.0.0.1/, base
    dc=skyw,dc=top
  • SMB now uses Samba's LDAP passdb (ldapsam) against the same directory tree
  • On this standalone server, the Samba account-domain object is expected to be
    sambaDomainName=SKYDICK, matching the NetBIOS name, not the browse workgroup WORKGROUP
  • Shared public / media access is carried by the LDAP posixGroup
    cn=storage,ou=posix_groups,dc=skyw,dc=top with gidNumber: 997
  • NFS still does not authenticate users; it trusts client IPs and export options
  • LDAP remains the password source of truth. After a user's SMB access is bootstrapped once,
    password changes should happen through the LDAP password-change flow (the password web UI or
    ldappasswd), which keeps Samba's sambaSamAccount password data aligned.

Current admin users on skydick intentionally use the same canonical usernames as their LDAP
identities, for example ye-lw21. In those collisions, local NSS lookup still wins for the final
Unix UID/GID/group resolution on the server, while SMB password data still comes from LDAP.

The bootstrap LDIF for the Samba domain object, the LDAP storage group, and the machine OU is
checked in at samba-ldap-bootstrap.ldif.

Quick start

What a user needs before using the data pool

  • A real LDAP account under ou=people,dc=skyw,dc=top
  • For shared public and media access: membership in LDAP group
    cn=storage,ou=posix_groups,dc=skyw,dc=top
  • For private personal storage: a provisioned /srv/users/<you> subtree on skydick
  • For SMB: one-time admin bootstrap with smbpasswd -a <you>

After the one-time SMB bootstrap, your ongoing password is your LDAP password. Change it through
the LDAP password-change flow, not routine smbpasswd use.

What an admin must do before telling a user "it is ready"

  1. Create or verify the LDAP posixAccount for the user.
  2. Add the user's uid to LDAP group storage if they should access public and media.
  3. If the user needs private storage, provision the per-user Nix export/tmpfiles entries and the
    ZFS subtree under dick/users/<user>.
  4. Deploy the updated skydick NixOS configuration if step 3 changed it.
  5. If the user needs SMB, run smbpasswd -a <user> once on skydick.
  6. If the bootstrap password was admin-set, tell the user to immediately change it through the LDAP
    password-change flow.
  7. Verify getent passwd <user>, zfs list -r dick/users/<user> if applicable, and
    pdbedit -L | grep '^<user>:' if SMB was enabled.

Password rules

  • First SMB use: admin runs smbpasswd -a <user> on skydick once to create the user's
    sambaSamAccount in LDAP.
  • Normal password changes after that: use the LDAP password web UI or ldappasswd.
  • Do not use smbpasswd as the normal password-change workflow. It is for first-time SMB bootstrap
    and emergency repair only.

Example ldappasswd command for users who have LDAP CLI access:

ldappasswd -x -H ldap://10.0.0.1 \
  -D "uid=<you>,ou=people,dc=skyw,dc=top" \
  -W -S "uid=<you>,ou=people,dc=skyw,dc=top"

This changes the LDAP password, and the LDAP server keeps Samba password hashes aligned for users
that already have a sambaSamAccount.

Connecting via SMB (Windows / macOS / Linux GUI)

Windows

Open File Explorer, type in the address bar:

\\10.0.1.1\public
\\10.0.1.1\media
\\10.0.1.1\<your-username>

When prompted, enter your LDAP password after the one-time SMB bootstrap. For a user's first SMB
login, an admin must bootstrap the account once on skydick with smbpasswd -a <user>, which
creates the LDAP sambaSamAccount data for that user. After that, change passwords through the
LDAP password UI or ldappasswd so LDAP remains authoritative and SMB stays in sync.

macOS

Finder → Go → Connect to Server (Cmd+K):

smb://10.0.1.1/public
smb://10.0.1.1/media
smb://10.0.1.1/<your-username>

Linux (GUI)

Nautilus/Dolphin/Thunar address bar:

smb://10.0.1.1/public
smb://10.0.1.1/<your-username>

Linux (CLI / fstab)

# One-off mount
sudo mount -t cifs //10.0.1.1/public /mnt/public -o username=<you>,uid=$(id -u),gid=$(id -g)

# /etc/fstab (persistent) — store password in /root/.smbcredentials (chmod 600)
//10.0.1.1/public  /mnt/public  cifs  credentials=/root/.smbcredentials,uid=1000,gid=100,_netdev  0  0

/root/.smbcredentials:

username=<you>
password=<your-smb-password>

Connecting via NFS (Linux)

NFS uses NFSv4 with a pseudo-root at /srv. Mount paths omit /srv.

One-off mount

# Public shared files
sudo mount -t nfs4 10.0.1.1:/public /mnt/public

# Media library (read-only)
sudo mount -t nfs4 10.0.1.1:/media/library /mnt/media

# Your private tree (all writes become your UID via all_squash)
sudo mount -t nfs4 10.0.1.1:/users/<you> /mnt/skydick-home

Persistent mount (/etc/fstab)

10.0.1.1:/public         /mnt/public        nfs4  rw,hard,_netdev                         0  0
10.0.1.1:/media/library  /mnt/media         nfs4  ro,hard,_netdev                         0  0
10.0.1.1:/users/<you>    /mnt/skydick-home  nfs4  rw,hard,_netdev                         0  0

Performance tuning (10GbE)

For large transfers on 10GbE with jumbo frames, add NFS mount options:

rsize=1048576,wsize=1048576,nconnect=16

Example:

10.0.1.1:/users/ldx  /mnt/skydick  nfs4  rw,hard,rsize=1048576,wsize=1048576,nconnect=16,_netdev  0  0

Share details

Public (/srv/public)

Collaborative shared space. All users in the storage group can read and write. New files
inherit group storage via setgid (mode 2775).

  • SMB: read-write for @storage
  • NFS: read-write with root_squash (root maps to nobody, normal UIDs pass through)

Shared access is governed by LDAP membership in cn=storage,ou=posix_groups,dc=skyw,dc=top.
skydick also keeps a local storage group at GID 997 so on-disk ownership, service accounts,
and same-name local admin overlays stay stable.

Media library (/srv/media/library)

Read-only organized media (movies, TV, music). Managed by the automation stack (qBittorrent +
Sonarr/Radarr/Lidarr). Users consume but do not write here.

  • SMB: read-only for @storage
  • NFS: read-only via /media/library export

The full /srv/media dataset (including /srv/media/data with raw torrent payload) is only
writable by the qbittorrent service account (UID 900). Hardlinks between data/ and
library/ work because they are directories on the same ZFS dataset.

Personal files (/srv/users/<you>/files)

Private per-user storage. Only you can access your tree.

  • SMB: Samba [homes] share — connect as \\SKYDICK\<you>, authenticates with your Samba password
  • SMB bootstrap: one-time smbpasswd -a <you> on skydick creates your sambaSamAccount
  • SMB password changes after bootstrap: use the LDAP password UI or ldappasswd
  • NFS: /users/<you> export with all_squash mapping all operations to your UID/GID

Your NFS export maps every client UID to your server-side UID. This means any process on any
host in 10.0.0.0/16 that mounts your export will write as you. NFS does not authenticate —
it trusts the network. For stronger isolation, use SMB (which requires a password).

First-time user checklist

  1. Confirm your LDAP username with the admin.
  2. Confirm whether you should have only a private home, or also shared public and media
    access via LDAP group storage.
  3. If you will use SMB, ask the admin whether your first-time SMB bootstrap has been done.
  4. If the admin set a temporary SMB password for bootstrap, change it immediately through the LDAP
    password-change flow.
  5. Connect with either SMB or NFS using the paths in this guide.

Per-user subtree layout

/srv/users/<you>/
├── files/          ← personal files (SMB [homes] points here)
├── bt-state/       ← private torrent/arr client state
│   ├── watch/      ← .torrent watch directory
│   ├── session/    ← client session/resume data
│   └── config/     ← client configuration
└── vm/
    └── files/      ← VM disk images (file-backed, NFS-visible by default)

bt-state holds your torrent client's configuration and state databases. The actual media
payload lives on the shared dick/media dataset, not here. This separation means:

  • No duplicate media storage across users
  • Your client state is private and independent
  • The shared media tree has one writer (the automation stack)

VM zvols (block devices for iSCSI) are created as ZFS children of dick/users/<you>/vm/<name>
and are managed by the admin. They are not visible in the filesystem tree.

Admin runbook

The workflow below is the full procedure for onboarding a new data-pool user.

Adding a new user

Admin procedure — run on skydick as root unless a step explicitly happens on the LDAP host:

1. Create or verify the user in LDAP

Preferred for storage-only users. The LDAP entry should already contain:

  • uid
  • uidNumber
  • gidNumber
  • homeDirectory
  • objectClass: posixAccount

If the user should see public and media, also add their LDAP uid as a memberUid of
cn=storage,ou=posix_groups,dc=skyw,dc=top.

Check it on skydick:

getent passwd <newuser>

2. Add a local NixOS user only if needed

Only do this if the user needs SSH login, sudo, or an intentional local override. If you do create
a same-name local admin user, remember that skydick will use the local Unix UID/GID for on-server
authorization while SMB passwords still come from LDAP.

In hosts/skydick/default.nix:

users.users.<newuser> = {
  extraGroups = [ "storage" ];
  hashedPassword = "<hash>";  # mkpasswd -m yescrypt
};

In modules/users.nix (if the user needs SSH/sudo access across all hosts):

users.users.<newuser> = {
  isNormalUser = true;
  extraGroups = [ "wheel" ];
  openssh.authorizedKeys.keys = [ "ssh-ed25519 ..." ];
};

3. Add per-user tmpfiles and NFS export

Use numeric UID/GID in tmpfiles rules for LDAP-only users. This avoids boot-time dependence on NSS
name resolution.

First get the IDs:

uid=$(getent passwd <newuser> | cut -d: -f3)
gid=$(getent passwd <newuser> | cut -d: -f4)

In hosts/skydick/datapool.nix, add to systemd.tmpfiles.rules:

"d /srv/users/<newuser> 0700 <UID> <GID> -"
"d /srv/users/<newuser>/files 0750 <UID> <GID> -"
"d /srv/users/<newuser>/bt-state 0750 <UID> <GID> -"
"d /srv/users/<newuser>/vm 0750 <UID> <GID> -"
"d /srv/users/<newuser>/vm/files 0750 <UID> <GID> -"

Add to services.nfs.server.exports:

/srv/users/<newuser>  10.0.0.0/16(rw,sync,no_subtree_check,all_squash,anonuid=<UID>,anongid=<GID>)

Replace <UID> and <GID> with the LDAP-backed numeric IDs from getent passwd.

Example: the user previously called ylw in local NixOS config is now canonicalized to
ye-lw21 everywhere, so the per-user share path is /srv/users/ye-lw21.

4. Deploy NixOS config

sudo git -C /etc/nixos pull && sudo nixos-rebuild switch --flake /etc/nixos

5. Create per-user ZFS datasets on skydick

The shared namespace datasets (dick/public, dick/media, dick/system, dick/templates, and
dick/users) already exist on the host. For a new user, create only that user's subtree:

# Get the user's UID/GID
uid=$(getent passwd <newuser> | cut -d: -f3)
gid=$(getent passwd <newuser> | cut -d: -f4)

# Create datasets
zfs create -o mountpoint=/srv/users/<newuser> -o quota=10T               dick/users/<newuser>
zfs create -o recordsize=128K -o mountpoint=/srv/users/<newuser>/files   dick/users/<newuser>/files
zfs create -o recordsize=16K  -o mountpoint=/srv/users/<newuser>/bt-state dick/users/<newuser>/bt-state
zfs create -o recordsize=64K  -o mountpoint=/srv/users/<newuser>/vm      dick/users/<newuser>/vm
mkdir -p /srv/users/<newuser>/vm/files

# Set ownership
chown "$uid:$gid" /srv/users/<newuser> && chmod 0700 /srv/users/<newuser>
for d in files bt-state vm vm/files; do
  chown "$uid:$gid" /srv/users/<newuser>/$d && chmod 0750 /srv/users/<newuser>/$d
done

6. Bootstrap SMB login if the user needs SMB

smbpasswd -a <newuser>

This one-time step is required even if the user already exists as a POSIX account in LDAP.
smbpasswd -a creates the user's sambaSamAccount attributes in LDAP and sets an initial SMB
password.

After this bootstrap, future password changes should happen through the LDAP password UI or
ldappasswd, not routine smbpasswd use. That keeps LDAP as the password source of truth while
the LDAP server updates the Samba password hashes.

The user can now connect via SMB and NFS.

7. Re-export NFS

exportfs -ra

8. Verify before handing over to the user

getent passwd <newuser>
id <newuser>
zfs list -r dick/users/<newuser>
exportfs -v | grep "/srv/users/<newuser>"

If SMB was enabled:

pdbedit -L | grep "^<newuser>:"

Then tell the user:

  • Their exact username
  • Whether they have public / media access or only the private home
  • Whether SMB bootstrap is done
  • To use the LDAP password-change flow for future password changes

Existing user enabling SMB for the first time

If a user already exists in LDAP and already has the private dataset/export, but has never used SMB
before, only this step is needed:

smbpasswd -a <user>

After that, the user should change the password through the LDAP password-change flow if the admin
set the initial value.

Quotas

When a user's dick/users/<user> dataset exists, its ZFS quota (default 10TB in the examples
above) caps the total across all child datasets (files + bt-state + vm). Check usage:

zfs list -o name,used,quota -r dick/users

Adjust quota:

zfs set quota=20T dick/users/<user>

The shared dick/media dataset has no per-user quota — it is managed at the service level.

Monitoring

Check pool health:

zpool status dick

Check dataset usage:

zfs list -o name,used,avail,refer,quota -r dick

Check NFS exports:

exportfs -v

Check active NFS clients:

ss -tn | grep :2049

Troubleshooting

"Permission denied" on NFS mount

  • Verify your IP is in 10.0.0.0/16: ip addr
  • Check the export exists: showmount -e 10.0.1.1
  • Per-user exports use all_squash — your local UID does not matter, everything maps to
    the server-side owner UID

"Permission denied" writing to NFS

  • Public: your server-side UID must be in the storage group
  • Media: read-only for all users (write is via the automation service account only)
  • Personal: should always work — if not, check that the ZFS datasets are mounted:
    ssh skydick zfs list -r dick/users/<you>

SMB authentication fails

  • Samba uses LDAP-backed sambaSamAccount entries for SMB auth, not just the Unix userPassword
  • getent passwd <user> succeeding only proves Unix account lookup works; it does not create an SMB login
  • If the user has never used SMB before, admin must run smbpasswd -a <user> once on skydick to
    bootstrap the sambaSamAccount
  • After bootstrap, change the password through the LDAP password UI or ldappasswd so Unix and
    SMB passwords stay aligned
  • If pdbedit -L shows the user but SMB still fails, reset with smbpasswd -a <user> and then
    have the user change the password again through the LDAP password-change flow
  • If public or media access fails but the home share works, check LDAP storage membership and
    verify the memberUid list for cn=storage,ou=posix_groups,dc=skyw,dc=top

Slow NFS transfers

  • Ensure MTU 9000 (jumbo frames) is set on both client and server interfaces
  • Add nconnect=16 to mount options for parallel NFS connections
  • Add rsize=1048576,wsize=1048576 for 1MB read/write blocks
  • Check link speed: ethtool <interface> | grep Speed

Files not showing up in media library

The arr stack (Sonarr/Radarr/Lidarr) hardlinks files from /srv/media/data/ to
/srv/media/library/. If a file exists in data/ but not in library/, the arr
import/rename has not run yet. Do not manually copy files into library/ — let the
automation stack manage it to preserve hardlinks and metadata.