Also, the new feature (v1.25+) uses hard links to preserve container state before migration. Conclusion: The Link is the Lost Art of Container Storage The CRI file system tools — crictl , ctr , crio-status —give you x-ray vision into how Kubernetes manages storage. But without understanding the link (whether symbolic, hard, or the conceptual parent pointer between layers), you are blind to half of the system.
# Find snapshot path SNAPSHOT_PATH=$(crictl inspect <container> | jq -r '.info.rootDir') cp -al $SNAPSHOT_PATH /tmp/clone-rootfs Now modify /tmp/clone-rootfs without affecting the original (COW at file level) cri file system tools link
ctr -n k8s.io snapshot rm <snapshot-key> ctr -n k8s.io snapshot gc # Garbage collects unlinked snapshots Check /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/metadata.db (a BoltDB file) for orphaned links. Tools like boltdb-viewer can inspect it. Scenario 3: Migrating container rootfs to another disk using symlinks Suppose your /var/lib/containerd partition is full. You can move the storage directory and create a symbolic link. Also, the new feature (v1
ctr namespace ls # List namespaces (e.g., k8s.io) ctr -n k8s.io snapshot ls # Show all snapshots (image layers) ctr -n k8s.io snapshot mount <key> /mnt # Mount a snapshot to inspect Snapshots are immutable directories linked together via overlayfs. Each snapshot has a "parent" link to the previous layer. 3. crio-status – CRI-O’s Inspection Tool For CRI-O users, crio-status dumps storage and runtime information. You can move the storage directory and create
# Get container PID crictl inspect <container> | grep pid nsenter -t <pid> -m bash Inside, check for broken symlinks find / -type l -xtype l 2>/dev/null