The Art of the LXC Config: Right-Sizing a Container for LLM Deployment

In the sprawling, multi-host infrastructure of an AI deployment pipeline, few moments are as deceptively simple as the one captured in message [msg 6777]. On its surface, it is a single bash command that writes a new LXC container configuration file — a routine administrative task. But beneath this veneer of simplicity lies a dense knot of engineering decisions, each carrying implications for performance, stability, and resource efficiency. This message is the culmination of a hardware migration, the resolution of a device-number mismatch, and a deliberate recalibration of resource allocation for hosting a 27-billion-parameter language model.

The Message

Here is the message in full:

[assistant] 104 cores, 1TB RAM. Let me adjust CT129 — give it some sensible cores (e.g. 32), keep memory at something reasonable for a 27B model, and fix the device numbers.

>

``bash ssh root@10.1.2.5 'cat > /etc/pve/lxc/129.conf << '\''EOF'\'' arch: amd64 cores: 32 features: nesting=1 hostname: llm-two memory: 65536 net0: name=eth0,bridge=nv,firewall=1,hwaddr=BC:24:11:82:2D:E3,ip=dhcp,type=veth ostype: ubuntu rootfs: archive:vm-129-disk-0,size=800G swap: 0 unprivileged: 0 lxc.cgroup2.devices.allow: c 195:* rwm lxc.cgroup2.devices.allow: c 502:* rwm lxc.cgroup2.devices.allow: c 505:* rwm lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file lxc.mount.entry: /dev/nvidia1 dev/nvidia1 none bind,optional,create=file lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file lxc.mount.entry: /dev/nvidia-caps dev/nvidia-caps none bind,optional,create=dir EOF cat /etc/pve/lxc/129.conf' ``

The Context: Why This Message Was Written

To understand this message, one must understand the migration that precipitated it. The assistant was in the process of moving the deployment of Qwen3.6-27B — a 27-billion-parameter dense language model — from a host called kpro6 to a new host called kpro5. The old host had been decommissioned, and the container CT129 (named llm-two) had previously been configured for a 4-GPU setup on kpro6, with device entries for nvidia0 through nvidia3. The new host, kpro5, had a completely different hardware configuration: two NVIDIA RTX A6000 GPUs (Ampere architecture, 48GB each), 104 CPU cores, and 1 terabyte of RAM.

The preceding messages in the conversation document the entire setup process. The assistant had installed the NVIDIA driver version 580.126.09 on the Proxmox host, blacklisted the open-source nouveau driver, unbounded the two A6000s from the vfio-pci driver (which had been binding them for VM passthrough), and verified that nvidia-smi correctly detected both GPUs. The device nodes — /dev/nvidia0, /dev/nvidia1, /dev/nvidiactl, /dev/nvidia-uvm, and /dev/nvidia-uvm-tools — were confirmed to exist with their correct major device numbers (195 for the control device, 502 for the UVM device, and 505 for the NVIDIA capabilities device).

But the container configuration still reflected the old reality. It listed mount entries for nvidia2 and nvidia3 — devices that no longer existed. It specified 128 cores, a relic from a different machine with a different CPU topology. It used cgroup device allow rules with major numbers (509, 237, 238 from earlier inspection) that corresponded to the old kpro6 driver version, not the new 580.126.09 driver. And it allocated 260 GB of memory — far more than the 55 GB that a 27B BF16 model actually requires, but perhaps appropriate for the previous workload.

This message is the moment when the assistant reconciles the container configuration with the new hardware reality. It is a classic infrastructure engineering task: update the configuration to match the deployed environment, neither over-provisioning nor under-provisioning, and ensuring that every device the container needs is correctly exposed.

The Decisions Embedded in the Configuration

Core Count: From 128 to 32

The assistant's choice of 32 cores is a study in pragmatic resource allocation. The host has 104 cores and is already running multiple other virtual machines — VMs 114 (cloudstack-root, 200 GB RAM), 122 (fgw-qa-head), 123 and 124 (fgw-ribs1 and fgw-ribs2), 125 and 128 (which use the RTX 3090 GPUs), and others. The host is not dedicated to this single container. Setting cores to 128 would have been greedy and potentially destabilizing, allowing the container to attempt to monopolize CPU resources that other VMs need. Setting cores to 32 provides ample CPU headroom for a 27B model's inference workload — which is primarily GPU-bound during decode — while leaving 72 cores for the host and other VMs. The assistant's comment "give it some sensible cores (e.g. 32)" reflects a heuristic judgment: 32 cores is a reasonable upper bound for a 2-GPU inference container, unlikely to be a bottleneck but also unlikely to starve the rest of the system.

Memory: 64 GB for a 27B Model

The choice of 65536 MB (64 GB) is directly tied to the model being deployed. Qwen3.6-27B in BF16 precision requires approximately 55 GB of GPU memory for the weights alone (27 billion parameters × 2 bytes per parameter = 54 GB). With 2 × 48 GB A6000 GPUs, the model fits with some headroom for KV cache and activations. The container memory allocation of 64 GB is sufficient for the host-side overhead — loading the model, running the inference server (SGLang or vLLM), and handling any CPU-side processing — without wasting resources. The old allocation of 260 GB was clearly designed for a different workload, perhaps a larger model or multi-model serving. The assistant's reasoning is explicit: "keep memory at something reasonable for a 27B model."

Device Numbers: The Hidden Trap of Driver Upgrades

The most technically subtle decision in this message is the correction of the cgroup device allow rules. LXC containers on Proxmox use cgroup v2 device access control to grant the container permission to interact with hardware devices. Each device is identified by its type (c for character device) and major number. The major numbers for NVIDIA devices are not fixed — they are assigned by the kernel module at load time and can vary between driver versions and kernel configurations.

On the old kpro6 host, the NVIDIA driver had assigned major numbers 509, 237, and 238 to the various NVIDIA device classes. On kpro5 with driver 580.126.09, the numbers are 195 (nvidia front-end devices), 502 (nvidia-uvm), and 505 (nvidia-caps). If the assistant had simply copied the old config without updating these numbers, the container would have been unable to access the GPUs — c 509:* rwm would match no actual device on the new host, and the container would fail with "permission denied" errors when trying to open /dev/nvidia0.

The assistant's verification of the device nodes in the preceding message ([msg 6774]) was the critical step that enabled this correction. By running ls -la /dev/nvidia*, the assistant captured the exact major numbers in use and could write the correct cgroup rules. This is a classic example of the principle "verify, don't assume" in systems engineering.

The New Mount Entries

Compared to the old configuration (shown in [msg 6775]), the new config adds two mount entries that were previously absent: /dev/nvidia-uvm-tools and /dev/nvidia-caps. The old config had only nvidia0, nvidia1, nvidia2, nvidia3, nvidiactl, and nvidia-uvm. The addition of nvidia-uvm-tools and nvidia-caps suggests either that the assistant learned from previous deployment issues that these devices are needed for certain operations (e.g., Unified Memory management tools, GPU capabilities queries), or that the new driver version exposes these devices and the container needs access to them for full functionality. The nvidia-caps entry uses create=dir instead of create=file, correctly reflecting that it is a directory of capability files rather than a single device node.

Assumptions and Potential Issues

The message makes several assumptions worth examining. First, the assistant assumes that 32 cores is sufficient for the inference workload. For a single 27B model serving requests, this is almost certainly true — the GPU does the heavy lifting. However, if the container is later used for training or fine-tuning, 32 cores might become a bottleneck for data loading and preprocessing. Second, the assistant assumes that the device major numbers (195, 502, 505) will remain stable across reboots. While Linux kernel module major numbers are typically stable for a given driver build, a kernel update or driver reinstallation could change them, requiring this config to be updated again. Third, the assistant assumes that the container's root filesystem (archive:vm-129-disk-0,size=800G) is still valid on the new host — that the ZFS archive or storage location is accessible from kpro5. If the container's disk was on local storage on kpro6, this could be a problem.

One potential oversight is the lack of CPU pinning or NUMA awareness in the configuration. With 104 cores across what is likely a multi-socket system, the container's 32 cores could be scheduled across NUMA domains suboptimally, potentially increasing memory latency for GPU DMA operations. A production configuration might include lxc.cgroup2.cpuset.cpus to pin the container to cores on the same NUMA node as the GPUs. However, for a first migration step, getting the container running is the priority; NUMA optimization can follow.

Knowledge Required and Knowledge Created

To fully understand this message, one needs knowledge of: Proxmox LXC container configuration syntax; the Linux device number system for character devices; NVIDIA GPU driver architecture (the distinction between /dev/nvidia[0-N], /dev/nvidiactl, /dev/nvidia-uvm, and /dev/nvidia-caps); the memory requirements of LLMs in BF16 precision; and the resource constraints of the host machine (other running VMs, total core count, total memory).

The message creates actionable knowledge: a corrected, production-ready LXC configuration for the CT129 container on kpro5. This configuration encodes the specific hardware topology (2 GPUs, 32 cores, 64 GB RAM) and the specific driver version's device numbers. It serves as a record of the deployment's resource baseline and can be diffed against future configurations to track infrastructure changes.

The Thinking Process

The assistant's reasoning, visible in the message's opening line, follows a clear pattern: assess the available resources ("104 cores, 1TB RAM"), identify the task ("adjust CT129"), and articulate the decision criteria ("sensible cores", "reasonable for a 27B model", "fix the device numbers"). The bash command itself is carefully constructed — using a heredoc to write the file, escaping the EOF delimiter properly for the nested SSH command, and then echoing the file back with cat for verification. The assistant does not simply overwrite the old config; it rebuilds it from scratch, ensuring that no stale entries (like nvidia2 or nvidia3) survive. This is a deliberate choice: incremental editing risks leaving orphaned lines, while a full rewrite guarantees a clean state.

In the broader arc of the conversation, this message represents a transition point. The hardware is set up, the driver is installed, the GPUs are unbound from vfio-pci, and now the container is configured. The next steps — starting the container, installing userspace libraries, downloading the model, and deploying the inference server — will build on this foundation. But without this correctly configured LXC file, none of those steps would work. The container would fail to start, or would start without GPU access, or would crash with permission errors. This message, for all its apparent simplicity, is the keystone of the entire deployment.