🟢 How does PyTorch inside it use the host's GPU?
You need to understand what is happening on the host side
The NVIDIA driver in the kernel provides GPU access through device files: /dev/nvidia0, /dev/nvidiactl, and so on
Any application communicates with the GPU exactly through these device files
PyTorch does not directly access the driver
It works through CUDA Runtime (libcudart.so) which is a high-level API that handles allocations, kernel launches, and synchronization
This runtime library is inside your container
The entire stack looks like this:
PyTorch → CUDA Runtime → CUDA Driver → /dev/nvidia0 → kernel → GPU
Runtime lives in the container
Driver lives on the host
🔵 How do they connect?
Look at the container launch:
Containerd → containerd-shim → OCI-runtime (runc) → container
But if the driver is on the host and the runtime is in the container, how does the application access everything at once?
Answer: OCI hooks
The OCI specification defines hooks (code) that runs at different stages of the container's lifecycle:
prestart/createRuntime
createContainer
startContainer
poststart
poststop
NVIDIA uses these hooks to add GPU support
Before the container starts, the hook does the following:
1. Mounts GPU devices (/dev/nvidia*)
2. Places driver libraries from the host into the container
3. Sets the necessary environment variables
4. Configures device cgroups
Your application has not even started yet
All of this is handled by NVIDIA Container Toolkit. It intercepts container creation and carefully inserts everything needed for GPU operation
Your image remains normal.
GPU capabilities appear at runtime.
••••••••••••••••••••••••••••••••••••
🤖 Data Science, ML & Big Data with @DataXplore