numa-programming
/numa-programming
Use when detecting NUMA topology, binding processes with numactl, using the libnuma API, building NUMA-aware data structures, or measuring remote memory access penalties.
- Author
- OutlineDriven ↗
- Repository
- outline-driven-development ↗
- Licence
- Apache-2.0
- Last repository change
- 28.09.2026
Assessment and security
- Files in this package
- 2
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 28.09.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
What it does and when to use it
Use when detecting NUMA topology, binding processes with numactl, using the libnuma API, building NUMA-aware data structures, or measuring remote memory access penalties.
Original author description
Use when detecting NUMA topology, binding processes with numactl, using the libnuma API, building NUMA-aware data structures, or measuring remote memory access penalties.
How it works
- Contract
- Inputs
- Procedure
- Failure and recovery
- Output
SKILL.md preview
---
name: numa-programming
description: 'Use when detecting NUMA topology, binding processes with numactl, using the libnuma API, building NUMA-aware data structures, or measuring remote memory access penalties.'
---
# NUMA programming
## Contract
| Field | Bound contract |
|---|---|
| Trigger | NUMA-aware programming or debugging on multi-socket Linux systems. |
| Authority | Read-only. No source or remote mutation. Chat output only. |
| Side effect | Emits a structured guidance report to chat. |
| Done | The report shows the NUMA topology, the right `numactl` binding, libnuma API usage, a per-node data structure pattern, remote access diagnosis, and a decision tree. |
## Inputs
1. **Target system** (required): a multi-socket Linux host or VM with NUMA exposed.
2. **Workload symptom** (optional): poor scaling, memory bandwidth saturation, remote access, or unexpected page migration.
3. **Tools available** (optional): `numactl`, `numactl-dev`, `libnuma-dev`, `perf`, `lstopo`.
## Procedure
1. **Detect the NUMA topology.** Run `numactl --hardware`, `lstopo --of console`, and read `/sys/devices/system/node/node*/meminfo` and `node*/cpulist`. A distance value of 10 usually means local access; larger values indicate remote access. Done when: the node count, CPU lists, memory sizes, and distance matrix are reported.
Typical output from `numactl --hardware`:
```
available: 2 nodes (0-1)
node 0 cpus: 0-15
node 0 size: 65536 MB
node 1 cpus: 16-31
node 1 size: 65536 MB
node distances:
node 0 1
0: 10 21
1: 21 10
```
2. **Bind a process with `numactl`.** Use `--cpunodebind` and `--membind` for strict local placement, `--interleave=all` to spread memory, `--preferred` for a preferred node with fallback, and `--show` to inspect the current policy. Done when: the binding command and policy are selected.
```bash
numactl --cpunodebind=0 --membind=0 ./myapp
numactl --interleave=all ./myapp
numactl --preferred=0 ./myapp
numactl --show
```
3. **Use the libnuma API.** Call `numa_available`, `numa_node_of_cpu`, `numa_alloc_onnode`, `numa_alloc_local`, `mbind`, `set_mempolicy`, and `move_pages`. Compile with `-lnuma`. Done when: the C example and API table are shown.
```c
#include <numa.h>
#include <numaif.h>
#include <stdio.h>
int main(void) {
if (numa_available() < 0) {
fprintf(stderr, "NUMA not available\n");
return 1;
}
int node = numa_node_of_cpu(0);
printf("CPU 0 on node %d\n", node);
size_t size = 1024 * 1024 * 1024;
void *mem = numa_alloc_onnode(size, 0);
if (!mem) return 1;
unsigned long nodemask = 1UL << 0;
mbind(mem, size, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0);
numa_free(mem, size);
return 0;
}
```
```bash
gcc -o numa_test numa_test.c -lnuma
```
| API | Purpose |
|---|---|
| `numa_alloc_onnode` | Allocate on a specific node |
| `numa_alloc_local` | Allocate on the current CPU node |
| `mbind` | Set policy on an existing mapping |
| `set_mempolicy` | Default policy for later allocations |
| `move_pages` | Migrate pages to a target node |
4. **Build NUMA-aware data structures.** Create a per-node pool and allocate from the node of the current CPU. Pin threads to cores on the same node. Done when: the per-node allocation pattern is shown.
```c
#include <numa.h>
#include <sched.h>
#define BLOCK_SIZE 64
#define MAX_NODES 8
struct per_node_pool {
void *free_list[MAX_NODES];
int node_count;
};
void *pool_alloc_numa(struct per_node_pool *p) {
int node = numa_node_of_cpu(sched_getcpu());
void *blk = p->free_list[node];
if (blk) {
p->free_list[node] = *(void **)blk;
return blk;
}
return numa_alloc_onnode(BLOCK_SIZE, node);
}
```
5. **Align thread affinity.** Bind threads to CPUs on a node with `pthread_setaffinity_np` or `numactl --cpunodebind`. Match memory allocation to the same node. Done when: the affinity and memory binding match.
```c
#include <pthread.h>
#include <sched.h>
cpu_set_t cpuset;
CPU_ZERO(&cpuset);
CPU_SET(target_cpu, &cpuset);
pthread_setaffinity_np(pthread_self(), sizeof(cpuset), &cpuset);
```
6. **Diagnose remote access.** Run `perf stat -e cache-misses,cache-references,node-load-misses` with CPU and memory bindings on different nodes. Compare against local-local binding. Use `node-loads` and `node-load-misses` if the kernel exposes them. Done when: the remote access signature is identified.
```bash
perf stat -e cache-misses,cache-references,node-load-misses \
numactl --cpunodebind=0 --membind=1 ./myapp
perf stat numactl --cpunodebind=0 --membind=0 ./myapp
perf stat -e node-loads,node-load-misses,node-stores ./myapp
```
7. **Measure the remote access penalty.** Touch a large buffer on the local node and then on a remote node. Time the loops. Remote access can be 1.5x to 3x slower than local access on recent multi-socket x86/AMD64 systems with QPI/UPI/Infinity Fabric, depending on the workload and interconnect. Done when: the measurement method is given.
```c
clock_t start = clock();
for (size_t i = 0; i < size; i += 4096)
sum += ((char *)mem)[i];
```
8. **Visualize with `lstopo`.** Run `lstopo` for a graphical view, `lstopo --of ascii` for text, or `lstopo file.png` for an image. Done when: the visualization command is selected.
```bash
lstopo
lstopo --of ascii
lstopo file.png
```
9. **Walk the decision tree.** Check topology, verify CPU and memory are on the same node, measure `node-load-misses`, then choose `numactl --membind=local`, per-node partitioning, or memory bandwidth reduction. Done when: the next action is named.
```
Poor scaling on multi-socket?
├── Check numactl --hardware
├── Verify thread and memory on the same node
├── perf stat node-load-misses
├── Remote misses high?
│ ├── numactl --membind=local
│ └── Per-node data partitioning
└── Still slow: memory bandwidth bound; reduce sharing
```
## Failure and recovery
| Failure class | Behavior |
|---|---|
| OOM on one node despite free RAM elsewhere | `MPOL_BIND` is too strict. Use `--preferred` or `--interleave`. |
| 2x slower after scaling threads | Threads access remote memory. Bind memory to the same node as the CPU. |
| Inconsistent benchmark results | The OS migrated pages. Use `mbind` with `MPOL_BIND` or lock pages with `mlock` if needed. |
| DPDK NIC on the wrong socket | The PCI device is far from the CPU. Use `lstopo` and bind EAL to the local socket. |
| libnuma not found | The `libnuma-dev` package is missing. Install it with the system package manager. |
| First-touch policy surprise | Allocation happened on node 0 but the thread ran on node 1. Allocate from a bound thread. |
## Output
1. The NUMA topology and distance matrix.
2. A `numactl` binding command.
3. libnuma API calls for the use case.
4. A per-node data structure and affinity pattern.
5. A remote access measurement and a decision tree.
Files in this package (2)
- SKILL.md 7 KB
- agents/openai.yaml 1 KB
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · OutlineDriven/outline-driven-development | 100 | Apache-2.0 |
| mohitmishra786/low-level-dev-skills | 100 | MIT |
Database snapshot: 13.11.2026 · package version 593500e9dd17 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 90hardware-aware-design
Hardware-aware architecture for high performance: memory hierarchy & cache behavior, NUMA, SIMD/vectorization, mechanical sympathy, false sharing, and lock-free vs locked concurrency. Architect-level guidance on designing software that matches the machine. USE WHEN:… - 100rs-domain-systems
Use for low-level systems work in Rust — eBPF programs, GPU compute, io_uring and thread-per-core I/O, kernel-bypass networking, SIMD, memory-mapped files, zero-copy parsing, NUMA and CPU affinity. Triggers: eBPF, BPF, aya, XDP, kprobe, tracepoint, kernel module, verifier, GPU,… - 100feed-handler-cpu-pinning-and-numa-awareness
Use when a feed handler or execution loop shows latency jitter unrelated to market activity. Pins workers to chosen cores and confirms pages are resident on the local NUMA node, removing scheduler migration and cross-socket interconnect hops. - 100run-qemu
Use this skill for building Linux kernels and booting QEMU VMs with custom topologies (CXL, NUMA, NVDIMMs). Covers building, launching, testing inside VMs, and managing VM lifecycle via MCP tools or CLI. - 100c-systems-programming
Use when c systems programming including file I/O, processes, signals, and system calls for low-level system interaction.