cpu-kernel-authoring
/cpu-kernel-authoring
Use when writing, optimizing, or benchmarking a C++ CPU kernel with AVX2 or AVX512 intrinsics for the Hugging Face kernels ecosystem. Not for CUDA kernels.
- Author
- gosuda ↗
- Repository
- dalcom-skills ↗
- Licence
- Apache-2.0
- Last repository change
- 10.11.2026
Assessment and security
- Files in this package
- 23
- Scripts
- 6
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 10.11.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
- 6 scripts in this package. Review them before installing, as Claude may execute them.
Scripts: scripts/analyze_op.py, scripts/benchmark_cpu.py, scripts/config.py, scripts/cpu_profiler.py, scripts/trial_manager.py, scripts/validate_cpu_kernel.py
What it does and when to use it
Use when writing, optimizing, or benchmarking a C++ CPU kernel with AVX2 or AVX512 intrinsics for the Hugging Face kernels ecosystem. Not for CUDA kernels.
Original author description
Use when writing, optimizing, or benchmarking a C++ CPU kernel with AVX2 or AVX512 intrinsics for the Hugging Face kernels ecosystem. Not for CUDA kernels.
How it works
- What you need, and what you may touch
- Work in this order
- What you hand back
SKILL.md preview
--- name: cpu-kernel-authoring description: 'Use when writing, optimizing, or benchmarking a C++ CPU kernel with AVX2 or AVX512 intrinsics for the Hugging Face kernels ecosystem. Not for CUDA kernels.' disable-model-invocation: true --- # CPU kernel authoring You write, optimize, or benchmark a C++ CPU kernel with AVX2 or AVX512 intrinsics for the Hugging Face kernels ecosystem, measured against a PyTorch baseline. This skill is not for CUDA kernels. The work has two phases. The correctness phase builds the tiers in order (generic ATen fallback, optional AVX2, AVX512), and each tier must pass correctness before the next starts. The performance phase iterates on the AVX512 tier through the trial tree until `max_trials` is exhausted or `early_stop_speedup` is exceeded. You are done when the kernel passes the correctness check in `scripts/benchmark_cpu.py`, every trial up to `max_trials` has run or the speedup exceeded `early_stop_speedup`, and the best trial is finalized into `output/` with its final measurement. Otherwise you report the failure class and the recovery step taken. You write locally and reversibly: C++ kernel sources, `build.toml`, and `torch_binding.cpp` under the kernel directory, a wheel under `dist/`, the installed kernel package in the active Python environment, and trial state under `trials/<kernel_name>/` and `output/`. Rollback is version control for the sources, `pip uninstall <package>` for the package, and removal of `dist/`, `trials/<kernel_name>/`, and `output/`. No remote mutation. ## What you need, and what you may touch Required inputs: - Kernel name: the trial-tree label, for example `my_rmsnorm`. `trial_manager.py` accepts it only as a single directory name under `trials/`, never a path. - Baseline file: a `baseline.py` defining `get_inputs()` and either `get_reference_output()` or a `Model` class (with optional `get_init_inputs()`). It is the ground truth for correctness and the speed reference. - Operation name: the plain name `analyze_op.py --op` looks up, for example `rms_norm`. - Input shapes: comma-separated shape strings for `analyze_op.py --shapes`, for example `"1024x4096,2048x8192"`. - Package and function path (needed from step 5 on): the installed package name, for example `my_kernel`, and its callable as `package.function`, for example `my_kernel.rms_norm`. `benchmark_cpu.py` and `cpu_profiler.py` take this path as their `--op`; it is not the operation name above. - Toolchain: Python 3.11 or later (`validate_cpu_kernel.py` parses `build.toml` with the standard-library `tomllib`), `kernel-builder`, `pip`, PyYAML (imported by `scripts/config.py`), `numactl` (used by the pinned benchmark in step 8), a C++ compiler with AVX512 support, and PyTorch. `perf` is required only when `perf_stat_enabled` is true. Modify only `.cpp` and `.hpp` files, `torch_binding.cpp`, and `build.toml`. Do not write new benchmark or timing scripts; `scripts/benchmark_cpu.py` is the only timing source. When a script fails, report the error rather than working around it. ## Work in this order 1. Read `scripts/config.yaml` and note `max_trials`, `early_stop_speedup`, `perf_stat_enabled`, `vtune_enabled`, `build_command`, and `install_command`. Use those two commands wherever this procedure builds or installs. If the config file is missing, stop and report; never assume trial counts. 2. Run `python scripts/analyze_op.py --op <op_name> --shapes <shapes>` and read the compute and memory characteristics and the suggested SIMD strategy. Read `references/workflow_details.md` for the analysis and design steps. The script recognizes norm, softmax, gemm, linear, matmul, attention, gelu, silu, relu, moe, and megablocks by name and classifies anything else as plain element-wise; when it reports no matmul, reduction, or activation for your op, classify the kernel type by hand from the baseline (element-wise, reduction, GEMM, or attention) and continue at step 3. 3. Run `python scripts/trial_manager.py init <kernel_name> <baseline_file>`, which creates `trials/<kernel_name>/` and records the baseline. 4. Write the generic tier: `<kernel>_cpu/cpu_features.hpp` in the kernel's own namespace, the dispatcher `<kernel>_cpu/<kernel>_cpu.cpp` with an ATen-only fallback, the bridge `<kernel>_cpu/<kernel>_cpu_torch.cpp`, `torch-ext/torch_binding.cpp` using the `registration.h` macros, and `build.toml` with one `[kernel.*]` section per tier and `include = ["<kernel>_cpu"]` in every section. Read `references/runtime_dispatch.yaml`, `references/build_system.md`, `references/implementation_reference.md`, and `references/correctness.yaml` while writing. Run `python scripts/validate_cpu_kernel.py <kernel_dir>` and fix the named file or `build.toml` section until validation reports no error. A validation fix does not count as a trial. 5. Build and install with the configured commands, by default `kernel-builder build --release` then `pip install dist/*.whl --force-reinstall --no-deps`. When the build fails, read the compiler output, fix the source or the section's `cxx-flags`, and rebuild. You are done when `python -c "import <package>"` succeeds. 6. Run `python scripts/benchmark_cpu.py <baseline_file> --kernel-package <package> --op <package>.<function>`. The correctness check walks tuples, lists, and dicts element-wise, requires equal structure, dtype, and shape, and compares each tensor leaf in its own dtype: half an ulp relative for bf16 and fp16, `atol=1e-6, rtol=1e-5` for fp32, `atol=1e-12, rtol=1e-9` for fp64, exact for integer and bool. Widen with `--atol` and `--rtol` only when the kernel's accumulation order legitimately differs from the reference, and record the reason in the trial's `--strategy`. On correctness failure, read the leaf path, dtype, and worst-element values in the benchmark output: a wrong second output or a dtype or shape change is a binding or dispatcher bug; a one-ulp bf16 difference on many elements is a rounding-mode or conversion bug; a tail-only difference is missing tail handling; a large scattered difference is an alignment bug. Fix on the same branch, rebuild, re-benchmark. Never enter the performance phase with a failing kernel. 7. Add the AVX512 tier in its own translation unit `<kernel>_cpu/<kernel>_avx512.cpp` with its own `cxx-flags` section (`-mavx512f -mavx512bf16 -mavx512vl` for element-wise kernels; GEMM kernels add `-mavx512dq -mavx512bw -mavx512vbmi -mamx-tile -mamx-bf16 -mamx-int8`), with `-fopenmp` in every SIMD section. Add an AVX2 tier only when it gives an element-wise kernel a measurable benefit; GEMM kernels dispatch AVX512 to fallback. Repeat steps 4 to 6, then run `python scripts/trial_manager.py save <kernel_name> <kernel_dir> --strategy "<description>"` and record the numbers with `python scripts/trial_manager.py result <kernel_name> <trial_id> --correctness pass --speedup <x> --baseline_us <us> --kernel_us <us>`. This ends the correctness phase with the AVX512 tier correct and trial t0 recorded. 8. Pin the benchmark to one NUMA node for every later measurement: `numactl --cpunodebind=0 --membind=0 python scripts/benchmark_cpu.py ... --baseline-us <cached>`, where the cached value comes from `python scripts/trial_manager.py baseline-us <kernel_name>`. 9. When `perf_stat_enabled` is true, run `python scripts/cpu_profiler.py --kernel-package <package> --op <package>.<function>` once after the first benchmarked trial. Read IPC together with the L1 and LLC miss rates: a pure AVX512 FMA loop has low IPC by design, so claim a memory bound only when a miss rate is also high. If `perf` is unavailable or `perf stat` returns no counters, continue without profiling, report it, and choose the next trial from `references/optimization_levels.yaml`. Either way, choose the next change from `references/optimization_strategies.md`. 10. For each remaining trial up to `max_trials`, change one thing in the AVX512 tier (blocking, prefetch, unrolling, threading, or a different algorithm from `references/simd_optimization_patterns.yaml`, `references/memory_patterns.yaml`, `references/threading_patterns.yaml`, `references/dtype_optimizations.yaml`, `references/brgemm_patterns.yaml`, `references/quantized_gemm_patterns.yaml`, and `references/optimization_levels.yaml`), then validate, build, benchmark, `save` with `--parent <best_or_current_id>`, and `result`. A regression branches back to the best trial (find its id with `python scripts/trial_manager.py best <kernel_name>`). A kernel slower than baseline on small tensors gets a `num_tokens` threshold below which the dispatcher calls the ATen fallback; see `references/threading_patterns.yaml`. A plateau after two trials changes the algorithm, data layout, or fusion instead of sweeping the same knobs. Stop early only when the speedup exceeds `early_stop_speedup`. When `max_trials` is reached below `early_stop_speedup`, finalize the best trial and report the speedup reached and the trial tree from `trial_manager.py status`. 11. Run `python scripts/trial_manager.py finalize <kernel_name> output/`, then re-run the pinned `benchmark_cpu.py` without `--baseline-us` for the final measurement. Read `references/huggingface-kernels-integration.md` if the kernel is to be published to the Hub. You are done when `output/` holds the best trial's sources with final correctness and speedup recorded. ## What you hand back - Kernel sources: `<kernel>_cpu/` with `cpu_features.hpp`, the dispatcher, the bridge, the AVX512 implementation, and any AVX2 implementation, plus `torch-ext/torch_binding.cpp` and `build.toml`. - Installed package: the wheel under `dist/` and the installed `<package>`. - Trial tree: `trials/<kernel_name>/` with each saved trial, its parent, strategy, correctness, and timing. - Correctness report: the `benchmark_cpu.py` output naming per-dtype tolerances and, on failure, the leaf path of each mismatch. - Performance report: baseline and kernel microseconds and speedup from the NUMA-pinned run. - Final kernel: `output/` holding the best trial's sources and its final measurement.
Files in this package (23)
- SKILL.md 10 KB
- agents/openai.yaml 1 KB
- references/brgemm_patterns.yaml 14 KB
- references/build_system.md 8 KB
- references/correctness.yaml 6 KB
- references/dtype_optimizations.yaml 5 KB
- references/huggingface-kernels-integration.md 5 KB
- references/implementation_reference.md 14 KB
- references/memory_patterns.yaml 3 KB
- references/optimization_levels.yaml 6 KB
- references/optimization_strategies.md 5 KB
- references/quantized_gemm_patterns.yaml 19 KB
- references/runtime_dispatch.yaml 9 KB
- references/simd_optimization_patterns.yaml 7 KB
- references/threading_patterns.yaml 3 KB
- references/workflow_details.md 9 KB
- scripts/analyze_op.py 11 KB
- scripts/benchmark_cpu.py 17 KB
- scripts/config.py 1 KB
- scripts/config.yaml 1 KB
- scripts/cpu_profiler.py 11 KB
- scripts/trial_manager.py 21 KB
- scripts/validate_cpu_kernel.py 13 KB
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · gosuda/dalcom-skills | 100 | Apache-2.0 |
| OutlineDriven/outline-driven-development | 100 | Apache-2.0 |
Database snapshot: 13.11.2026 · package version 56e43c904f90 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 90cuda-cpp-kernels
How to write CUDA C++ kernels for fast-kernel candidates via torch.utils.cpp_extension.load_inline with the pip-installed nvcc - fused epilogues, warp-level reductions, grid barriers / persistent kernels, wmma/mma tensor cores. CUDA C++ is the implementation backend for every… - 100cuda
CUDA C/C++ skill for NVIDIA GPU kernel programming. Use when writing CUDA kernels, managing thread/block/grid hierarchy, optimizing memory access patterns, using streams and async copies, configuring nvcc flags, or integrating Thrust. Activates on queries about CUDA kernels,… - 95metal-kernel
Write Metal/MPS kernels for PyTorch operators. Use when adding MPS device support to operators, implementing Metal shaders, or porting CUDA kernels to Apple Silicon. Covers native_functions.yaml dispatch, host-side operators, and Metal kernel implementation. - 90cuda-kernel-basics
Write first CUDA kernels that map work onto the grid, coalesce global memory, and keep enough warps resident to hide latency. Use when hand-writing or reviewing a CUDA kernel and it runs far below the bandwidth or FLOPs the card should reach. - 85kernel-evolution
Decide whether a GPU kernel is the right thing to change, then write, verify and measure one honestly. Use when a profile points at an operator, when choosing between Triton and CUDA, when deciding whether to fuse, when a kernel is fast but the server is not, or when checking… - 85kernel-evolution
Decide whether a GPU kernel is the right thing to change, then write, verify and measure one honestly. Use when a profile points at an operator, when choosing between Triton and CUDA, when deciding whether to fuse, when a kernel is fast but the server is not, or when checking…