Generate Pipeline

The generation pipeline provides a high-level API for parsing C/C++ headers and producing writer output with a two-layer cache (IR cache + output cache). On cache hit, no libclang installation is required.

Functions

generate()

Parse a header and produce output using a single writer, with caching.

generate

generate(header_path, writer_name=None, *, code=None, backend_name=None, runtime=None, language=None, classification=None, include_dirs=None, defines=None, extra_args=None, writer_options=None, output_path=None, store_dir=None, no_cache=False, no_ir_cache=False, no_output_cache=False, project_prefixes=None, auto_install_libclang=None, target=None, _result_meta=None)

Parse a C/C++ header and generate output using a single writer.

Uses the two-layer cache: first checks IR cache, then output cache. Falls back to parsing with the backend if IR cache misses.

Parameters:

Name	Type	Description	Default
header_path	str | Path | InputSpec	Path to the header file or an InputSpec.	required
writer_name	str | None	Writer to use (default: "json").	None
code	str | None	Raw code string. When provided, this content is parsed instead of reading header_path from disk.	None
backend_name	str | None	Backend to use (default: "libclang").	None
runtime	str | None	Target runtime environment (e.g. "nim", "mojo").	None
language	str | None	Input source language (e.g. "c", "cpp").	None
classification	str | None	Input classification (e.g. "header", "source").	None
include_dirs	list[str] | None	Include directories for parsing.	None
defines	list[str] | None	Preprocessor defines (without -D prefix).	None
extra_args	list[str] | None	Additional backend args.	None
writer_options	dict[str, object] | None	Writer constructor kwargs.	None
output_path	str | Path | None	If provided, write output to this file.	None
store_dir	str | Path | None	Store directory. When None, falls back to the HEADERKIT_STORE_DIR environment variable, then auto-detects .headerkit/ in the project root.	None
no_cache	bool	Disable all caching.	False
no_ir_cache	bool	Disable IR cache only.	False
no_output_cache	bool	Disable output cache only.	False
project_prefixes	tuple[str, ...] | None	Tuple of project prefix directories for backend.	None
auto_install_libclang	bool | None	Explicitly enable (True) or disable (False) automatic libclang installation. When None, falls back to the HEADERKIT_AUTO_INSTALL_LIBCLANG env var, then pyproject.toml config, then defaults to False.	None
target	str | None	Target triple for cross-compilation (e.g., "aarch64-apple-darwin"). When None, auto-detects the host triple.	None

Returns:

Type	Description
str	Generated output string.

Raises:

Type	Description
FileNotFoundError	If header_path does not exist and code is not provided.

Example:

from headerkit import generate

# Parse and generate CFFI output (default writer)
output = generate("mylib.h")

# Generate CFFI bindings with custom include dirs
output = generate(
    "mylib.h",
    writer_name="cffi",
    include_dirs=["/usr/local/include"],
    defines=["MY_DEFINE=1"],
)

# Write output to a file
generate("mylib.h", writer_name="ctypes", output_path="mylib_bindings.py")

# Parse from a string instead of a file
output = generate("virtual.h", code="int add(int a, int b);")

# Pass an InputSpec directly or specify a runtime target
from headerkit.ir import InputSpec

spec = InputSpec.from_path("api.h", content="int compute(int x);", language="c")
output = generate(spec, writer_name="json", runtime="nim")

generate_all()

Parse a header once and produce output for multiple writers.

generate_all

generate_all(header_path, writers=None, *, backend_name=None, runtime=None, language=None, classification=None, include_dirs=None, defines=None, extra_args=None, writer_options=None, output_dir=None, output_paths=None, store_dir=None, no_cache=False, no_ir_cache=False, no_output_cache=False, auto_install_libclang=None, target=None)

Parse a C/C++ header and generate output for multiple writers.

Parses the header once (or loads from IR cache), then runs each writer (or loads from output cache).

Parameters:

Name	Type	Description	Default
header_path	str | Path | InputSpec	Path to the header file or an InputSpec.	required
writers	list[str] | None	List of writer names. Default: ["json"].	None
backend_name	str | None	Backend to use (default: "libclang").	None
runtime	str | None	Target runtime environment (e.g. "nim", "mojo").	None
language	str | None	Input source language (e.g. "c", "cpp").	None
classification	str | None	Input classification (e.g. "header", "source").	None
include_dirs	list[str] | None	Include directories for parsing.	None
defines	list[str] | None	Preprocessor defines (without -D prefix).	None
extra_args	list[str] | None	Additional backend args.	None
writer_options	dict[str, dict[str, object]] | None	Per-writer kwargs, keyed by writer name.	None
output_dir	str | Path | None	Base directory for output files (auto-named).	None
output_paths	dict[str, str | Path] | None	Explicit output paths per writer name.	None
store_dir	str | Path | None	Store directory. When None, falls back to the HEADERKIT_STORE_DIR environment variable, then auto-detects .headerkit/ in the project root.	None
no_cache	bool	Disable all caching.	False
no_ir_cache	bool	Disable IR cache only.	False
no_output_cache	bool	Disable output cache only.	False
auto_install_libclang	bool | None	Explicitly enable (True) or disable (False) automatic libclang installation. Passed through to generate().	None
target	str | None	Target triple for cross-compilation. Passed through to generate().	None

Returns:

Type	Description
list[GenerateResult]	List of GenerateResult, one per writer.

Raises:

Type	Description
FileNotFoundError	If header_path does not exist.

Example:

from headerkit import generate_all

results = generate_all(
    "mylib.h",
    writers=["cffi", "ctypes", "json"],
    output_dir="generated/",
)
for r in results:
    print(f"{r.writer_name}: cached={r.from_cache}, path={r.output_path}")

batch_generate()

Resolve header paths from glob patterns and generate output with per-pattern overrides and collision checking.

batch_generate

batch_generate(*, patterns, exclude_patterns=None, writers=None, backend_name=None, runtime=None, language=None, classification=None, include_dirs=None, defines=None, extra_args=None, writer_options=None, output_templates=None, store_dir=None, no_cache=False, no_ir_cache=False, no_output_cache=False, auto_install_libclang=None, target=None, project_root=None, header_overrides=None)

Generate output for multiple headers resolved from glob patterns.

Resolves header paths from patterns, applies per-pattern overrides, checks for output collisions, then generates output for each header/writer combination.

Parameters:

Name	Type	Description	Default
patterns	list[str]	Glob patterns or literal paths for header selection.	required
exclude_patterns	list[str] | None	Glob patterns for paths to exclude.	None
writers	list[str] | None	List of writer names. Default: ["json"].	None
backend_name	str | None	Backend to use (default: "libclang").	None
runtime	str | None	Target runtime environment (e.g. "nim", "mojo").	None
language	str | None	Input source language (e.g. "c", "cpp").	None
classification	str | None	Input classification (e.g. "header", "source").	None
include_dirs	list[str] | None	Include directories for parsing.	None
defines	list[str] | None	Preprocessor defines (without -D prefix).	None
extra_args	list[str] | None	Additional backend args.	None
writer_options	dict[str, dict[str, object]] | None	Per-writer kwargs, keyed by writer name.	None
output_templates	dict[str, str] | None	Per-writer output path templates (highest priority).	None
store_dir	str | Path | None	Store directory. When None, falls back to the HEADERKIT_STORE_DIR environment variable, then auto-detects .headerkit/ in the project root.	None
no_cache	bool	Disable all caching.	False
no_ir_cache	bool	Disable IR cache only.	False
no_output_cache	bool	Disable output cache only.	False
auto_install_libclang	bool | None	Explicitly enable or disable auto-install.	None
target	str | None	Target triple for cross-compilation.	None
project_root	Path | None	Project root directory. Auto-detected if not provided.	None
header_overrides	dict[str, dict[str, object]] | None	Per-pattern override dicts (pattern -> config dict).	None

Returns:

Type	Description
BatchResult	BatchResult with results for each header/writer combo.

Raises:

Type	Description
ValueError	If no headers match, output paths collide, or no output template is resolvable for a header/writer combo.

Example:

from headerkit import batch_generate

result = batch_generate(
    patterns=["include/**/*.h"],
    writers=["ctypes", "json"],
)
print(f"Processed {result.headers_processed} headers, {len(result.results)} outputs")

Data Classes

GenerateResult

GenerateResult dataclass

GenerateResult(writer_name, output, output_path, from_cache)

Result of a single writer's generation.

BatchResult

BatchResult dataclass

BatchResult(results, headers_processed, headers_skipped)

Result from batch_generate().

Store Merge

store_merge()

Merge multiple headerkit store directories into a single target directory. Useful for combining platform-specific cache entries collected from CI.

store_merge

store_merge(*, sources, target)

Merge one or more source store directories into a target store.

For each source, finds all ir/ and output/*/ layers and: - Copies entry subdirectories (slug dirs) into the corresponding target layer, skipping entries whose slug already exists with the same cache_key. - Merges index.json by combining entries dicts (later sources win on conflict).

Parameters:

Name	Type	Description	Default
sources	list[str | Path]	Source store directory paths (e.g., ["/tmp/store-linux"]).	required
target	str | Path	Target store directory path (e.g., ".headerkit/").	required

Returns:

Type	Description
MergeResult	A :class:MergeResult summarizing what was merged.

Raises:

Type	Description
FileNotFoundError	If a source directory does not exist.

Example:

from headerkit import store_merge

result = store_merge(
    sources=["store-linux/", "store-macos/"],
    target=".headerkit/",
)
print(f"New: {result.new_entries}, Skipped: {result.skipped_entries}")

MergeResult

MergeResult dataclass

MergeResult(new_entries=0, skipped_entries=0, overwritten_entries=0, errors=list())

Result of a store merge operation.

Parameters:

Name	Type	Description	Default
new_entries	int	Number of entry directories copied to the target.	0
skipped_entries	int	Number of entries skipped (same slug and cache_key).	0
overwritten_entries	int	Number of entries overwritten (same slug, different cache_key).	0
errors	list[str]	List of error messages for entries that failed to copy.	list()

JSON Deserialization

json_to_header()

Deserialize a JSON string or dict back into a Header IR object. This is the inverse of the JSON writer's serialization.

json_to_header

json_to_header(data)

Deserialize a JSON string or dict into a Header IR object.

Accepts either a JSON string (parsed with json.loads) or the dict that json.loads would return. This is the inverse of headerkit.writers.json.header_to_json / header_to_json_dict.

Parameters:

Name	Type	Description	Default
data	str | dict[str, Any]	JSON string or dict from header_to_json / header_to_json_dict.	required

Returns:

Type	Description
Header	Reconstructed Header IR.

Raises:

Type	Description
ValueError	If the JSON structure is invalid or contains unknown declaration/type kinds.

Example:

from headerkit import json_to_header, get_writer

# Round-trip: IR -> JSON -> IR
json_writer = get_writer("json")
json_str = json_writer.write(header)
restored = json_to_header(json_str)