BarunAction-35M

<a href="https://barunaction-d9123728.nip.io" target="_blank" rel="noopener noreferrer">

</a>

A 35M-parameter local compiler for typed personal actions.

BarunAction-35M turns a natural-language request, explicit tool schemas, context, and a reference
time into strict Action IR JSON. It is small enough for local research, hash-verifies its own
checkpoint, validates every proposal deterministically, and never invokes a real tool.

<a href="https://barunaction-d9123728.nip.io" target="_blank" rel="noopener noreferrer">Try the live browser demo ↗</a> ·
Download the model ·
Read the model card ·
Reproduce the benchmark

Release highlights

The matched Qwen baseline has 494.03M parameters. BarunAction retains 90.80% of its exact-match
rate while trailing by 8.07 points. This is a grouped development split, not the official test.

Try it

<a href="https://barunaction-d9123728.nip.io" target="_blank" rel="noopener noreferrer">Open the live demo ↗</a>, choose an action type, and edit the request.
It runs the real checkpoint, validates the proposal, and never executes a tool.

Best first example: save Mira Shah's name, phone, and email from one sentence. If a result is wrong,
the page includes a short feedback form with an optional screenshot.

Quick start

git clone https://github.com/harrrshall/barunaction-35m.git
cd barunaction-35m
python -m venv .venv
source .venv/bin/activate
pip install -e .

# Weight-free contract and sandbox demonstration
barunaction demo

# Download the versioned release from Hugging Face and verify every required SHA-256
barunaction download --output ./models/BarunAction-35M

# Verify again at any time
barunaction verify --checkpoint ./models/BarunAction-35M

# Run deterministic CPU inference
barunaction infer \
  --checkpoint ./models/BarunAction-35M \
  --tools examples/barunaction_tools.example.json \
  --context examples/barunaction_empty_context.example.json \
  --now 2026-08-03T20:00:00+05:30 \
  --request "Turn on the flashlight" \
  --device cpu

Every successful result still carries policy.execution_permitted: false. A proposal becomes a
real action only if an application supplies its own authentication, authorization, confirmation,
rate limiting, and tool execution layer outside this package.

Python API

import json

from barunaction import BarunActionCompiler, download_candidate_checkpoint

downloaded = download_candidate_checkpoint("./models/BarunAction-35M")
compiler = BarunActionCompiler(downloaded.path, device="cpu")

with open("examples/barunaction_tools.example.json", encoding="utf-8") as source:
    tools = json.load(source)

result = compiler.infer(
    request="Turn on the flashlight",
    tool_schemas=tools,
    context={},
    now="2026-08-03T20:00:00+05:30",
)
print(json.dumps(result.to_dict(), indent=2))

Inputs are rejected unless the request is non-empty, tool declarations are strict and
non-duplicated, context is a JSON object, and now is timezone-aware. Outputs are parsed as one
complete JSON object without extraction, repair, coercion, or implicit defaults.

What the released model learned

The released model was post-trained for one epoch with full-parameter, response-only supervised
fine-tuning from the exact harrrshall/BarunLM-35M base checkpoint. It saw 7,937 derived examples
once and completed 126 optimizer steps. It used no synthetic teacher data, distillation,
preference optimization, DPO, or reinforcement learning.

The released training population covers seven Mobile Actions tools:

• create_calendar_event
• create_contact
• open_wifi_settings
• send_email
• show_map
• turn_off_flashlight
• turn_on_flashlight

See Training for the exact recipe and Data card
for revisions, derivation, licenses, hashes, and redistribution boundaries.

Benchmark result and terminology

The primary metric is strict Action IR AST exact match. A prediction must be one valid JSON
object, satisfy the supplied tool schema, match the decision and call mode, preserve ordered calls
for SINGLE/SERIAL, and match typed arguments exactly. PARALLEL calls are compared as an
order-independent multiset. Missing outputs, parse failures, schema failures, truncations, and
generation failures remain incorrect.

GPT-2 used a different benchmark, so its score is not directly comparable with Action IR exact
match. SmolLM2's zero is specific to that structured-output adaptation run, not general ability.

The comparison used identical 7,937 training IDs, 756 development IDs, semantic labels, one pass,
effective batch 63, 126 optimizer steps, seed 17, final-checkpoint selection, and unconstrained
deterministic greedy decoding. Model-native tokenizers/templates and learning rates differed, and
BarunAction had an earlier development sweep. This is therefore a one-seed matched
semantic-adaptation comparison, not a universal model ordering.

The source is google/mobile-actions revision
e920309bc2acbc2e99a5e3201cf37df2b9fd9151 (CC BY 4.0). The reported 756 rows are grouped public
development data and contain only CALL cases. The separate 961-row official evaluation split
was never read or materialized.

Full definitions, evaluator logic, exact model revisions, data hashes, decoding settings, and
curated sample-level outcome evidence are in Benchmark and
Evaluation. The executable scorer is
src/barunlm/evaluation/mobile_actions.py.

Reproduce

The public repository separates four layers so results are inspectable without internal lab
state:

src/                 model, training, inference, schema, scorer, quantization
configs/             released candidate and matched-baseline contracts
benchmarks/          curated manifests, aggregates, paired outcomes, verification
docs/                model, data, training, evaluation, retrieval, deployment cards
space/               public Gradio demonstration

Common verification commands:

# Product runtime and scorer tests
pytest -q

# Style and formatting
ruff check .
ruff format --check .

# Rebuild the grouped public train/dev manifests from the pinned source.
# Official evaluation rows remain opaque and are never parsed or materialized.
python -m barunlm.datasets.mobile_actions \
  --download-pinned \
  --tokenizer-json ./models/BarunAction-35M/tokenizer.json \
  --tokenizer-revision v1.0.0 \
  --output-dir ./build/mobile-actions

# Re-score the published BarunAction predictions without generating new outputs
barunaction score-mobile \
  --manifest ./build/mobile-actions/dev.jsonl \
  --predictions benchmarks/evidence/barunaction-predictions.jsonl \
  --output ./reproduced-score

No raw training dataset or sealed evaluation label is redistributed. The repository supplies
pinned source revisions, derivation code, membership and content hashes, licenses, evaluator code,
aggregate results, and curated sample-level comparison outcomes. See
Release evidence.

Model files

The canonical Hugging Face mirror contains:

The source artifacts remain immutable public W&B v0 releases. Hugging Face is a byte-identical,
freshly verified mirror. See Retrieval for both locations.

Citation

@software{harrrshall2026barunaction35m,
  author  = {Harrrshall},
  title   = {BarunAction-35M: A Compact Typed Personal-Action Compiler},
  year    = {2026},
  version = {v1.0.0},
  url     = {https://github.com/harrrshall/barunaction-35m}
}

License

Source code and released model weights are licensed under Apache License 2.0. Mobile Actions is
licensed CC BY 4.0 and retains its upstream attribution and terms. See NOTICE and the
data card.