Inference, from the metal up

The default way of running on-device AI at Scale

We hand-write the GPU and NPU kernels that make consumer silicon fast — and open-source the SDKs, infrastructure, and console that run models on every platform.

Use the SDKs Talk to us

# one C++ core · six bindings
pip install onchip
onchip init --silicon consumer

On-device

Inference on Apple GPUs, Qualcomm NPUs, CPU, and WASM — the silicon already in the machine.

Privacy

Tokens and compute stay on the device instead of a cloud API. Models ship with the app.

Scale

Ship on-device AI to iOS, Android, macOS, Windows, Linux, web, and embedded.

How it fits together

From the metal up.

A research-first inference lab. Custom engines for your silicon, or open-source SDKs for your app.

The silicon is already there. Every consumer device ships with a supercomputer — Apple GPUs, Qualcomm NPUs — that generic runtimes leave idle.
We write the kernels by hand. No generic abstraction layers. Hand-designed memory layouts, fused operators, engines built per silicon.
One C++ core above them. Inference, model management, routing, telemetry — one runtime shared by every platform we target. All of it open source.
Six SDKs, one behavior. Thin bindings over the same core, so every platform gets the same models, the same API, and the same speed.
Your app, on every platform. Ship on-device AI across the fleet, and manage models from the console.

Open source

One core, six SDKs

Everything above the kernel is open source: one C++ core with SDKs for Swift, Kotlin, React Native, Flutter, TypeScript, and C++. Write contact@onchiplab.space for repository access.

import OnChip

let runtime = OnChip.Runtime.onDevice()
try runtime.load("local-model")
let stream = try runtime.generate(prompt)

import space.onchiplab.OnChip

val runtime = OnChip.Runtime.onDevice()
runtime.load("local-model")
val stream = runtime.generate(prompt)

import { Runtime } from "onchip"

const runtime = Runtime.onDevice()
await runtime.load("local-model")
const stream = runtime.generate(prompt)

#include <onchip/runtime.hpp>

auto rt = onchip::Runtime::on_device();
rt.load("local-model");
auto stream = rt.generate(prompt);

llmstttts vlmembeddingsrouting

01 · Inference — Research

Engines for consumer silicon

Hand-written inference engines for Apple GPUs and Qualcomm NPUs. LLM, VLM, STT, TTS, and embeddings run on the device. Every claim published with numbers — we do not print placeholders here.

Explore the engines

02 · Open source — SDKs

Infrastructure and console

The hosted console adds OTA model updates and fleet ops on top of the same core. Thin bindings, one behavior, every platform.

Read the docs

Run it on the die.

Custom engines for your silicon, or open-source SDKs for your app — either way, it starts with a conversation or a clone. OnChip Lab · onchiplab.space

Talk to us Use the SDKs