What AI can this device actually run?

Q: How accurate are the tokens-per-second numbers?

Exact when the runtime reports token counts (Ollama, LM Studio, WebLLM). Estimated as characters divided by four over the generation phase, and labelled as such, when it does not (Apple Foundation Models, Chrome's Prompt API).

DeviceBench finds every on-device model your machine already has or can fetch, runs real tasks on each one, and shows what it costs in CPU, GPU, memory and network while it happens.

Open the lab Run the companion

Nothing you type leaves your machine. Browser models work here directly; Ollama and Apple's model need the free companion.

This browser, right now

checking

Feature detection only. Nothing is downloaded until you run a model in the lab.

Coverage

Three tiers of on-device AI, one place to try them

Every provider speaks the same interface in the lab, so a 200 MB task model and an 8B chat model get measured the same way on the same prompt.

In your browser, now

No install. Works on this site.

Chrome built-in Gemini NanoPrompt API for chat and structured JSON, plus Summarizer, Translator, Language Detector, Writer, Rewriter and Proofreader where Chrome exposes them.
WebGPU models via WebLLMLlama 3.2, Qwen, Gemma, SmolLM and Phi families, fetched once and cached by the browser. Runs in Safari on iPhone too.

Chrome desktopSafari 26FirefoxiPhone (WebGPU)

With the local companion

A small Node server on 127.0.0.1. One command.

Ollama, LM Studio, llama.cpp, MLXAny model you have pulled, with exact token counts and load times from the runtime.
Apple Foundation ModelsThe ~3B model behind Apple Intelligence, through a Swift bridge, including schema-constrained JSON.
Apple NaturalLanguageSentiment, entities, language ID and sentence embeddings in single-digit milliseconds.
System loadCPU per core, memory the way Activity Monitor counts it, GPU utilization, network, and which process is using what.

macOSWindows and Linux (Ollama)

Native only, mapped for you

Not reachable from a page. Listed so the map is complete.

Foundation Models on iPhone and iPadiOS 26, iPhone 15 Pro and later, every 16 and 17, M-series iPads. Swift only.
Gemini Nano on AndroidML Kit GenAI APIs on AICore devices: Pixel 8+, Galaxy S24 and S25 series.
Phi Silica on WindowsCopilot+ PCs through the Windows AI APIs.

iOSAndroidWindows NPU

What you measure

Numbers you can put in a decision

Every run records the same fields, and the lab says which ones are reported by the runtime and which are estimated.

231ms

Time to first token

Wall clock from send to the first streamed character. Includes model load when the runtime had to load.

94.6tok/s

Generation speed

From the runtime's own token counts when available, otherwise estimated and labelled.

86% GPU

Peak load during the run

CPU total and per core, GPU utilization, memory delta, sampled twice a second and windowed to the run.

0bytes

Network during inference

Proof it stayed local. First downloads show up as the only spikes.

Example figures: gemma4 8B on an M4 Max through Ollama, warm. Your numbers replace these the moment you run something.

Platform map, September 2026

What ships where, and what gates it

Ordered by how many real users can run it without installing anything.

Where	What is there	How you call it	Reach
iPhone, iPad, Mac	NaturalLanguage, Vision, Speech, Translation, Core MLNo Apple Intelligence gate. Every supported device.	NLTagger, NLEmbedding, VNRecognizeTextRequest, SpeechAnalyzer, TranslationSession	Every Apple device
iPhone, iPad, Mac	Apple Foundation Models, ~3BOS 26, Apple Intelligence on. iPhone 15 Pro and later, M-series iPads and Macs.	FoundationModels, LanguageModelSession (Swift). The companion bridges it on Mac.	Recent Apple devices, opt-in
Android	Gemini Nano through ML Kit GenAIAICore devices only: Pixel 8+, Galaxy S24/S25 series and a growing list. Mid-range Android is out.	ML Kit GenAI Prompt, Summarization, Proofreading, Rewriting, Image Description (Kotlin)	Flagships
Chrome desktop	Gemini Nano built-in AIChrome 138+ for task APIs, 148+ for the Prompt API. 22 GB free disk, 4 GB VRAM or 16 GB RAM, one background download.	LanguageModel, Summarizer, Translator, LanguageDetector from any page. Not on Android or iOS.	Capable laptops
Any WebGPU browser	In-page models via WebLLM, Transformers.js, ONNX Runtime Web300 MB to 2 GB first load per model. About 1B parameters of headroom on phones.	CreateMLCEngine(modelId) and an OpenAI-shaped chat API	Chrome, Edge, Safari 26, Firefox 141+
Mac, Windows, Linux	Ollama, LM Studio, llama.cpp, MLXInstall required. The most capable models, the smallest audience.	HTTP on localhost: /api/chat or an OpenAI-compatible /v1/chat/completions	Developers and enthusiasts
Windows	Phi Silica and Windows AI APIsCopilot+ PC hardware only (40+ TOPS NPU).	Windows App SDK, Microsoft.Windows.AI.Text	Copilot+ PCs

Run locally

The companion in two commands

Node 20 or newer. It binds to 127.0.0.1 only and accepts browser requests solely from this site and localhost.

Clone and startInstalls one dependency (Express) and starts the server on port 4242. Ollama is detected if it is running.
Open the labEither the hosted lab, which connects to 127.0.0.1:4242 on its own, or the copy the companion serves at 127.0.0.1:4242. Chrome may ask once to allow this site to reach your local network; that prompt is the connection.
Optional: Apple Foundation ModelsOn macOS 26 with Xcode 26, build the Swift bridge once. The companion starts it automatically afterwards.

Runs are appended to data/runs.jsonl on your machine and never uploaded. Export them as CSV from the Runs tab.

git clone https://github.com/elowinlabs/devicebench
cd devicebench && npm install && npm start

# optional, macOS 26 + Xcode 26
npm run bridge:build

# then open
https://devicebench.io/app/

FAQ

Questions people ask before they run it

What on-device AI models does my computer already have?

On a Mac with macOS 26 and Apple Intelligence on: Apple's ~3B Foundation Model plus the NaturalLanguage, Vision, Speech and Translation models. In Chrome 138+ on desktop: Gemini Nano behind the Prompt, Summarizer, Translator and Language Detector APIs. In any WebGPU browser: models fetched into the page such as Llama 3.2 1B. And anything you have pulled into Ollama or LM Studio. The lab detects each of these live.

Does DeviceBench send my prompts anywhere?

No. Every model runs on your machine or inside your browser. The only network traffic is a model's first download, which the Network chart shows you: Chrome's model from Google, WebGPU weights from Hugging Face, Ollama pulls from Ollama's registry.

What do I need the local companion for?

Ollama, LM Studio, llama.cpp, MLX and Apple Foundation Models run outside the browser, and a browser cannot read system CPU, GPU or memory counters. The companion is a small Node server on 127.0.0.1 that bridges both. Browser models work on this site without it.

Can I run on-device AI in a web page on iPhone?

Safari 26 ships WebGPU, so WebGPU models up to roughly 1B parameters run in a page on iPhone. Apple's Foundation Model is not exposed to web pages; it needs a native Swift app, which is why it is listed as native only.

Which phones have Gemini Nano?

Android devices with AICore: Pixel 8 and later, Galaxy S24 and S25 series and a growing list of flagships. It is reachable through the ML Kit GenAI APIs in native apps, not from Chrome for Android pages.

How accurate are the tokens-per-second numbers?

Exact when the runtime reports token counts, which Ollama, LM Studio and WebLLM do. Estimated as characters divided by four over the generation phase, and labelled as an estimate, when it does not, which is the case for Apple Foundation Models and Chrome's Prompt API.