Writing

Blog

Thoughts on software, technology, and whatever else I find interesting.

P

Latest post

Putting ONNX Runtime Behind a NativeAOT .NET Engine: the On-Device AI Roadmap

The dotnet-native-interop POC was built around one swap-in seam — an ILanguageModel interface that the whole app streams through. This is the plan for what goes behind it: ONNX Runtime for on-device inference in .NET 10, why ONNX fits the NativeAOT/no-cloud constraints, and the toolchain risks that make on-device AI the last and hardest phase. Honest roadmap, with links to the seam and the design spec.

June 5, 2026 · 6 min read

Read the post

Jun 5, 2026 · 7 min

ONNX Runtime Inside NativeAOT .NET: The Embedding Pipeline, Tensor Plumbing, and Linking Gauntlet

A deep technical design for running ONNX Runtime on-device behind the dotnet-native-interop swap-in seam: the exact embedding pipeline (tokenize → int64 tensors → InferenceSession → mean-pool → L2-normalize → cosine search), OrtValue zero-copy vs DenseTensor, SessionOptions and execution providers on mobile, and the NativeAOT trimming/native-linking gauntlet that makes this the riskiest phase. Real Microsoft.ML.OnnxRuntime C#.

Read more