Zero-allocation LLM inference in pure C#.
Overfit is the attempt to make local AI feel like a normal .NET library: a model file, a NuGet package, explicit memory ownership and no Python process.
What changed recently
The branch now includes a Bielik preset for the ASP.NET local-agent demo: local Bielik GGUF, Polish documents, Polish RAG, C# tool calling, business-decision JSON and metrics.
cd Demo/LocalAgentAspNetDemo
.\download-bielik.cmd
.\download-embedder.cmd
.\run-bielik.cmdThe current result
| Metric | Headline |
|---|---|
| Decode hot path | 0 B/token target in the benchmark path |
| Qwen2.5-3B Q4_K_M | ~19 tok/s on CPU, up to ~24 tok/s with the opt-in repacked-GEMV mode |
| Bielik-4.5B Q4_K_M | ~17 tok/s on CPU, −36% working set vs same-file llama.cpp |
| Managed heap | ~220 MB live heap for mmap-backed 3B model load path |
This is not a claim that Overfit beats llama.cpp on raw throughput. The point is deployment shape: local AI inside .NET, with RAG, C# tools and structured output, without a separate runtime.