Recursive Flint 40B MoE coding model runs 125+ tokens/sec locally

What changed

Recursive Superintelligence released Flint, an open-weight mixture-of-experts coding model with 39.8 billion total parameters and 2.5 billion active parameters per token. The company built the model using Eureka Machine, its automated research system that handled architecture design, training recipes, quantization strategy, and inference optimization. Flint weights are available on Hugging Face under the Apache 2.0 license.

The sparse activation design allows Flint to run efficiently on consumer hardware. Recursive developed the model in under four months with a small team of people and agents working together.

Flint supports context windows of up to 512,000 tokens, though the company recommends staying within 256,000 tokens for optimal performance. The model comes in multiple quantized formats. INT4 weights on Apple Silicon require approximately 22.6 gigabytes of memory, while NVFP4 weights on NVIDIA GPUs require approximately 22.9 gigabytes. For production deployment at full precision, Recursive recommends one NVIDIA B300 or H200 GPU per replica running the 80-gigabyte BF16 checkpoint.

Why it matters

Flint enables developers to run capable coding agents directly on local machines without sending data to remote APIs. This approach eliminates connectivity dependencies and avoids cloud API costs. The inference speed is significant: Recursive reports that Flint runs more than five times faster than dense models of 25 to 40 billion parameters that achieve comparable coding benchmark scores, and up to twice as fast as comparable mixture-of-experts models.

For teams building AI agents, this speed-to-capability ratio has practical implications. Developers can iterate faster during local testing than they could using cloud APIs, and the model fits on standard consumer hardware such as an M4 Pro MacBook Pro or a 48-gigabyte GPU.

According to Recursive’s benchmarking, Flint leads mixture-of-experts models with fewer than 50 billion total parameters on agentic coding benchmarks including SWE-Bench Pro v2, Terminal-Bench 2.1, and SWE-Bench Verified. The model scores 74.2% on SWE-Bench Verified and 95.1% on IFEval, though performance varies across benchmarks. On Terminal-Bench 2.1, Flint scores 54.9%, below dense Qwen 3.8 at 73.0%, indicating tradeoffs between speed and performance on certain coding tasks.

What to test

Before adopting Flint for production coding agent work, verify these claims independently:

• Benchmark reproducibility. Recursive reports scores on SWE-Bench Pro v2, SWE-Bench Verified, Terminal-Bench 2.1, LiveCodeBench v6, IFBench, and IFEval. Run tests on your own codebases and coding tasks. The company’s scores are self-reported; third-party validation would provide stronger confidence in the results.

• Speed on your hardware. The throughput claims of 125+ tokens per second on M4 Pro and 150+ on RTX Spark use short-context sessions. Test at the context lengths your agent actually requires. Prefill time is excluded from the reported speed measurements.

• Quantization accuracy. The NVFP4 model scores within 0.7 points of the 16-bit model across Recursive’s benchmarks. Measure this performance gap on your specific coding tasks to verify that 4-bit quantization meets your accuracy requirements.

• Tool integration. Flint works with llama.cpp, Ollama, Unsloth, LM Studio, vLLM, and SGLang. Test the inference engine that matches your workflow. Recursive also released rec, a lightweight engine for M4 and M5 chips.

• Agentic limitations. Recursive notes that Flint may require more explicit guidance than larger closed-source models for complex tasks. Structure work into smaller subtasks with clear instructions, and test your specific agent loop before deployment.

Common questions

How does Flint’s architecture enable fast inference?

Flint uses a mixture-of-experts design with 49 layers: 2 dense layers and 47 MoE layers. Each MoE layer contains 256 routed experts, but the router selects only 6 per token plus 1 always-active shared expert, keeping active parameters at 2.5 billion. The attention layers use two types: 35 layers employ a 513-token sliding window, and 14 use full attention. This sparse activation is why Flint achieves speed gains compared to dense models.

What did the Eureka Machine contribute to Flint?

Recursive’s Eureka Machine proposed and tested three major contributions. First, it searched expert configurations starting from a 35-billion-parameter baseline and identified the 40-billion total, 2.5-billion-active architecture. Second, it discovered a quantization approach that quantizes the output layer to 4 bits and uses offline distillation to recover accuracy, reducing training steps by approximately half versus standard quantization-aware training. Third, it wrote specialized kernels for low-bit projections, MoE operations, attention, and prompt processing, including a Metal megakernel on M4 and M5 chips that fuses the entire decode forward pass.

Where can I download Flint and what frameworks does it support?

Flint weights are available on Hugging Face under Apache 2.0. The model works with llama.cpp, Ollama, Unsloth, LM Studio, vLLM, and SGLang. Recursive also released rec, a lightweight inference engine optimized for Flint on M4 Pro and M5 Pro/Max hardware with simple setup.

The conclusion

Flint is a credible local coding model designed for developers and teams building agentic workflows. The mixture-of-experts architecture and optimized inference stack deliver measurable speed gains on consumer hardware, and the open weights reduce barriers to adoption. Recursive’s completion of the model in under four months using automated research is noteworthy, though automation does not eliminate the need for independent validation of the benchmark claims.

The model’s primary limitation is that it may require more explicit instruction than larger closed-source alternatives. For teams comfortable with this tradeoff and seeking to keep data local and avoid API costs, Flint merits evaluation. Watch for third-party benchmark reports and real-world agentic performance data as adoption increases.

Related reading

• RTX Spark PC AI launches Oct 16 with local agents
• llm-d 1.0 Kubernetes inference stack for TPU and GPU
• DeepInfer: DeepSeek open-sources inference engine