Why Cursor's Mixture-of-Kittens MoE Megakernel Solves Blackwell's Hidden CPU Bottleneck

AI coding startup Cursor has open-sourced Mixture-of-Kittens (MoK), a production-grade Mixture-of-Kittens MoE megakernel designed specifically to unlock massive throughput on Nvidia's next-generation GB300 NVL72 hardware. By fusing Mixture-of-Experts (MoE) communication and computation into a single, fully deterministic kernel, MoK solves a critical latency bottleneck that has plagued large-scale model training.

The Architecture Bottleneck: Why Blackwell's Grace CPU Sinks MoE Performance

To understand why the Mixture-of-Kittens MoE megakernel matters, one must look at the physical architecture of modern AI clusters. Nvidia's flagship GB300 NVL72 platform operates as a single, massive logical GPU by connecting 72 Blackwell GPUs via a high-bandwidth NVLink backplane. On paper, this scale-up design is an ideal match for MoE architectures like DeepSeek-V3, which route specific tokens to specialized expert networks across the network.

In practice, however, these networks run into a severe coordination tax. Traditional MoE implementations rely on host CPUs—specifically Nvidia's ARM-based Grace processors—to orchestrate the all-to-all communication required to dispatch tokens to the correct expert GPUs and combine them afterward. This continuous round-trip synchronization between CPU and GPU wastes precious clock cycles, leaving expensive Blackwell chips sitting idle while waiting for the CPU to finish its bookkeeping.

Inside the Megakernel: How Mixture-of-Kittens Bypasses the CPU

Cursor's engineering team resolved this overhead by completely redesigning the MoE layer from first principles. Instead of handling communication and matrix multiplication as discrete, CPU-coordinated steps, the Mixture-of-Kittens MoE megakernel fuses communication directly into the GPU execution path.

This design introduces several key architectural innovations:

• Ring Token Buffers: By managing token queues directly on the GPU using ring buffers, MoK eliminates the need for CPU-GPU synchronization boundaries.
• Structured Overlap: The kernel continuously overlaps the dispatch and combine communication phases with the active computation of the Feed-Forward Network (FFN) expert layers.
• Determinism and Precision: MoK maintains complete mathematical determinism during training while offering native support for MXFP8 and NVFP4 low-precision data formats.

As we have scaled the training and inference of Composer, our agentic coding model, the mixture-of-experts layer has consistently remained the major bottleneck. Depending on the workload and training configuration, it can consume more than half of end-to-end training time.

Cursor Engineering Team

Shattering Baselines: 2.37x Higher MXFP8 Forward Throughput

By keeping the entire MoE pipeline on the GPU, Cursor achieves dramatic performance gains over established open-source baselines like Nvidia's Megatron-LM and DeepSeek's DeepEP. In production environments spanning multiple NVL72 racks, MoK delivers up to a 2.37x increase in MXFP8 forward throughput. More importantly, this translates to a 1.41x improvement in end-to-end tokens-per-second during actual training run of Cursor's flagship coding model, Composer.

The kernel is specifically optimized to support DeepSeek-style architectures that utilize a combination of shared and routed experts, a pattern increasingly adopted by frontier models from Qwen, GLM, and Kimi.

What This Means for the Open-Source AI Ecosystem

Cursor's decision to open-source the Mixture-of-Kittens MoE megakernel is a significant win for the broader AI community. Highly optimized kernel engineering has traditionally been a closely guarded secret of massive labs with proprietary hardware runtimes. By releasing MoK under an open-source license, Cursor lowers the barrier to entry for training ultra-efficient, DeepSeek-style MoE models on top-tier Nvidia silicon.

For founders and machine learning engineers, it serves as a stark reminder: as physical hardware limits are pushed, the most substantial performance gains will no longer come from generic compiler optimizations, but from writing custom, hardware-aware CUDA kernels that treat compute and networking as a unified resource.