# GitHub - synchopate/llama-modchip: 🐋 Run 284B DeepSeek-V4-Flash (<59GiB RAM, 44.4 tok/s pp, 9.1-14.7 tok/s) & 30B MoEs (25-111 tok/s) on a pure 64GB CPU at 256K ctx (0 GPUs)

> GitHub project runs 284B DeepSeek-V4-Flash and 30B MoE models entirely on CPU with 64GB RAM.
- Title: GitHub - synchopate/llama-modchip: 🐋 Run 284B…
- Summary: GitHub project runs 284B DeepSeek-V4-Flash and 30B MoE models entirely on CPU with 64GB RAM.
- Keywords: llama-modchip, deepseek-v4-flash, mixture-of-experts, cpu-first-inference, quantization, swiglu-h2, stockfish-pvs, fractal-64, llama-cpp, gguf, GitHub
- Source: GitHub — https://github.com/synchopate/llama-modchip
- Read time: 2 min
- Topics: llama-modchip, deepseek-v4-flash, mixture-of-experts, cpu-first-inference, quantization, swiglu-h2, stockfish-pvs, fractal-64, llama-cpp, gguf, technology, startups
## llama-modchip Overview
llama-modchip is a set of ggml / llama.cpp kernel patches, quantization transcoders, and runtime tools designed to run sparse Mixture-of-Experts (MoE) checkpoints entirely in system RAM on x86_64 and ARM64 processors.
## 284B DeepSeek-V4-Flash Model
284.33B Total MoE params (13.17B active) fit in 64GB RAM with 56.56 GiB disk size and 1.71 bpw quantization.
## 30B Nemotron Model
30.2B Total MoE params (3.0B active) fit in 16-32GB RAM with 14.35 GiB disk size and 3.75 bpw quantization.
## CPU-First Inference
llama-modchip runs 284B DeepSeek-V4-Flash and 30B MoE models entirely on CPU with zero GPUs required.
## Throughput Metrics
44.4 tok/s prefill and 9.1-14.7 tok/s token generation for 284B DeepSeek-V4-Flash on 64GB CPU RAM.
## Quantization Breakdown
DeepSeek-V4-Flash uses 1.71 bpw quantization with 56.56 GiB disk size and 57.61 GiB peak VmRSS.
## Model-Specific Optimizations
Nemotron-3.5-30B-A3B uses Fractal-64 (3.75 bpw) with 14.35 GiB disk size and 25.0-60.4 tok/s throughput.
## Hardware Requirements
64GB RAM required for 284B DeepSeek-V4-Flash with 16-32GB RAM for 30B Nemotron models.
## CPU Cache Optimizations
4-level register tiling (ZMM AVX-512BW) improves prefill speed from 25.0 to 44.36 tok/s on 16-core CPUs.
## Self-Speculative Drafting
Stockfish-PVS v2 with 1.64 MB L2/L3 Zobrist n-gram drafter enables instant agent context restore.
## Quantization Profiles
DeepSeek-V4-Flash uses Fractal-64 (1.71 bpw) with 56.56 GiB disk size and 57.61 GiB peak VmRSS.
## Model Benchmarks
284B DeepSeek-V4-Flash achieves 44.36 tok/s prefill and 9.06-10.74 tok/s decode on 64GB CPU RAM.
## Hardware-Specific Optimizations
AMD Zen 5 processors use 48 KB L1D 3D tiling for optimal MoE performance.
## Community Leaderboard
Community benchmarks show 386 tok/s prefill on AMD Threadripper PRO 7985WX with 256GB RAM.
## GitHub Repository Structure
llama-modchip includes patches, documentation, and scripts for CPU-first inference on various hardware.
## Model Performance
DeepSeek-V4-Flash runs at 44.36 tok/s prefill and 9.06-10.74 tok/s decode on 64GB CPU RAM.
## Key takeaway

llama-modchip enables running 284B DeepSeek-V4-Flash and 30B MoE models on 64GB CPU RAM without GPUs.