From GitHub · · 1m 40s
ggml v1.0.0 release
ggml 1.0.0 drops deprecated APIs and adds FP8 support across multiple backends.
In brief
ggml 1.0.0 drops deprecated APIs and adds FP8 support across multiple backends. ggml releases v1.0.0 with pruned API, FP8 support, and backend optimizations for CUDA, Metal, Vulkan, and others. Originally reported by GitHub.
New release
ggml 1.0.0 drops deprecated public API and legacy DIAG_MASK ops.
API changes
Bumped RPC protocol to v9 for faster loading and graceful failure on unreachable endpoints.
Core improvements
Scheduler skips uploading unused inputs and prefetches only inputs produced by earlier splits.
MoE cache
MoE cache computes cache misses on CPU and copies results back asynchronously.
Backend updates
Added few-row quantized matmuls and MoE fusion on CUDA/Metal, Vulkan prefill FlashAttention.
CUDA enhancements
Added few-row MMQ for batch sizes 2-24 and MMVF for BF16 few-row matmuls.
FP8 support
Added OCP FP8 support and fused NVFP4 gate/up matmuls with GLU in MMQ.
FlashAttention
Extended FlashAttention with sm70 head-size-256 config and MMA head sizes 40/72.
Metal improvements
Fused MoE routing and kept fusable node groups inside a single command buffer.
Vulkan updates
Added Intel FlashAttention prefill kernel and batched tensor copies.
SYCL changes
Added device-side MoE scheduler and cached device id to avoid VRAM overflows.
OpenCL additions
Added decomposed Adreno FlashAttention and improved decode performance.
Hexagon support
Added Q5_0/Q5_1 support and optimized conv_2d while adding conv_3d.
CPU optimizations
Switched MUL_MAT to F32 src1 and accumulate FlashAttention V in F32.
CANN updates
Fail early when a buffer type is unsupported.
ZDNN improvements
Accept non-zdnn tensors and invalidate compute tensors after each op.
What to remember
ggml releases v1.0.0 with pruned API, FP8 support, and backend optimizations for CUDA, Metal, Vulkan, and others.
