The truncation problem

When you embed a 4,000-word document with a 512-token model, only the first 380 words get embedded. The rest vanishes, and nothing warns you. Content buried past the window can never be retrieved.

“The model read the first 380 words and threw away the other 3,600.”

Chunking is now built in

Add chunk_strategy to your vector column definition. Manticore splits each document into chunks, embeds every chunk, and searches all of them. No pipeline, no library, no second table needed.

Five chunking strategies

Truncate (default, one vector), mean (one vector, whole document), fixed (predictable windows), recursive (respects structure), sentence (complete thoughts). Choose based on your document type and needs.

Truncate for short text

Keep truncate for product titles, descriptions, tags, log lines, chat messages, anything under 380 words. It's the fastest and smallest. When your 95th-percentile document fits the model's window, stop reading.

Mean for single-topic documents

Splits the document, embeds every chunk, averages them into one vector. Same storage and search cost as truncate, but nothing is thrown away. Use when you can't afford multiple vectors or the column is already float_vector.

Fixed for unstructured text

Cuts every max_tokens tokens regardless of content. Chunk count is predictable, so index size is knowable before you load anything. Use for OCR, scraped HTML, machine transcripts, log dumps.

Recursive for prose

Respects natural boundaries: blank lines first, then line breaks, then sentences, then spaces. Same token budget as fixed but chunks stop where text stops. Best general default for documentation, wikis, READMEs.

Sentence for complete thoughts

Detects sentence boundaries, packs whole sentences until the token budget is reached. A chunk never starts or ends mid-sentence. Use for support tickets, email threads, transcripts, legal text, anything feeding to an LLM.

Three tuning knobs

max_tokens sets chunk size (128-512 for English prose). overlap_tokens repeats tail of each chunk at head of next (10-20% of max_tokens). max_chunks caps vectors per document to guard against outliers.

Search stays simple

No chunk table, no nested field, no join, no GROUP BY. A document is still one search result. Chunks compete individually, and Manticore returns the document once, scored by its best chunk.

Real-world test results

On the Manticore manual (189 pages, 298k words), recall@5 for deep content jumped from 55.1% to 83.3%, and MRR from 0.44 to 0.70. Cost: 2.5x RAM, 4x ingest time.

“recall@5 went from 55.1% → 83.3% and MRR from 0.44 → 0.70”

Smaller chunks win

Going from 512 to 128 tokens buys five points of recall and a large jump in ranking quality. It costs 4x the vectors and 2.5x the index RAM. Below 128 tokens, chunks stop containing a whole thought.

Overlap doesn't help much

At 128 tokens, adding 25% overlap moved recall from 85.2% to 85.2% while adding 41% more vectors and 5 MB of RAM. Overlap earns its keep only when facts routinely straddle boundaries.

Test your own settings

Create multiple vector columns with different strategies in one table. Load your corpus once, run the same query against each column, and compare recall@5. Then drop the losing column.

Documentation search recipe

Use recursive chunking with max_tokens 192 on title and body joined together. The title lands in the first chunk and gives it context. Works for Markdown pages and user questions.

Support tickets recipe

Use sentence chunking with max_tokens 256, overlap 32, and max_chunks 64. Threads have no natural length limit, so cap the vectors. Filtering works exactly as before.

RAG retrieval recipe

Use sentence chunking with max_tokens 512 and overlap 64. Larger chunks read as prose when pasted into prompts. Ask for more results since chunks are bigger.

Chunking doesn't fix everything

A chunk doesn't know where it came from. Splitting a document loses context about surrounding text. Prepending chunk-specific descriptions before embedding cuts retrieval failures by 35-49%, but Manticore doesn't do this automatically.

Manticore vs other engines

Most vector engines now embed in-engine, but few chunk in-engine. Elasticsearch, OpenSearch, and Azure AI Search require separate pipeline stages. Manticore handles chunking as a column option with one result per document.