Models and ONNX
Cortrix runs two local models. Here’s where they come from, how they’re verified at startup, and where the files live.
Models
A Docker deployment downloads two families of local neural models into the cortrix-data volume. They aren’t baked into the image.
| Model | Role | Download size | Pinned source | Upstream and license |
|---|---|---|---|---|
| BGE-M3 | Embedding | 585,562,194 bytes | onnx-community/bge-m3-ONNX@25b9af8 | BAAI/bge-m3@5617a9f, MIT |
| bge-reranker-v2-m3 | Reranking | 587,809,994 bytes | onnx-community/bge-reranker-v2-m3-ONNX@6f5ff65 | BAAI/bge-reranker-v2-m3@953dc6f, Apache-2.0 |
The onnx-community repositories are format-conversion sources for the BAAI upstream revisions listed above. Cortrix doesn’t claim authorship of these models.
These models are independent of any LLM configuration. With no LLM role configured, embedding and reranking work as usual.
Model manifest
deploy/model-manifest.tsv is the authoritative, machine-readable record of model identity. For every model and tokenizer file it records:
- The repository, revision, and file path
- The expected byte size and SHA-256
- The upstream repository, upstream revision, and upstream license
The manifest lists four files:
| Component | Installed at |
|---|---|
embedding_model | bge-m3/model.onnx |
embedding_tokenizer | bge-m3/tokenizer.json |
reranker_model | bge-reranker-v2-m3/model.onnx |
reranker_tokenizer | bge-reranker-v2-m3/tokenizer.json |
Download and verification at startup
On the first start, the container downloads the manifest’s files into /data/models. The downloader:
- Fetches the exact revision and path in the manifest.
- Rejects symbolic links.
- Checks the byte size and SHA-256.
- Installs each file atomically.
If anything doesn’t match, or a download fails, the container doesn’t report ready. Readiness also requires both models to finish loading and the demo data to finish importing. The quickstart never falls back silently to a stub embedder or reranker.
Later starts reuse the models already in the volume.
Start without access to Hugging Face
The first start downloads the model files from Hugging Face. On a machine that can’t reach it, copy the files in before the first start. The downloader verifies every file it finds in /data/models against the manifest and downloads only what is missing or doesn’t match, so a complete, valid copy means nothing is downloaded.
On a machine where the quickstart has already started once, copy the model directory out of the container. The container can be stopped. Make sure
./modelsdoesn’t exist yet, or the files land one level too deep:The
modelsdirectory holdsbge-m3/,bge-reranker-v2-m3/, andMANIFEST.tsv, about 1.17 GB in total.Move the
modelsdirectory to the machine that has no access.On that machine, create the container without starting it, copy the directory in, hand the files to the user the service runs as, and then start:
docker cpcreates the copied files as root, and the service runs ascortrix. Without thechown, the service can read the models but can never replace one, so the container would exit at start the first time a file fails verification or a later release pins a different model revision.
The log shows that the cached files were verified and nothing was downloaded:
A copied file that fails the size or SHA-256 check isn’t used. The downloader tries to fetch it again, and without network access it can’t download a replacement, so the container exits without starting the service. Copy the directory again from a machine where the quickstart reports ready.
Building the image still needs network access, for the base images and the build dependencies. This section covers only the model download at first start.
Remove the cached models
This deletes the whole volume, including your data, not just the models.
Model settings when you run from source
When you run from source, the configuration file sets the model paths:
dimensionis fixed at 1024 and can’t be changed.- An empty
embedding.model_pathis the only explicit stub mode. If the path is set but the model is missing or invalid, the server fails to start instead of degrading to a stub. - An empty
reranker.model_direxplicitly selects the stub reranker. If the path is set, the model and tokenizer must be valid.
What the quickstart doesn’t include
The Docker quickstart covers only text files, BGE-M3 embedding, and bge-reranker-v2-m3 reranking. It doesn’t install or validate PDF, DOCX, images, OCR, query-complexity routing, external LLM roles, or the built-in Agent. Those are separate features. Check Compatibility and status before you rely on them.
Upgrade ONNX Runtime
The repository pins the ONNX Runtime version. The steps and verification requirements for upgrading it are written for maintainers. See ONNX Runtime upgrade and rollback.