Skip to content

Models and ONNX

Where the bundled models come from, how they’re downloaded, and how their integrity is checked.

Cortrix runs two local models. Here’s where they come from, how they’re verified at startup, and where the files live.

Models

A Docker deployment downloads two families of local neural models into the cortrix-data volume. They aren’t baked into the image.

ModelRoleDownload sizePinned sourceUpstream and license
BGE-M3Embedding585,562,194 bytesonnx-community/bge-m3-ONNX@25b9af8BAAI/bge-m3@5617a9f, MIT
bge-reranker-v2-m3Reranking587,809,994 bytesonnx-community/bge-reranker-v2-m3-ONNX@6f5ff65BAAI/bge-reranker-v2-m3@953dc6f, Apache-2.0

The onnx-community repositories are format-conversion sources for the BAAI upstream revisions listed above. Cortrix doesn’t claim authorship of these models.

These models are independent of any LLM configuration. With no LLM role configured, embedding and reranking work as usual.

Model manifest

deploy/model-manifest.tsv is the authoritative, machine-readable record of model identity. For every model and tokenizer file it records:

  • The repository, revision, and file path
  • The expected byte size and SHA-256
  • The upstream repository, upstream revision, and upstream license

The manifest lists four files:

ComponentInstalled at
embedding_modelbge-m3/model.onnx
embedding_tokenizerbge-m3/tokenizer.json
reranker_modelbge-reranker-v2-m3/model.onnx
reranker_tokenizerbge-reranker-v2-m3/tokenizer.json

Download and verification at startup

On the first start, the container downloads the manifest’s files into /data/models. The downloader:

  1. Fetches the exact revision and path in the manifest.
  2. Rejects symbolic links.
  3. Checks the byte size and SHA-256.
  4. Installs each file atomically.

If anything doesn’t match, or a download fails, the container doesn’t report ready. Readiness also requires both models to finish loading and the demo data to finish importing. The quickstart never falls back silently to a stub embedder or reranker.

Later starts reuse the models already in the volume.

Start without access to Hugging Face

The first start downloads the model files from Hugging Face. On a machine that can’t reach it, copy the files in before the first start. The downloader verifies every file it finds in /data/models against the manifest and downloads only what is missing or doesn’t match, so a complete, valid copy means nothing is downloaded.

  1. On a machine where the quickstart has already started once, copy the model directory out of the container. The container can be stopped. Make sure ./models doesn’t exist yet, or the files land one level too deep:

    docker cp "$(docker compose -f deploy/docker-compose.yml ps -aq cortrix)":/data/models ./models

    The models directory holds bge-m3/, bge-reranker-v2-m3/, and MANIFEST.tsv, about 1.17 GB in total.

  2. Move the models directory to the machine that has no access.

  3. On that machine, create the container without starting it, copy the directory in, hand the files to the user the service runs as, and then start:

    docker compose -f deploy/docker-compose.yml create
    docker cp ./models/. "$(docker compose -f deploy/docker-compose.yml ps -aq cortrix)":/data/models/
    docker compose -f deploy/docker-compose.yml run --rm --no-deps --user root \
      --entrypoint chown cortrix -R cortrix:cortrix /data/models
    docker compose -f deploy/docker-compose.yml up --wait

    docker cp creates the copied files as root, and the service runs as cortrix. Without the chown, the service can read the models but can never replace one, so the container would exit at start the first time a file fails verification or a later release pins a different model revision.

The log shows that the cached files were verified and nothing was downloaded:

[provision] verifying pinned embedding and reranker assets
[models] verified cached embedding_model
[models] verified cached embedding_tokenizer
[models] verified cached reranker_model
[models] verified cached reranker_tokenizer
[models] all pinned embedding and reranker assets verified

A copied file that fails the size or SHA-256 check isn’t used. The downloader tries to fetch it again, and without network access it can’t download a replacement, so the container exits without starting the service. Copy the directory again from a machine where the quickstart reports ready.

Note

Building the image still needs network access, for the base images and the build dependencies. This section covers only the model download at first start.

Remove the cached models

docker compose -f deploy/docker-compose.yml down --volumes
Caution

This deletes the whole volume, including your data, not just the models.

Model settings when you run from source

When you run from source, the configuration file sets the model paths:

config.yaml
embedding:
  model_path: "./models/bge-m3/model.onnx"
  tokenizer_path: "./models/bge-m3/tokenizer.json"
  dimension: 1024
  max_seq_length: 512

reranker:
  model_dir: "./models/bge-reranker-v2-m3"
  • dimension is fixed at 1024 and can’t be changed.
  • An empty embedding.model_path is the only explicit stub mode. If the path is set but the model is missing or invalid, the server fails to start instead of degrading to a stub.
  • An empty reranker.model_dir explicitly selects the stub reranker. If the path is set, the model and tokenizer must be valid.

What the quickstart doesn’t include

The Docker quickstart covers only text files, BGE-M3 embedding, and bge-reranker-v2-m3 reranking. It doesn’t install or validate PDF, DOCX, images, OCR, query-complexity routing, external LLM roles, or the built-in Agent. Those are separate features. Check Compatibility and status before you rely on them.

Upgrade ONNX Runtime

The repository pins the ONNX Runtime version. The steps and verification requirements for upgrading it are written for maintainers. See ONNX Runtime upgrade and rollback.

Next steps

Table reader