Skip to content
🌲🎉 We've officially partnered with Black Forest Labs to bring you the latest Flux.1 family of models.

Turbo registry

High-performance AI model loader

Mystic Turbo Registry is an optimized container loader which can reduce cold-start times by 90%, written in Rust.

SDXL (10GB Docker image)

with Turbo Registry

Hardware
allocation

Docker image
loading

0s

Running /
inference

✨

Others

Hardware
allocation

Docker image
loading

0s

Running /
inference

😔

Llama-70B (85GB Docker image)

with Turbo Registry

Hardware
allocation

Docker image
loading

30s

Running /
inference

✨

Others

Hardware
allocation

Docker image
loading

6min

Running /
inference

😔

Benchmarks

Image size
Turbo registry
Others
Speed increase
5GB image
10.23s
82.21s
x8
10GB image
14.72s
147.21s
x10
20GB image
23.72s
270.47s
x11.4

What are cold-starts?

We break down cold-start times into four main parts:

  1. Hardware allocation: How long a cloud provider takes to give you a fresh instance. This varies per instance type, region and cloud provider.

  2. Container downloading: How long downloading the image onto the fresh instance takes.

  3. Container extraction: Once downloaded, containers must be extracted and layers processed.

  4. Pipeline loading: This includes loading the ML model into GPU memory and the first inference pass. The next runs are faster because the model is already cached.

Why Turbo Registry?

Mystic Turbo Registry provides market leading container loading, enabling your models to run faster.

The problem

  • Downloading containers takes a very long time - a typical registry will allow you to download layers at ~150MB/s.

  • Once downloaded containers have to be extracted before being run, this can take minutes.

  • Cold start times mean servers are running for longer when not being used and cannot be shut down.

  • Your customers sit waiting for a response from a model.

The solution

  • After analyzing the 4 main processes in cold-starts, we found that loading containers onto hardware can be greatly optimized, allowing loading the model in a fraction of the time.

  • Rust, building the registry natively in a high performance language allowed us to increase the total download throughput substantially.