Gigatoken

Docs

Overview

Gigatoken Overview

Gigatoken connects provider-owned hardware running local models with inference consumers who use OpenAI-compatible access across active provider devices.

How Gigatoken works

Providers run the Gigatoken CLI and daemon on supported machines. The daemon connects to the coordinator, keeps the local device in sync, and applies model intent chosen from the Provider console.

From the console, providers choose catalog-managed models to download, keep present, and make active. Those GGUF models are served locally from the provider device. When a model fits with less than 4 GiB of VRAM headroom, the console warns that the GPU must be effectively dedicated and idle before Download, Retry, or Set active proceeds.

Inference consumers create inference API keys and send requests to OpenAI-compatible endpoints. Gigatoken routes eligible requests to active devices in the provider pool that can serve the selected model.

What the current product supports

  • Provider devices connect through the Gigatoken CLI and daemon.
  • Catalog-managed local models can be downloaded, retried, kept present, and made active from the Provider console when the device reports enough usable VRAM; Download, Retry, and Set active may show a dedicated-fit advisory when the device has less than 4 GiB of headroom.
  • Inference consumers create API keys for OpenAI-compatible access.
  • Requests are routed across active provider devices that can serve the selected model.
  • This documentation can be read without an account or active session.

FAQ

Short answers for readers who are new to Gigatoken.

What is Gigatoken?

Gigatoken connects provider-owned hardware running local models with inference consumers who want OpenAI-compatible access across active provider devices.

Who is Gigatoken for?

It is for providers who want to connect real hardware and manage local model availability, and for inference consumers who need API keys that route requests through the provider pool.

Is the inference API OpenAI-compatible?

Yes. Inference consumers use OpenAI-compatible endpoints and inference API keys, while Gigatoken routes eligible requests to active provider devices.

What hardware and models are supported?

Gigatoken is built around supported provider machines running catalog-managed GGUF models served locally. Each catalog model has a minimum usable VRAM requirement; devices with unknown VRAM fail closed until the provider sets GIGATOKEN_VRAM_GB and reconnects. Eligible models close to the minimum VRAM floor can show a dedicated-fit confirmation before provider actions.

Do I need to sign in to read the docs?

No. The docs are public and sign-in-free. Signing in is required for console workflows such as managing provider devices, models, and inference keys.