Welcome
CosmicAC is a self-hosted platform for running GPU workloads on your Kubernetes cluster. It runs GPU Container Jobs and serves open source language models.
CosmicAC is a self-hosted platform for running GPU workloads. You deploy it on your host machine. It runs GPU Container Jobs and serves open source language models on your Kubernetes cluster.
Get started
Deploy CosmicAC on your host machine, install the CLI, and create your first job.
Installation
Deploy CosmicAC on your host machine to run GPU workloads on your Kubernetes cluster.
Install the CLI
Install the CosmicAC CLI to run jobs and manage resources from your terminal.
Create a GPU Container Job Using the CLI
Launch a GPU Container Job with the CLI and open a shell into it.
Create a vLLM Managed Inference Job Using the CLI
Deploy and serve a model behind an OpenAI-compatible endpoint.
GPU Container Jobs
Work on a GPU machine like a remote computer, with shell access.
Managed Inference
Serve open source language models behind an OpenAI-compatible API.
Overview
What a Managed Inference Job is, when to use one, and how you call it.
Create a Managed Inference Job Using the Web UI
Serve a model behind an OpenAI-compatible endpoint from the web interface.
Connect to a vLLM Managed Inference endpoint
Create an API key and call the model from a client or the CLI.
Model masters
List, add, update, and archive the default serving parameters for each model.
Platform
Understand how CosmicAC runs on your cluster, and keep the deployment running.
Reference
Look up the commands, routes, fields, and values CosmicAC accepts, and what changed in each release.
CLI commands
Every CosmicAC CLI command, with usage, arguments, and options.
Task deployment commands
Every task command that deploys, upgrades, and operates the stack.
API reference
The inference, monitor, and observability settings HTTP routes.
Deployment configuration
Deployment environment variables and kubeconfig requirements.
Job configuration
Fields you set when you create a GPU Container Job or a Managed Inference Job.
Recommended model parameters
Recommended serving parameters for supported models.
Changelog
New features, changes, and fixes in each release.