Templates

A template stores a launch configuration: an image, an instance type, a startup command and, optionally, a fixed resource pool and size. The Gallery and the ready-made templates under Deploy type: Model API on Home are templates.

Except for browsing public templates, these endpoints accept only a browser session. Paths are relative to your site’s console address.

Browse public templates

GET /api/templates None, API key or session

Returns published templates. With an API key or a session, it also returns your own private templates.

List model service templates

GET /api/inference/templates Session

Returns the public vLLM templates that Deploy type → Model API offers, each with the command it runs and the size it needs.

{
  "templates": [
    {
      "id": 412,
      "title": "MiniCPM5-1B",
      "description": "...",
      "model_source": "modelscope",
      "start_command": "vllm serve OpenBMB/MiniCPM5-1B --host 0.0.0.0 --port 8000 --served-model-name MiniCPM5-1B",
      "pod_type": "one-click",
      "resource_template": "resource-16c55g1u",
      "gpu_count": 1
    }
  ]
}

Launch a template

POST /api/templates/{template_id}/launch Session

The console sends an empty body. A template with a fixed pool and size launches on that size.

Field	Type		Description
gpu_count	integer	Optional	Only for templates without a fixed size. Defaults to 1.
ssh_enabled	boolean	Optional	Turn on SSH for this launch.

Only jupyter and vllm templates can be launched. The response and the follow-up calls are the same as for any other launch: poll GET /api/notebook/status, or GET /api/inference/current for a vllm template.

Launches are rate limited per IP address. Going over returns 429 with a Retry-After header.

Manage your templates

Method	Path	Does
GET	/api/profile/templates	Lists your templates and the images you can pick.
POST	/api/profile/templates	Creates a template.
PUT	/api/profile/templates/{template_id}	Replaces a template. Same body as creating.
DELETE	/api/profile/templates/{template_id}	Deletes a template.

The main fields:

Field	Type		Description
title	string	Required	Display name.
image	string	Required	An image from the catalog.
instance_type	string	Optional	jupyter or vllm. Other types can be saved but can’t be launched.
start_command	string	Optional	Command run at startup. Required for vllm.
model_source	string	Optional	huggingface or modelscope, for vllm.
pod_type, resource_template	string	Optional	A fixed pool and size. Send both or neither.
use_pvc	boolean	Optional	Launch with your persistent /workspace.
description, category, tags, cover_url		Optional	How the template appears in the Gallery.
repo_url, branch, notebook_path	string	Optional	A Git repository to clone and a notebook to open on start.
enabled	boolean	Optional	Publish to everyone. Only editors can publish; for everyone else it’s forced to false.

Changing or deleting a template doesn’t affect an instance already running from it.