moonshotai
/

Kimi-K2-Instruct

Text Generation

Model card Files Files and versions

Instructions to use moonshotai/Kimi-K2-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use moonshotai/Kimi-K2-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="moonshotai/Kimi-K2-Instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("moonshotai/Kimi-K2-Instruct", trust_remote_code=True, device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use moonshotai/Kimi-K2-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "moonshotai/Kimi-K2-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K2-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/moonshotai/Kimi-K2-Instruct

How to use moonshotai/Kimi-K2-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "moonshotai/Kimi-K2-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K2-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "moonshotai/Kimi-K2-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K2-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use moonshotai/Kimi-K2-Instruct with Docker Model Runner:
```
docker model run hf.co/moonshotai/Kimi-K2-Instruct
```

Resources

View closed (30)

Add community evaluation results for SWE-Bench Multilingual

#74 opened 2 months ago by

Listed on OpenModelMap

#73 opened 3 months ago by

Listed on OpenModelMap

#72 opened 4 months ago by

TemporalMesh Transformer: 29.4 PPL at 48% compute — beats Mamba, new open-source architecture

#71 opened 4 months ago by

Add EvalEval community eval results

#70 opened 5 months ago by

Add Terminal-Bench evaluation result (27.8%)

#69 opened 6 months ago by

Create .eval_results/apex-agents.yaml

#68 opened 6 months ago by

Add converted tokenizer (no trust_remote_code needed)

#63 opened 9 months ago by

Should the chat template include <think> in the generation prompt for tool calling?

#61 opened 9 months ago by

Quality compare in IQ4_NL (582Gb RAM) with Q5_K_XLARGE (735Gb RAM) on $150 ancient Xeon PC from 2014

#55 opened about 1 year ago by

Question Regarding Muon optimizer over AdamW

#54 opened about 1 year ago by

Please, someone distill this model!

#53 opened about 1 year ago by

Update chat_template.jinja

#51 opened about 1 year ago by

Kimi-K2 Open-Source Model Tool Call Output Format Anomaly: Non-Standard tool_call_id Triggers Parsing Failures Compared with Official Mode

#48 opened about 1 year ago by

Please considering training a smaller dense variant in the same style.

#46 opened about 1 year ago by

Function Call Format Error in Multi-turn Dialogues

#45 opened about 1 year ago by

Chinese AI company true open source champion!

#43 opened about 1 year ago by

Can Kimi-k2 tokenizer please be added to transformers.js ??

#42 opened about 1 year ago by

lebronjames6michealjordan23

Function calling always outputs the <|tool_calls_section_begin|> tags.

#41 opened about 1 year ago by

Issue with Kimi K2 Model Support

#40 opened about 1 year ago by

jerin-scalers-ai

Model seems to reason even though it's an instruct model?

#37 opened about 1 year ago by

Amazing quality in Q4 on 2014 ANCIENT Xeon CPU with just shy 582Gb RAM

#34 opened about 1 year ago by

Really appreciate the work you put into this.🤍

#32 opened about 1 year ago by

What actually is the EOS token for this model?

#31 opened over 1 year ago by

web_search built-in tool via API

#29 opened over 1 year ago by

K2 reasoning

#27 opened over 1 year ago by

vllm nightly build + H200 only achieve Avg generation throughput: 7.2 tokens/

#25 opened over 1 year ago by

Good work - Great potential and current results

#24 opened over 1 year ago by

Model halucinate

#23 opened over 1 year ago by

Why not use PreTrainedTokenizerFast

#22 opened over 1 year ago by

Bug in multi run function-call

#21 opened over 1 year ago by

I may not know the maybe obvious answer to this but I'm curious.

#20 opened over 1 year ago by

Kimi K2 Great model, but performance vs. resource tradeoff?

#19 opened over 1 year ago by

How about hyper-fragmented sparse MoE ?

#18 opened over 1 year ago by

Add notebook examples for structured outputs and function calling

#17 opened over 1 year ago by

Question

#16 opened over 1 year ago by

PrincelyEndeavor

Run 1T-param on A100/H100(80G)x8 using FP4

#9 opened over 1 year ago by

Any plan to release a Vision enabled version with the same or near the same base and instruct model?

#7 opened over 1 year ago by

Synthetic data generation

#6 opened over 1 year ago by

Multimodality

#5 opened over 1 year ago by

Thorough Testing Video of Kimi K2 - Step by Step

#4 opened over 1 year ago by

Can you provide Machine Specs

#2 opened over 1 year ago by