Temperature and top_p values are swapped in the example code:

#2
by sszymczyk - opened

At least comparing to the values recommended in the model card. We have:

top_p = 0.95
top_k = 50
min_p = 0.0
temperature = 0.8

But in the code there is:

response = client.chat.completions.create(
    model=model_name,
    messages=messages,
    max_tokens=32768,
    temperature=0.95,
    top_p=0.8,
    extra_body=dict(spaces_between_special_tokens=False),
    tools=tools)

Also I see that in generation_config.json there's another set of values:

    "top_p": 0.8,
    "top_k": 20,
    "temperature": 0.7,

Sign up or log in to comment