Instructions to use Dongwei/Rationalyst_reasoning_datasets with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Dongwei/Rationalyst_reasoning_datasets with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Dongwei/Rationalyst_reasoning_datasets")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Dongwei/Rationalyst_reasoning_datasets")
model = AutoModel.from_pretrained("Dongwei/Rationalyst_reasoning_datasets")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Dongwei/Rationalyst_reasoning_datasets with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Dongwei/Rationalyst_reasoning_datasets"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dongwei/Rationalyst_reasoning_datasets",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Dongwei/Rationalyst_reasoning_datasets

SGLang

How to use Dongwei/Rationalyst_reasoning_datasets with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Dongwei/Rationalyst_reasoning_datasets" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dongwei/Rationalyst_reasoning_datasets",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Dongwei/Rationalyst_reasoning_datasets" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dongwei/Rationalyst_reasoning_datasets",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Dongwei/Rationalyst_reasoning_datasets with Docker Model Runner:
```
docker model run hf.co/Dongwei/Rationalyst_reasoning_datasets
```

Rationalyst

This model is a fine-tuned version of the LLaMa-3-Instruct-8B. It was introduced in RATIONALYST: Pre-training Process-Supervision for Improving Reasoning. The code for the rationale extraction, model training, and inference can be found here.

Model description

Implicit rationales are often embedded in the unlabelled text, reflecting the natural thought processes behind speech and writing. RATIONALYST is a self-supervised approach to extract and filter these implicit rationales from unlabelled text and apply them to supervise reasoning.

How to use

To use it, simply input question and partial reasoning trajectory, and the model will output the rationale to supervise the next reasoning step.

Training data

This Rationalyst is trained using 65k implicit rationales from The Pile and 14k implicit rationales from GSM8K and ECQA. The data used can be found here

Evaluation results

When used to evaluate on downstream tasks, this model achieves the following results:

Task	GSM8K	MATH	ECQA	HellaSwag	ProofWriter	ARC	MMLU-Pro
	81.6	32.5	75.2	60.3	90.7	80.7	45.3

Downloads last month: 34

Safetensors

Model size

8B params

Tensor type

F32

Paper for Dongwei/Rationalyst_reasoning_datasets

RATIONALYST: Pre-training Process-Supervision for Improving Reasoning

Paper • 2410.01044 • Published Oct 1, 2024 • 35