Text Generation
Transformers
Safetensors
Undetermined
indus-script
ancient-scripts
archaeology
nlp
sequence-modeling
grammar-analysis
undeciphered-script
Instructions to use hellosindh/indus-script-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hellosindh/indus-script-models with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hellosindh/indus-script-models")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("hellosindh/indus-script-models", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hellosindh/indus-script-models with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hellosindh/indus-script-models" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hellosindh/indus-script-models", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/hellosindh/indus-script-models
- SGLang
How to use hellosindh/indus-script-models with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hellosindh/indus-script-models" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hellosindh/indus-script-models", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hellosindh/indus-script-models" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hellosindh/indus-script-models", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use hellosindh/indus-script-models with Docker Model Runner:
docker model run hf.co/hellosindh/indus-script-models
Update inference.py
Browse files- inference.py +3 -2
inference.py
CHANGED
|
@@ -299,7 +299,8 @@ def task_validate(seq_str, models):
|
|
| 299 |
|
| 300 |
def task_predict(seq_str, models):
|
| 301 |
tok, cls, ngram, elec_tok, elec_disc, glyph_map = models
|
| 302 |
-
|
|
|
|
| 303 |
seq = parse_sequence(seq_str)
|
| 304 |
preds = bert_predict_mask(seq, tok, mlm, top_k=5)
|
| 305 |
print(f"\n Input: {seq_str}")
|
|
@@ -406,4 +407,4 @@ def main():
|
|
| 406 |
|
| 407 |
|
| 408 |
if __name__ == "__main__":
|
| 409 |
-
main()
|
|
|
|
| 299 |
|
| 300 |
def task_predict(seq_str, models):
|
| 301 |
tok, cls, ngram, elec_tok, elec_disc, glyph_map = models
|
| 302 |
+
model_dir, data_dir = get_model_dir()
|
| 303 |
+
mlm = load_bert_mlm(model_dir)
|
| 304 |
seq = parse_sequence(seq_str)
|
| 305 |
preds = bert_predict_mask(seq, tok, mlm, top_k=5)
|
| 306 |
print(f"\n Input: {seq_str}")
|
|
|
|
| 407 |
|
| 408 |
|
| 409 |
if __name__ == "__main__":
|
| 410 |
+
main()
|