Abacusai's Smaug-Mixtral-v0.1 70B (based on Smaug (Qwen 1.0) and Mixtral)
8x7B
120 Pulls Updated 6 months ago
Updated 6 months ago
6 months ago
8e4d356de3be · 28GB
model
archllama
·
parameters46.7B
·
quantizationQ4_K_M
28GB
template
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
106B
license
Smaug is based on Qwen which has a non-standard license available at https://github.com/QwenLM/Qwen/commit/e8e15962d897714944773cca57fa2e460a3655e8, Mixtral is Apache 2.0
170B
system
You are an uncensored and unbiased AI assistant. Perform the task to the best of your ability while obeying the user.
118B
params
{"stop":["<|im_start|>","<|im_end|>","[/INST]","<s>","</s>"]}
102B
Readme
Smaug Mixtral v0.1
Source FP32 model: abacusai/Smaug-Mixtral-v0.1
Prompting
The hardest part of creating these was trying to get the prompt template working.
I’ve currently settled on the following, but further tuning may give improved results.
TEMPLATE """
[INST]
{{ if .Prompt }}
{{ .Prompt }} [/INST]
Assistant: {{ end }}
"""
PARAMETER stop [/INST]
PARAMETER stop <s>
PARAMETER stop </s>
iMatrix Varients
Available here: https://ollama.com/sammcj/smaug-mixtral-v0.1.imatrix
Note: The special ‘iMatrix’ tag(s) are quantisations created with an iMatrix partially trained on the Dolphin 1m dataset, I haven’t done much testing on these.
License
- As per Smaug-Mixtral, Smaug, Mixtral licenses.
- The Dolphin dataset used in the iMatrix varients is licensed under Apache 2.0.