image/pngFurther into the future. Better. New achievement: Zeta 2.
Zeta 2 is a new version of Zeta that aims to take the next step forward from Zeta 1, which proved that "ordinary consumers can create LLMs."
Quantized Model (GGUF)
About Zeta 2
Zeta 2 is an LLM with about 460 million parameters.
It might be better to call it an SLM.
Zeta 2 is an SLM that was painstakingly created on a consumer computer.
Prompt Template
{{ if .System }}{{ .System }}{{ end }}
{{ if .Prompt }}<USER>{{ .Prompt }}</USER>{{ end }}
<ASSISTANT>
Computer Spec
Machine: Mac Mini (M2 Pro, 32GB RAM, 2023)
This is a technical matter, but on a Mac with Apple Silicon you can only choose between CPU or MPS (Apple's proprietary API).
CPU has some compatibility but is too slow.
MPS is somewhat faster but doesn't allow the use of optimization systems such as fp16.
Zeta 2 was built using MPS.
Arguments of Trainer
- train epochs: 3
- warmup steps: 100
Datasets
Details of the dataset used can be found
hereLinks
Zeta is just a small SLM.
But don't forget that it has big dreams inside.
