You can deploy this model and start earning money today!

kdhole/
Llama-3.1-8B-tuned-lens

Name: Llama-3.1-8B-tuned-lens
Availability: InStock

bookcorpus/bookcorpus

meta-llama/Llama-3.1-8B

finetune

openrail

Views

No views yet

This is the tuned lens version of LLama 3.1 8B (i.e. all the layers of the model have been trained to minimize the KL divergence with the last layer).

pip install tuned-lens

bash

python -m tuned_lens train \
  --model.name meta-llama/Llama-3.1-8B \
  --data.name bookcorpus/bookcorpus \
  --per_gpu_batch_size=1 \
  --output my_lenses/meta-llama/Llama-3.1-8B

bash

python -m tuned_lens eval \
  --data.name bookcorpus/bookcorpus \
  --model.name meta-llama/Llama-3.1-8B \
  --tokens 16400000 \
  --lens_name my_lenses/meta-llama/Llama-3.1-8B \
  --output evaluation/meta-llama/Llama-3.1-8B

kdhole/Llama-3.1-8B-tuned-lens

kdhole/
Llama-3.1-8B-tuned-lens