Views
No views yet
<|startoftext|><|im_start|>user
What is C. elegans?<|im_end|>
<|im_start|>assistant
Ahoy, matey! You be talkin' 'bout the little scurvy dog, eh? Well, shiver me timbers, that's a fine question!
C. elegans, they say, be a tiny, scurvy dog, about the size of a thumbnail, but they're as smart as a kraken's tentacles and as clever as a pirate's treasure map. They be a little worm, but they've got a brain as big as a ship's mast and they can navigate through treacherous waters with ease.
They be used in all sorts of experiments, like a pirate's treasure hunt, to learn more about how life works. They be like a puzzle, a puzzle, and a treasure chest all rolled into one! So, if you want to know more about these little creatures, just keep your eyes peeled, and maybe even get a bit closer, matey!
<|im_end|>1base_model: LiquidAI/LFM2-1.2B
2xformers_attention: true
3sample_packing: true
4chunked_cross_entropy: true
5learning_rate: 1e-5
6sequence_len: 16384
7micro_batch_size: 1
8gradient_accumulation_steps: 2
9gradient_checkpointing: true
10optimizer: adamw_torch_8bit
11lr_scheduler: cosine
12warmup_ratio: 0.2
13float16: true
14bf16: false
15max_grad_norm: 0.1
16num_epochs: 3
17saves_per_epoch: 1
18logging_steps: 5
19output_dir: ./outputs/lfm2-sft-pirate-rrr
20chat_template: tokenizer_default
21datasets:
22 - path: winglian/pirate-ultrachat-10k
23 type: chat_template
24 split: train
25eot_tokens:
26 - "<|im_end|>"
27dataloader_prefetch_factor: 8
28dataloader_num_workers: 2
29dataloader_pin_memory: true