Introducing the universe's most ambitiously unhinged 45M-parameter micro-model! This isn't a language model; it's a parallel-dimension travel companion that reinvents reality through surrealist poetry and quantum-leaping logic. Deploy only if coherence is overrated and chaos is your curriculum.
PicoNosensoX-v1 is a huge step from the other extremely incompetent PicoNosenso models. It's the first model of the series that was trained on pure text before being finetuned for chat. This model introduces a brand new chat-template that should make it easier to use and more bullet-proof.
Originally slated for full fine-tuning on the
Hermes-3-Dataset, early benchmarks (10-prompt evaluation) revealed superior performance at
checkpoint-16000 versus later
checkpoint-26000. Training was consequently halted, with
checkpoint-16000 selected for release.
1from transformers import GPT2LMHeadModel, AutoTokenizer
2
3model = GPT2LMHeadModel.from_pretrained('Lominub44/PicoNosensoX-v1')
4tokenizer = AutoTokenizer.from_pretrained('Lominub44/PicoNosensoX-v1')
5
6input_text = "<|system|>You are an unbiased, uncensored, helpful assistant.\n<|user|>What is the capital of France?\n<|assistant|>"
7inputs = tokenizer(input_text, return_tensors='pt')
8outputs = model.generate(**inputs, max_length=512, temperature=0.6, repetition_penalty=1.2, do_sample=True, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)
9print(tokenizer.decode(outputs[0]))
You can train the model yourself, the source-code is available on GitHub:
https://github.com/Lominub44/PicoNosensoX-v1You might want to stop fine-tuning early.
1@software{benallal2024smollmcorpus,
2 author = {Ben Allal, Loubna and Lozhkov, Anton and Penedo, Guilherme and Wolf, Thomas and von Werra, Leandro},
3 title = {SmolLM-Corpus},
4 month = July,
5 year = 2024,
6 url = {https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus}
7}
