nkpz/

falcon-thought-7b-v0-1

Fine-tune of https://huggingface.co/suayptalha/Falcon3-Jessi-v0.4-7B-Slerp using a custom training script + custom optimizer.

This is a variant of https://huggingface.co/nkpz/falcon-thought-7b-v0 and it isn't hugely different.

The one big difference from v0 is that I used NLL loss instead of an experimental AI-generated loss function. v0-1 had a much smoother loss curve for this reason.

It still gets "I"/"you" confused pretty frequently or generates <|eot_id|> prematurely. Keep expectations low in terms of stability and correctness (it hallucinates), but it's capable of reasoning in a roleplay context which is cool.