Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
tinyllama-proteinpretrain-quinoa – AI Model by monsoon-nlp | AlphaNeural AI
You can deploy this model and start earning money today!
monsoon-nlp
/
tinyllama-proteinpretrain-quinoa
like
0
transformers
safetensors
llama
text-generation
en
cerebras/SlimPajama-627B
bigcode/starcoderdata
monsoon-nlp/greenbeing-proteins
TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
finetune
apache-2.0
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
tinyllama-proteinpretrain-quinoa
Full model finetuning of TinyLLaMA-1.1B on the "research" split (quinoa protein sequences) of GreenBeing-Proteins dataset.
Notes: pretraining only on sequences leads the model to only generate protein sequences, eventually repeating VVVV ot KKKK.
This model may be replaced with mixed training (bio/chem text and protein).
This model might need "biotokens" to represent the amino acids instead of using the existing tokenizer.
More details TBD
