Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
MistralKashaloti_v0.1 – AI Model by noxneural | AlphaNeural AI
You can deploy this model and start earning money today!
noxneural
/
MistralKashaloti_v0.1
like
0
peft
pytorch
mistral
sq
noxneural/kashaloti
Open-Orca/Mistral-7B-SlimOrca
adapter
mit
8-bit
bitsandbytes
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Model Card for Model ID
Model Details
Version 1 of 71, qLora finetune of the Mistral-7B model using 1/71 of the GPT4 based part of the Orca Dataset, using approx ~14k records from a total of 1 million records.
Model Description
Developed by:
Marlind Maksuti @ StochastX
Model type:
Mistral-7B
Language(s) (NLP):
Albanian, Shqip
Finetuned from model:
Mistral-7B-SlimOrca
Model Sources
Repository:
https://huggingface.co/Open-Orca/Mistral-7B-SlimOrca
Uses
Text generation in Albanian.
Bias, Risks, and Limitations
Model is just version v0.1, outputs are still not optimal.
Training Hyperparameters
Training regime:
[More Information Needed]
Training procedure
The following
bitsandbytes
quantization config was used during training:
quant_method: QuantizationMethod.BITS_AND_BYTES
load_in_8bit: True
load_in_4bit: False
llm_int8_threshold: 6.0
llm_int8_skip_modules: None
llm_int8_enable_fp32_cpu_offload: False
llm_int8_has_fp16_weight: False
bnb_4bit_quant_type: nf4
bnb_4bit_use_double_quant: True
bnb_4bit_compute_dtype: float16
Framework versions
PEFT 0.6.0.dev0
Training procedure
The following
bitsandbytes
quantization config was used during training:
quant_method: QuantizationMethod.BITS_AND_BYTES
load_in_8bit: True
load_in_4bit: False
llm_int8_threshold: 6.0
llm_int8_skip_modules: None
llm_int8_enable_fp32_cpu_offload: False
llm_int8_has_fp16_weight: False
bnb_4bit_quant_type: nf4
bnb_4bit_use_double_quant: True
bnb_4bit_compute_dtype: float16
Framework versions
PEFT 0.6.0.dev0
