Developed by: Solshine (Caleb DeLeeuw)
License: apache-2.0
Finetuned from model : inceptionai/jais-adapted-7b-chat ( after quantization transformation into Solshine/jais-adapted-7b-chat-Q4_K_M-GGUF )
Dataset: CopyleftCultivarinceptionai/jais-adapted-7b-chats/Natural-Farming-Real-QandA-Conversations-Q1-2024-Update (Real world Natural Farming advise, from over 12 countries and a multitude of real-world farm operations, using semi-synthetic data curated by domain experts)
V4 (best training loss curve of unsloth configs tested) of LORA adapter trained, merged into this quantized gguf.
This llama model was trained 2x faster with
Unsloth and Huggingface's TRL library.
==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1
\ /| Num examples = 169 | Num Epochs = 2
O^O/ _/ \ Batch size per device = 2 | Gradient Accumulation steps = 4
\ / Total batch size = 8 | Total steps = 38
"-____-" Number of trainable parameters = 39,976,960
[38/38 03:29, Epoch 1/2]
Step Training Loss
1 2.286800
2 2.205600
3 2.201700
4 2.158100
5 2.021100
6 1.820200
7 1.822500
8 1.565700
9 1.335700
10 1.225900
11 1.081000
12 0.947700
13 0.828600
14 0.830200
15 0.796300
16 0.781200
17 0.781600
18 0.815000
19 0.741400
20 0.847600
21 0.736600
22 0.714300
23 0.706400
24 0.752800
25 0.684600
26 0.647800
27 0.775300
28 0.613800
29 0.679500
30 0.752900
31 0.589800
32 0.729400
33 0.549500
34 0.638500
35 0.609500
36 0.632200
37 0.686400
38 0.724200
