Developed by: Solshine (Caleb DeLeeuw)
License: apache-2.0
Finetuned from model : inceptionai/jais-family-256m-chat
Dataset: CopyleftCultivars/Natural-Farming-Real-QandA-Conversations-Q1-2024-Update (Real world Natural Farming advise, from over 12 countries and a multitude of real-world farm operations, using semi-synthetic data curated by domain experts)
Please note, for inference you will need to use trust_remote_code=True due to the unique nature of the JAIS tokenizer.
Training Logs:
==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1
\ /| Num examples = 1,126 | Num Epochs = 1
O^O/ _/ \ Batch size per device = 2 | Gradient Accumulation steps = 4
\ / Total batch size = 8 | Total steps = 60
"-____-" Number of trainable parameters = 39,976,960
[60/60 22:39, Epoch 0/1]
Step Training Loss
1 1.499500
2 1.594600
3 1.210000
4 1.291600
5 1.195100
6 1.128400
7 1.357600
8 1.142600
9 1.273200
10 1.141300
11 0.784400
12 1.089100
13 0.923800
14 1.022000
15 1.074200
16 1.061700
17 1.029500
18 0.942500
19 0.952800
20 1.110500
21 0.979600
22 1.002900
23 0.992000
24 0.899800
25 0.922700
26 0.890500
27 0.931000
28 1.071200
29 0.964500
30 0.698300
31 0.845000
32 0.931100
33 0.943900
34 0.921000
35 0.955200
36 0.964800
37 0.888700
38 2.008400
39 1.158700
40 1.477300
41 0.898100
42 0.842900
43 0.826200
44 0.854500
45 0.959100
46 0.841400
47 1.139300
48 0.790600
49 0.702300
50 0.961000
51 0.620100
52 0.646900
53 1.154100
54 0.631300
55 0.259500
56 2.067900
57 0.896700
58 1.504800
59 1.769100
60 0.921500
This llama model was trained 2x faster with
Unsloth and Huggingface's TRL library.
