A PEFT (LoRA) fine-tuned Mixtral-8x7B-Instruct-v0.1 model for agricultural question answering, developed using the Adaption Labs AutoScientist workflow.
---
Overview
AgroAdapt-Mixtral-8x7B is a domain-adapted language model designed to improve agricultural instruction following and question answering. The model was fine-tuned using the Adaption Labs AutoScientist workflow on a curated agricultural instruction dataset covering practical farming knowledge.
The model was trained using an adapted agricultural instruction dataset containing high-quality question-answer pairs covering:
Crop Production
Soil Science
Fertilizer Recommendations
Pest Management
Plant Disease Diagnosis
Irrigation
Livestock
Poultry
Sustainable Agriculture
Climate-Smart Agriculture
Farm Business
Agricultural Extension
Quick Start
The following example demonstrates how to load the LoRA adapter and perform inference using the Hugging Face Transformers ecosystem. The adapter is automatically merged with the base Mixtral model during inference.
1prompt ="""
2Farmer:
3My tomato leaves are turning yellow with brown spots.
4What could be the cause and how can I treat it?
5"""67inputs = tokenizer(prompt, return_tensors="pt")89outputs = model.generate(10**inputs,11 max_new_tokens=200,12 temperature=0.7,13)1415print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Expected Output
The model returns a practical, step-by-step agricultural recommendation based on the farmer's question, including possible causes, preventive measures, and treatment options.
Training Configuration
Parameter
Value
Fine-tuning Method
LoRA
Base Model
Mixtral-8x7B-Instruct
Epochs
5
LoRA Rank (r)
64
LoRA Alpha
128
Warmup Ratio
0.03
Optimizer
Cosine
Gradient Clipping
1.0
Target Modules
q_proj, k_proj, v_proj, o_proj
LoRA Equation
The adapter follows the standard LoRA formulation:
[
W' = W + \frac{\alpha}{r}BA
]
Where:
r = 64
α = 128
Scaling factor:
[
\frac{\alpha}{r} = \frac{128}{64} = 2
]
This scaling improves learning efficiency while keeping the number of trainable parameters small.
Training Summary
Fine-tuned using Adaption Labs AutoScientist
5 training epochs
Win rate improved from 24% → 76%
Agriculture benchmark improved from 26% → 75%
Training loss decreased consistently throughout optimization.
Gradient clipping stabilized training and reduced optimization spikes.
Example
Input
How can I prevent maize leaf blight?
Output
Maize leaf blight can be reduced by planting resistant varieties, practicing crop rotation, avoiding overhead irrigation, removing infected crop residues, and applying recommended fungicides when disease pressure is high.
Evaluation Results
Evaluation performed using Adaption Labs AutoScientist
The fine-tuned model demonstrates a substantial improvement over the baseline model after domain adaptation using Adaption Labs AutoScientist. The evaluation shows a significant increase in both overall and agriculture-specific performance while maintaining stable optimization throughout training.
These results indicate that the model effectively learned agricultural reasoning and instruction-following capabilities from the domain-specific dataset.
Conclusion
Training Dynamics
text
1Training Loss ↘ steadily decreased
2Validation Loss → remained stable
3Learning Rate ↘ cosine decay schedule
4Gradient Norm → controlled through clipping
5Adaptation Strategy → LoRA parameter-efficient fine-tuning
AgroAdapt-Mixtral-8x7B successfully adapts the Mixtral foundation model into a specialized agricultural assistant through efficient LoRA fine-tuning, stable optimization, and domain-specific instruction learning, enabling practical, context-aware support for modern farming applications.
Intended Uses
This model is suitable for:
AI Farming Assistants
Agricultural Chatbots
Farm Advisory Systems
Agricultural Education
Agricultural Research
Smart Farming Applications
Decision Support Systems
Credit
Adaptive data by Adaption.
Limitations
Although the model performs well on agricultural instruction tasks, responses should be verified before being used for real-world farming decisions. Local farming practices and expert guidance should always take precedence.
License
This model is released under the CC BY 4.0 License.
Acknowledgements
This project was developed as part of the Adaption Labs AutoScientist Challenge using the Mixtral-8x7B-Instruct-v0.1 base model.
Citation
If you use this model in your research or applications, please cite:
