Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
shuPT-xiaohongshu-Llama3-8B-Instruct – AI Model by ilovesushiandkimchiandmalaxiangguo | AlphaNeural AI
You can deploy this model and start earning money today!
ilovesushiandkimchiandmalaxiangguo
/
shuPT-xiaohongshu-Llama3-8B-Instruct
like
0
transformers
safetensors
llama
text-generation
xiaohongshu
conversational
ilovesushiandkimchiandmalaxiangguo/shuPT_data
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Model
Model Card for Model ID
Introducing 🍠(shǔ)PT-Llama3-8B-Instruct ! A SFT (Supervised Fine-Tuning) model trained on Xiaohongshu data.
Model Details
Base Model
: Meta-Llama-3-8B-Instruct
Architecture
: LLaMA 3 with LoRA fine-tuning
Language
: Chinese
Task
: Instruction following and conversational responses
License
: Same as base LLaMA 3 model.
Training Details
Training Data
: 30k crawled from Xiaohongshu (小红书)
Framework
: Hugging Face Transformers, PEFT, TRL
Training Data Details
Data source: 30k entries crawled from Xiaohongshu (小红书), specifically targeting 'city walk' related posts
Copyright: All content rights belong to Xiaohongshu
Usage restrictions: Academic/research purposes only
Format: Alpaca-style instruction tuning format
Known issues:
Incomplete conversation formatting in some samples
EOS token placement inconsistencies
Response length variations
Training Parameters
Learning rate: 2e-4
Weight decay: 0.001
Max gradient norm: 0.3
Warmup ratio: 0.03
LR scheduler: Cosine
Training precision: 4-bit quantization (QLoRA)
LoRA rank (r): 64
LoRA alpha: 16
LoRA dropout: 0.1
Intended Use
Chinese language instruction following
Conversational responses
Location-based recommendations
City navigation assistance
Cultural and historical information sharing
Limitations
Limited to Chinese language understanding and generation
Domain-specific knowledge biased towards Xiaohongshu content
Inherits base model limitations
May generate inconsistent responses due to temperature-based sampling
Performance
Shows improved performance on Chinese instruction following
Demonstrates strong capabilities in location-based recommendations
Ethical Considerations
Model inherits potential biases from Xiaohongshu data
Should be used in compliance with base model's usage policies
Content generation should be monitored for accuracy and appropriateness
Optimization Opportunities
Training parameters could be optimized for better performance
Data cleaning and formatting could be improved
Result
Question: 请推荐一下北京的city walk路线 (Please recommend some city walk routes in Beijing)
Model
