Model Card for CodeLlama-7B-Instruct-Luau
Fine-tuned version of codellama/CodeLlama-7b-Instruct-hf targeted toward the Luau programming language, Roblox’s Lua-derived scripting language.
This model is distributed as a LoRA adapter and is intended to improve the base model’s performance on Roblox-specific scripting tasks.
Model Details
Model Description
This model is a parameter-efficient fine-tuning (LoRA) of CodeLlama 7B Instruct, specialized for generating, explaining, and refactoring Luau code.
The fine-tuning focuses on Roblox development patterns, including common services, APIs, gameplay scripting idioms, and client/server logic. The model is designed to assist developers during prototyping, learning, and general scripting workflows.
- Developed by: darwinkernelpanic
- Funded by: Not applicable
- Shared by: darwinkernelpanic
- Model type: Causal Language Model (decoder-only, LoRA adapter)
- Language(s) (NLP): English
- License: Apache-2.0
- Finetuned from model: codellama/CodeLlama-7b-Instruct-hf
Model Sources
- Repository: https://huggingface.co/darwinkernelpanic/CodeLlama-7b-Instruct-hf-luau
- Paper: Code Llama: Large Language Models for Code (Meta AI)
- Demo: Not available
Uses
Direct Use
This model can be used directly for:
- Writing Luau scripts for Roblox
- Explaining Roblox APIs and services
- Refactoring or debugging Luau code
- Prototyping gameplay systems and utilities
- Learning Luau and Roblox scripting concepts
The model is intended as a developer assistant, not an autonomous system.
Downstream Use
Potential downstream uses include:
- Further fine-tuning on proprietary Roblox frameworks
- Integration into IDEs or editor tooling
- Chat-based assistants for Roblox development
- Educational or documentation tooling
Out-of-Scope Use
This model should not be used for:
- Safety-critical or production-critical systems
- Legal, medical, or financial advice
- Malware, exploit, or cheat development
- Fully automated code deployment without review
Bias, Risks, and Limitations
- Inherits biases and limitations from the base CodeLlama model
- May hallucinate Roblox APIs or outdated behaviors
- Does not validate code at runtime
- Output correctness depends on prompt quality
Recommendations
Users should:
- Review all generated code manually
- Test scripts in Roblox Studio
- Cross-check with official Roblox documentation
- Treat outputs as suggestions rather than authoritative solutions
How to Get Started with the Model
1from transformers import AutoTokenizer, AutoModelForCausalLM
2from peft import PeftModel
3
4base_model = "codellama/CodeLlama-7b-Instruct-hf"
5adapter_model = "darwinkernelpanic/CodeLlama-7b-Instruct-hf-luau"
6
7tokenizer = AutoTokenizer.from_pretrained(base_model)
8model = AutoModelForCausalLM.from_pretrained(base_model)
9model = PeftModel.from_pretrained(model, adapter_model)
10
11prompt = "Write a Luau function that creates a Part and parents it to Workspace."
12inputs = tokenizer(prompt, return_tensors="pt")
13
14outputs = model.generate(
15 **inputs,
16 max_new_tokens=300,
17 temperature=0.7,
18 do_sample=True
19)
20
21print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Details
Training Data
The model was fine-tuned on a curated mixture of:
- Luau scripts
- Roblox API usage examples
- Open-source Roblox projects
- Synthetic instruction-style prompts
All data was filtered to avoid private, proprietary, or sensitive content.
Training Procedure
The model was trained using parameter-efficient fine-tuning with LoRA while keeping the base model weights frozen.
Preprocessing
- Code formatting normalization
- Instruction-style prompt structuring
- Removal of low-quality or irrelevant samples
Training Hyperparameters
- Training regime: fp16 mixed precision
Speeds, Sizes, Times
- Base model size: ~7B parameters
- Trainable parameters: <1% (LoRA adapters only)
- Adapter checkpoint size: ~100–200 MB
Evaluation
Testing Data, Factors & Metrics
Testing Data
- Hand-written Luau prompts
- Roblox-specific scripting scenarios
Factors
- Luau syntax correctness
- Roblox API familiarity
- Instruction-following behavior
Metrics
- Qualitative human evaluation
- Manual code review and comparison with base model
Results
The LoRA adapter demonstrates improved performance over the base model in:
- Generating idiomatic Luau
- Correct Roblox service usage
- Following game-development-oriented instructions
Summary
The model performs best when used as a Roblox development assistant and is not intended for general-purpose natural language tasks.
Model Examination
No formal interpretability or probing analysis was conducted.
Environmental Impact
Carbon emissions were not formally measured.
- Hardware Type: Consumer-grade GPU
- Hours used: < 24 hours
- Cloud Provider: None (local training)
- Compute Region: Not applicable
- Carbon Emitted: Not estimated
Technical Specifications
Model Architecture and Objective
- Decoder-only Transformer
- Next-token prediction objective
- LoRA adapters applied to attention layers
Compute Infrastructure
Hardware
- Single consumer-grade GPU
Software
- PyTorch
- Transformers
- PEFT
Citation
BibTeX:
1@misc{darwinkernelpanic2025luau,
2 title={CodeLlama 7B Instruct Luau LoRA},
3 author={darwinkernelpanic},
4 year={2025},
5 howpublished={Hugging Face},
6 note={LoRA fine-tuned for Luau / Roblox scripting}
7}
APA:
darwinkernelpanic. (2025). CodeLlama 7B Instruct Luau LoRA. Hugging Face.
Model Card Authors
darwinkernelpanic
Model Card Contact
Use the Hugging Face repository issues or the author’s profile.
Framework versions
