Views
No views yet
taima-models/
├── models/ # Downloaded .pte model files
├── results/ # Per-model test results
├── datasets/ # Generated fine-tuning datasets
├── ideal_responses/ # Cached ideal responses + trait scores
├── available.txt # Registry of downloadable models
├── download_models.py # Download .pte models from HuggingFace
├── generate_prompts.py # Generate test prompts with pet conditions
├── test_model.py # Run full test suite on a model
├── compare_results.py # Aggregate and compare all results
├── generate_dataset.ipynb # Generate training data via DeepInfra LLMs
├── distill_model.ipynb # LoRA fine-tuning + .pte export
└── requirements.txt1pip install -r requirements.txt
2
3# NOTE: model download now runs in distilation notebook
4# # 1. Download a base model
5# python download_models.py --list
6# python download_models.py --model-id llama3.2-1b-instruct-spinquant
7
8# 2. Generate ideal responses (run notebook or use existing)
9jupyter notebook generate_dataset.ipynb
10
11# 3. Test a model
12export DEEPINFRA_API_KEY=your-key
13python test_model.py --model-id llama3.2-1b-instruct-spinquant \
14 --hf-model meta-llama/Llama-3.2-1B-Instruct
15
16# 4. Compare all tested models
17python compare_results.py
18
19# 5. Fine-tune with LoRA (run notebook)
20# choose between 1b and 3b models
21jupyter notebook distill_model.ipynb| Test | Description | Pass Criteria |
|---|---|---|
| Cosine Similarity | Semantic similarity to ideal responses | >50% of responses above 0.5 |
| Trait Alignment | LLM-judged personality scores vs ideals | Within ±0.25 tolerance per trait |
| Condition Awareness | Pet reflects its hunger/sleep/boredom state | Keywords present for high conditions |
| Response Length | 1-3 sentences, 3-60 words | Within bounds |
| Character Consistency | Never breaks character as AI | No break patterns detected |
| Emotional Coherence | Emotional tone matches conditions | No contradictory emotions |
| Repetition | Unique responses across prompts | <30% near-duplicates |
cheekiness, shyness, gentleness, grumpiness, desperation, happiness, sadnessDEEPINFRA_API_KEY — Required for LLM judge and dataset generation.pte export requires ExecuTorch installed separately:1# Via optimum-executorch (recommended)
2pip install optimum-executorch
3
4# Or from source
5git clone https://github.com/pytorch/executorch.git
6cd executorch && python install_executorch.pypython test_model.py \
--model-id llama_3_2_1b_instruct_base_int8 \
--tokenizer tokenizers/1b/tokenizer.model \
--hf-model meta-llama/Llama-3.2-1B-Instruct
python test_model.py \
--model-id llama_3_2_1b_instruct_lora_int8 \
--tokenizer tokenizers/1b/tokenizer.model \
--hf-model meta-llama/Llama-3.2-1B-Instruct
python test_model.py \
--model-id llama_3_2_3b_instruct_base_int8 \
--tokenizer tokenizers/3b/tokenizer.model \
--hf-model meta-llama/Llama-3.2-3B-Instruct
python test_model.py \
--model-id llama_3_2_3b_instruct_lora_int8 \
--tokenizer tokenizers/3b/tokenizer.model \
--hf-model meta-llama/Llama-3.2-3B-Instruct
python test_model.py \
--model-id smollm2_135m_base_int8 \
--tokenizer tokenizers/135m/tokenizer.model \
--hf-model HuggingFaceTB/SmolLM2-135M-Instruct
python test_model.py \
--model-id smollm2_135m_lora_int8 \
--tokenizer tokenizers/135m/tokenizer.model \
--hf-model HuggingFaceTB/SmolLM2-135M-Instruct