Poetic_Llama-3.2-1B is a specialized language model fine-tuned for creative poetry generation. Built upon Meta's Llama-3.2-1B foundation model, this adapter has been optimized using supervised fine-tuning (SFT) with Low-Rank Adaptation (LoRA) techniques to generate coherent, stylistically diverse, and contextually appropriate poetic content in English. The model is designed to understand poetic instructions and generate verses across various styles, forms, and themes, making it suitable for creative writing applications, educational tools, and literary experimentation.

How to Get Started with the Model

Use the code below to get started with the model:

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Load the model and tokenizer
model_name = "Exquisique/Poetic_Llama-3.2-1B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
# Example poetry generation
prompt = "Human: Write a poem about hope and dawn.\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
# Generate
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        temperature=0.7,
        top_p=0.9,
        do_sample=True
    )
poem = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(poem)

Important: Prompt Formatting for Best Results

For the most consistent poetic outputs, format your prompts to match the model’s training style:

Use the dialog-style prefix: Human: <your instruction or poetic request>\nAssistant:
After Assistant:, let the model continue the poem.

Recommended examples:

Human: Write a poem about hope and dawn.\nAssistant:
Human: Compose a haiku about autumn leaves and wind.\nAssistant:
Human: Write a Shakespearean sonnet on time and memory.\nAssistant:

Why this matters: The model was fine-tuned on instruction-following data that uses this Human/Assistant schema. Adhering to this format helps the model align to the intended poetic style and structure.

Usage Tips

Use clear, specific instructions in your prompts (e.g., "Write a sonnet about...", "Create a free verse poem describing...")
Adjust temperature (0.6-0.9) for creativity vs. coherence tradeoff
Use top_p sampling for diverse outputs
Keep the Human:/Assistant: format for consistent results

Model Details

Model Description

Developed by: Exquisique
Model type: Causal Language Model (Fine-tuned Adapter)
Language(s): English
License: Apache 2.0
Finetuned from model: meta-llama/Llama-3.2-1B
Architecture: Llama 3.2 with LoRA adapter
Parameters: 1.24B (base model) + LoRA adapter weights
Precision: BF16

Model Sources

Repository: Exquisique/Poetic_Llama-3.2-1B
Base Model: meta-llama/Llama-3.2-1B
Training Framework: TRL (Transformer Reinforcement Learning)
Adapter Method: PEFT 0.17.1 (LoRA)

Uses

Direct Use

This model can be directly used for:

Poetry Generation: Creating original poems based on prompts, themes, or specific instructions
Creative Writing Assistance: Helping writers with poetic language, metaphors, and verse structure
Educational Applications: Teaching poetic forms, literary devices, and creative expression
Content Creation: Generating poetic content for social media, marketing, or artistic projects
Literary Exploration: Experimenting with different poetic styles, forms (sonnets, haiku, free verse, etc.), and themes

Downstream Use

The model can be integrated into:

Creative writing applications and platforms
Educational software for literature and creative writing courses
Chatbots and virtual assistants with creative capabilities
Content generation pipelines for marketing and social media
Research tools for computational creativity and natural language generation studies

Out-of-Scope Use

Factual Information Retrieval: This model is optimized for creative poetry generation, not factual accuracy or information retrieval
Professional Medical, Legal, or Financial Advice: Not suitable for generating content requiring professional expertise
Harmful Content Generation: Should not be used to create content that promotes violence, hatred, discrimination, or illegal activities
Impersonation: Not intended for generating content that impersonates specific authors or public figures without disclosure
Academic Dishonesty: Should not be used to generate content for plagiarism or academic fraud

Bias, Risks, and Limitations

Known Limitations

Creative Scope: While fine-tuned on poetry, the model's output quality may vary across different poetic styles and forms
Cultural Representation: The training data (poem-instructions-dataset) may have limited representation of non-Western poetic traditions
Context Length: As a 1B parameter model, it has limitations in maintaining coherence over very long compositions
Factual Accuracy: Creative outputs may contain factually incorrect information, as the model prioritizes poetic expression over accuracy
Originality: Generated poems may occasionally reflect patterns or themes from training data

Potential Biases

The model may exhibit biases present in the base Llama-3.2-1B model and the poem-instructions-dataset
Potential over-representation of certain poetic styles, themes, or cultural perspectives in the training data
Language biases toward formal or conventional English poetic traditions

Risks

Misattribution: Generated content might be mistaken for human-authored work without proper disclosure
Content Appropriateness: May occasionally generate content with mature themes or sensitive subject matter
Over-reliance: Users may become overly dependent on AI-generated content, potentially limiting human creativity

Recommendations

Users should:

Clearly disclose when content is AI-generated, especially in published or commercial contexts
Review and edit generated content for appropriateness, quality, and alignment with intended use
Use the model as a creative tool to augment, not replace, human creativity and judgment
Be aware of potential biases and work to ensure diverse and inclusive creative outputs
Not use generated poetry for academic submissions without proper attribution and instructor approval
Implement content filtering mechanisms when deploying in user-facing applications

Training Details

Training Data

The model was fine-tuned on the iamketan25/poem-instructions-dataset, which contains instruction-following examples for poetry generation. This dataset includes:

Diverse poetic prompts and instructions
Examples covering various poetic forms and styles
Instruction-output pairs designed for supervised fine-tuning
Approximately 2,100 examples (based on dataset statistics) The dataset focuses on teaching the model to understand and execute creative instructions for generating contextually appropriate poetic content.

Training Procedure

Training Method

Fine-tuning Approach: Supervised Fine-Tuning (SFT) using the TRL library
Parameter-Efficient Method: LoRA (Low-Rank Adaptation) for efficient training and deployment
Base Model: meta-llama/Llama-3.2-1B (1.24B parameters)
Adapter Framework: PEFT (Parameter-Efficient Fine-Tuning) version 0.17.1
Training Framework: Hugging Face Transformers with TRL

Training Configuration

The model was trained using parameter-efficient fine-tuning techniques:

LoRA adapters were applied to enable efficient training without modifying all base model parameters
Training focused on instruction-following capabilities for creative poetry generation
BF16 precision was used for efficient computation and memory usage Note: Specific hyperparameters (learning rate, batch size, number of epochs, LoRA rank, alpha values, etc.) were not provided in the model repository.

Evaluation

Testing Data and Metrics

Testing Approach

Formal quantitative evaluation metrics for creative text generation (especially poetry) are challenging, as quality is often subjective and context-dependent. The model's performance should be evaluated based on:

Coherence: Logical flow and consistency within generated poems
Instruction Following: Adherence to specified themes, styles, or constraints
Creativity: Originality and interesting use of language and literary devices
Grammaticality: Proper syntax and language structure
Stylistic Appropriateness: Alignment with requested poetic forms or styles

Qualitative Assessment

Users are encouraged to evaluate the model through:

Human evaluation of generated poetry quality
Comparison with baseline models (unmodified Llama-3.2-1B or other poetry-generation models)
User feedback in creative writing applications Note: Specific quantitative evaluation results, test sets, or benchmark scores were not provided in the model repository.

Environmental Impact

Carbon emissions and environmental impact can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Training Infrastructure

Hardware Type: GPU (specific model not specified)
Training Time: Not specified
Cloud Provider: Not specified
Compute Region: Not specified
Estimated Carbon Emitted: Not measured Note: Using LoRA and parameter-efficient fine-tuning significantly reduces computational requirements and environmental impact compared to full model training.

Inference Efficiency

The model uses:

BF16 precision for efficient inference
LoRA adapters for reduced memory footprint
1.24B parameters (relatively small model size) enabling deployment on consumer hardware These characteristics make the model more environmentally friendly and accessible for inference compared to larger language models.

Technical Specifications

Model Architecture and Objective

Base Architecture: Llama 3.2 (Causal Language Model)
Model Size: 1.24B parameters
Adapter Type: LoRA (Low-Rank Adaptation)
Tensor Format: BF16 (Brain Floating Point 16-bit)
Objective: Supervised instruction-following fine-tuning for poetry generation
Tokenizer: Llama tokenizer (from base model)

Compute Infrastructure

Hardware

Training was conducted using GPU acceleration (specific hardware details not provided in repository).

Software

Framework: Hugging Face Transformers
Fine-tuning Library: TRL (Transformer Reinforcement Learning)
Adapter Framework: PEFT 0.17.1
Precision: BF16
Model Format: Safetensors

Citation

BibTeX

If you use this model in your research or applications, please cite:

bibtex

@misc{poetic_llama_3_2_1b,
  author = {Exquisique},
  title = {Poetic_Llama-3.2-1B: A Fine-tuned Language Model for Poetry Generation},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/Exquisique/Poetic_Llama-3.2-1B}}
}

Please also cite the base model:

bibtex

@article{dubey2024llama,
  title={The Llama 3 Herd of Models},
  author={Dubey, Abhimanyu and others},
  journal={arXiv preprint arXiv:2407.21783},
  year={2024}
}

And the training dataset:

bibtex

@misc{poem_instructions_dataset,
  author = {iamketan25},
  title = {Poem Instructions Dataset},
  year = {2023},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/datasets/iamketan25/poem-instructions-dataset}}
}

APA

Exquisique. (2024). Poetic_Llama-3.2-1B: A Fine-tuned Language Model for Poetry Generation. HuggingFace. https://huggingface.co/Exquisique/Poetic_Llama-3.2-1B

Model Card Authors

This model card was created by Exquisique with comprehensive updates to meet Hugging Face model card standards.

Model Card Contact

For questions, issues, or collaboration opportunities regarding this model, please:

Open an issue in the model repository discussions
Visit the Exquisique profile for contact information

Acknowledgments

Meta AI for developing and releasing the Llama 3.2 base model
iamketan25 for creating and sharing the poem-instructions-dataset
Hugging Face for the Transformers, PEFT, and TRL libraries
The open-source AI community for tools and frameworks enabling this work