Views
No views yet
pip install transformers torch1from transformers import BertForSequenceClassification, BertTokenizer
2import torch
3
4# Load quantized model
5quantized_model_path = "path/to/bert_finetuned_fp16"
6
7
8def generate_script(prompt):
9 device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # Check available device
10 model.to(device) # Move model to the appropriate device
11
12 inputs = tokenizer(f"Generate a movie script: {prompt}", return_tensors="pt", truncation=True, padding="max_length", max_length=256)
13 inputs = {key: value.to(device) for key, value in inputs.items()} # Move inputs to same device as model
14
15 with torch.no_grad():
16 outputs = model.generate(**inputs, max_length=256, num_return_sequences=1)
17
18 return tokenizer.decode(outputs[0], skip_special_tokens=True)
19
20# Test the script generator
21prompt = "SCENE: EXT. DARK ALLEY - NIGHT"
22print(generate_script(prompt))
23
24
25## Performance Metrics
26
27- **Accuracy:** 0.82
28- **Inference Speed:** Faster due to FP16 quantization
29
30## Fine-Tuning Details
31
32### Dataset
33
34
35
36### Training Configuration
37
38- **Number of epochs:** 3
39- **Batch size:** 8
40- **Evaluation strategy:** Per epoch
41- **Learning rate:** 2e-5
42- **Optimizer:** AdamW
43
44### Quantization
45
46The model is quantized using **Post-Training Quantization (PTQ)** with **Float16 (FP16)**, which reduces model size and improves inference efficiency while maintaining accuracy.
47
48## Repository Structure
49
## Limitations
- The model is optimized for English-language next-word prediction tasks.
- While quantization improves speed, minor accuracy degradation may occur.
- Performance on out-of-distribution text (e.g., highly technical or domain-specific data) may be limited.
## Contributing
Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.
``