Views
No views yet
1 trainer = SFTTrainer(
2 model = model,
3 tokenizer = tokenizer,
4 train_dataset = train,
5 dataset_text_field = "text",
6 max_seq_length = max_seq_length,
7 dataset_num_proc = 2,
8 packing = False, # Can make training 5x faster for short sequences.
9 args = TrainingArguments(
10 per_device_train_batch_size = 2,
11 gradient_accumulation_steps = 4,
12 warmup_steps = 5,
13 max_steps = 50,
14 learning_rate = 2e-4,
15 fp16 = not is_bfloat16_supported(),
16 bf16 = is_bfloat16_supported(),
17 logging_steps = 1,
18 optim = "adamw_8bit",
19 weight_decay = 0.01,
20 lr_scheduler_type = "linear",
21 seed = 3407,
22 output_dir = "outputs",
23 ),
24)```
25#### Speeds, Sizes, Times [optional]
26
27<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
28
29[More Information Needed]
30
31## Evaluation
32
33<!-- This section describes the evaluation protocols and provides the results. -->
34
35### Testing Data, Factors & Metrics
36
37#### Testing Data
38
39<!-- This should link to a Dataset Card if possible. -->
40
41[More Information Needed]
42
43#### Factors
44
45<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
46
47[More Information Needed]
48
49#### Metrics
50
51<!-- These are the evaluation metrics being used, ideally with a description of why. -->
52
53[More Information Needed]
54
55### Results
56
57[More Information Needed]
58
59#### Summary
60
61
62
63## Model Examination [optional]
64
65<!-- Relevant interpretability work for the model goes here -->
66
67[More Information Needed]
68
69## Environmental Impact
70
71<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
72
73Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
74
75- **Hardware Type:** 1 T4 GPU, RAM: 16 GB
76- **Hours used:** [More Information Needed]
77- **Cloud Provider:** Google CoLab
78- **Compute Region:** [More Information Needed]
79- **Carbon Emitted:** [More Information Needed]
80
81## Technical Specifications [optional]
82
83### Model Architecture and Objective
84
85[More Information Needed]
86
87### Compute Infrastructure
88
89[More Information Needed]
90
91#### Hardware
92
93[More Information Needed]
94
95#### Software
96
97[More Information Needed]
98
99## Citation [optional]
100
101https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing#scrollTo=kR3gIAX-SM2q
102
103**BibTeX:**
104
105[More Information Needed]
106
107**APA:**
108
109[More Information Needed]
110
111## Glossary [optional]
112
113<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
114
115[More Information Needed]
116
117## More Information [optional]
118
119[More Information Needed]
120
121## Model Card Authors [optional]
122
123[More Information Needed]
124
125## Model Card Contact
126
127[More Information Needed]