Views
No views yet
1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2from peft import get_peft_model, LoraConfig
3
4# Load tokenizer and model
5tokenizer = AutoTokenizer.from_pretrained("InstaDeepAI/nucleotide-transformer-2.5b-multi-species")
6model = AutoModelForSequenceClassification.from_pretrained("alakob/DraPLASMID-2.5b-v1")
7
8# Example inference
9sequence = "ATGC..." # Replace with your nucleotide sequence
10inputs = tokenizer(sequence, truncation=True, max_length=1000, return_tensors="pt")
11outputs = model(**inputs)
12prediction = outputs.logits.argmax(-1).item() # 0 = non-PLASMID, 1 = PLASMID
13
14## Training Details
15
16### Training Data
17
18The model was trained on the DraPLASMID-2.5b-v1 dataset, consisting of 1200 overlapping sequences:
19
20- **Negative sequences (non-PLASMID):**
21 `DSM_20231.fasta`, `ecoli-k12.fasta`, `FDA.fasta`
22
23- **Positive sequences (PLASMID):**
24 Plasmid sequences
25
26### Training Procedure
27
28#### Preprocessing [optional]
29
30Sequences were tokenized using the Nucleotide Transformer tokenizer with a maximum length of 1000 tokens and truncation applied where necessary.
31
32#### Training Hyperparameters
33
34- **Training regime:** fp16 mixed precision
35- **Learning rate:** 5e-5
36- **Batch size:** 8 (with gradient accumulation steps = 8)
37- **Epochs:** 10
38- **Optimizer:** AdamW (default in Hugging Face Trainer)
39- **Scheduler:** Linear with 10% warmup
40- **LoRA parameters:** `r=32`, `alpha=64`, `dropout=0.1`, `target_modules=["query", "value"]`
41
42#### Speeds, Sizes, Times [optional]
43
44Training was performed on Google Colab with checkpointing every 500 steps, retaining the last 3 checkpoints.
45Exact throughput and times depend on Colab's hardware allocation (typically T4 GPU).
46
47---
48
49## Evaluation
50
51### Testing Data, Factors & Metrics
52
53#### Testing Data
54
55The test set was derived from a 10% split of the DraPLASMID-2.5b-v1 dataset, stratified by PLASMID labels.
56
57#### Factors
58
59Evaluation was performed across PLASMID and non-PLASMID classes.
60
61#### Metrics
62
63- **Accuracy:** Proportion of correct predictions
64- **F1 Score:** Harmonic mean of precision and recall (primary metric)
65- **Precision:** Positive predictive value
66- **Recall:** Sensitivity
67- **ROC-AUC:** Area under the receiver operating characteristic curve
68
69### Results
70
71[More Information Needed]
72
73#### Summary
74
75[More Information Needed]
76
77---
78
79## Model Examination [optional]
80
81[More Information Needed]
82
83---
84
85## Environmental Impact
86
87Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
88
89- **Hardware Type:** Google Colab GPU (typically NVIDIA T4)
90- **Hours used:** [More Information Needed]
91- **Cloud Provider:** Google Colab
92- **Compute Region:** [More Information Needed]
93- **Carbon Emitted:** [More Information Needed]
94
95---
96
97## Technical Specifications [optional]
98
99### Model Architecture and Objective
100
101The model uses the Nucleotide Transformer architecture (2.5B parameters) with a sequence classification head, fine-tuned with LoRA for PLASMID prediction.
102
103### Compute Infrastructure
104
105Training was performed on Google Colab with persistent storage via Google Drive.
106
107#### Hardware
108
109- NVIDIA T4 GPU (typical Colab allocation)
110
111#### Software
112
113- Transformers (Hugging Face)
114- PyTorch
115- PEFT (Parameter-Efficient Fine-Tuning)
116- Weights & Biases (wandb) for logging
117
118---
119
120## Citation [optional]
121
122**BibTeX:**
123[More Information Needed]
124
125**APA:**
126[More Information Needed]
127
128---
129
130## Glossary [optional]
131
132- **PLASMID:** Antimicrobial Resistance
133- **LoRA:** Low-Rank Adaptation
134- **Nucleotide Transformer:** A transformer-based model for nucleotide sequence analysis
135
136---
137
138## More Information [optional]
139
140[More Information Needed]
141
142---
143
144## Model Card Authors [optional]
145
146Blaise Alako
147
148---
149
150## Model Card Contact
151
152[More Information Needed]