Views
No views yet
| Feature | Value |
|---|---|
| Base Model | caduceus-ph (4-layer) |
| Pre-training Genome | GRCg6a (galGal6) |
| Sequence Length | 65,536 bp |
| Hidden Dimension | 256 |
| Layers | 4 |
| Vocab Size | 16 |
| Training Steps | 10,000 |
1from transformers import AutoModelForMaskedLM
2
3# Load model
4model = AutoModelForMaskedLM.from_pretrained(
5 "jamie0315/PoultryCaduceus",
6 subfolder="checkpoint-10000",
7 trust_remote_code=True
8)1import torch
2
3# DNA vocabulary
4DNA_VOCAB = {'A': 7, 'C': 8, 'G': 9, 'T': 10, 'N': 5, '[MASK]': 4}
5
6# Tokenize sequence
7sequence = "ATGCGATCGATCGATCG"
8input_ids = torch.tensor([[DNA_VOCAB.get(c, 5) for c in sequence]])
9
10# Get embeddings
11model.eval()
12with torch.no_grad():
13 outputs = model(input_ids, output_hidden_states=True)
14 embeddings = outputs.hidden_states[-1] # (batch, seq_len, 256)chicken_pretrain_data_GRCg6a/:train_65k.h5 - Training set (~58,000 sequences)val_65k.h5 - Validation set (~1,200 sequences)PoultryCaduceus/
├── checkpoint-10000/ # Model weights
│ ├── config.json
│ └── model.safetensors
└── chicken_pretrain_data_GRCg6a/ # Pre-training data
├── train_65k.h5
└── val_65k.h5