netFound is a network traffic encoder model that uses transformer architecture and includes a pretraining phase on unlabeled data to achieve high results.
Key features:
netFound takes raw PCAP data as input
netFound can (and need) be pretrained on the unlabeled dataset
netFound uses Hierarchical Transformer architecture to take into account packet burst and flow behavior
netFound uses burst metadata (inter arrival time, number of bytes per burst, etc)
Details
Model config: large (24 layers, 16 attention heads, 1024 hidden size)
Pretraining details: 128 GPUs, 15k steps, ~7bln tokens seen
