Views
No views yet
pidgin14, an encoder-decoder sequence-to-sequence system for Nigerian Pidgin English ("Naija") built by Ephraim at Analytics Intelligence.pidgin14 is composed of two halves published as separate repositories:Ephraimmm/pidgin14-encoder, based on AfriBERTa, reads source text and produces contextual representations.EncoderDecoderModel, whose full weights are published at Ephraimmm/pidgin14. The architecture facts below are taken directly from that combined model's config.json (decoder sub-config), since this component repository itself contains only tokenizer files (tokenizer.json, tokenizer_config.json, special_tokens_map.json, vocab.json, merges.txt) and not a standalone config.json or weight file.decoder sub-configuration of the combined Ephraimmm/pidgin14 model:| Field | Value |
|---|---|
| Base model | gpt2-medium |
| Model type | gpt2 (architecture class GPT2LMHeadModel), configured with add_cross_attention: true so it can act as the decoder half of an EncoderDecoderModel |
Layers (n_layer) | 24 |
Hidden size (n_embd) | 1024 |
Attention heads (n_head) | 16 |
Context length (n_positions / n_ctx) | 1024 |
| Vocabulary size | 50,257 |
| Activation function | gelu_new |
GPT2Tokenizer (byte-level BPE)vocab.json with 50,000 merge rules in merges.txt) — this matches the standard, unmodified GPT-2 tokenizer vocabulary rather than a Pidgin-specific retrained vocabulary.<|endoftext|> used as bos/eos/pad/unk (token id 50256).decoder_start_token_id: 50256 (per the combined model's config).gpt2-medium, used as the decoder half of the pidgin14 EncoderDecoderModel (with cross-attention layers added to attend to the encoder's outputs).transformers (the combined model's config records transformers_version: 4.44.2).float32 (per the combined model's config).trainer_state.json, training-step/epoch counts, optimizer settings, or training-dataset identifiers are published in this repository or in the combined Ephraimmm/pidgin14 repository. These details are therefore omitted rather than estimated.pidgin14 sequence-to-sequence pipeline (e.g. translation, paraphrasing, conversational response generation).pidgin14-encoder tokenizer and the trained weights in Ephraimmm/pidgin14 to produce output.1from transformers import AutoTokenizer, EncoderDecoderModel
2
3# Tokenizers for each half of the system
4encoder_tokenizer = AutoTokenizer.from_pretrained("Ephraimmm/pidgin14-encoder")
5decoder_tokenizer = AutoTokenizer.from_pretrained("Ephraimmm/pidgin14-decoder")
6
7# The trained combined encoder-decoder weights
8model = EncoderDecoderModel.from_pretrained("Ephraimmm/pidgin14")
9
10text = "How you dey?"
11inputs = encoder_tokenizer(text, return_tensors="pt")
12
13output_ids = model.generate(
14 **inputs,
15 decoder_start_token_id=decoder_tokenizer.bos_token_id,
16 max_length=50,
17)
18print(decoder_tokenizer.decode(output_ids[0], skip_special_tokens=True))pidgin14; it is not a usable standalone model and contains no weight file or config.json of its own.Ephraimmm/pidgin14-encoder and the weights in Ephraimmm/pidgin14 to perform any task.