Views
No views yet
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3tokenizer = AutoTokenizer.from_pretrained("abirmaheshwari/abirhinv1")
4model = AutoModelForCausalLM.from_pretrained("abirmaheshwari/abirhinv1")
5
6input_text = "भारत एक महान देश है क्योंकि"
7
8inputs = tokenizer(input_text, return_tensors="pt")
9
10outputs = model.generate(
11 **inputs,
12 max_length=100,
13 temperature=0.7
14)
15
16print(tokenizer.decode(outputs[0]))
17
18---
19
20# Training Details
21
22## Training Data
23
24ABIRHINv1 was trained on a custom Hindi-focused dataset designed to capture linguistic patterns, conversational structures, and general language usage.
25
26The dataset includes:
27
28- Hindi natural text corpus
29- Hinglish conversational text
30- General-purpose text data
31
32Dataset specifications:
33
34- Total samples: ~161,000
35- Tokenizer: Custom-trained BPE tokenizer
36- Tokenizer trained entirely from scratch
37
38---
39
40## Training Procedure
41
42### Preprocessing
43
44The dataset was processed using a custom Byte Pair Encoding (BPE) tokenizer with the following specifications:
45
46- Vocabulary size: 32,000 tokens
47- Maximum sequence length: 512 tokens
48- Tokenizer trained on the same dataset as the model
49
50---
51
52### Training Hyperparameters
53
54The model was trained using the following configuration:
55
56- Optimizer: AdamW
57- Learning rate: 5 × 10⁻⁵
58- Training precision: FP16 mixed precision
59- Epochs: 3
60- Batch size: 4
61- Training objective: Causal Language Modeling
62
63---
64
65### Training Hardware
66
67Training was performed using GPU acceleration.
68
69- GPU: NVIDIA GPU (CUDA-enabled)
70- Framework: PyTorch
71- Library: HuggingFace Transformers
72
73---
74
75# Evaluation
76
77## Testing Data
78
79Evaluation was performed using custom Hindi text samples representative of real-world usage.
80
81---
82
83## Metrics
84
85Primary evaluation metric:
86
87- Training loss monitoring
88
89---
90
91## Results
92
93The model successfully learns:
94
95- Hindi sentence structure
96- Token relationships
97- Language continuity
98- Contextual text generation
99
100The model demonstrates functional Hindi generation capability suitable for research and fine-tuning.
101
102---
103
104# Technical Specifications
105
106## Architecture
107
108ABIRHINv1 uses a decoder-only Transformer architecture consisting of:
109
110- Token embedding layer
111- Learned positional embeddings
112- Multi-head self-attention layers
113- Feedforward neural network layers
114- GELU activation function
115- Weight tying between embedding and output layers
116
117---
118
119## Model Size
120
121- Total parameters: ~96 Million
122- Context length: 512 tokens
123- Vocabulary size: 32,000 tokens
124
125---
126
127# Compute Infrastructure
128
129## Hardware
130
131- NVIDIA GPU
132
133---
134
135## Software
136
137- Python
138- PyTorch
139- HuggingFace Transformers
140- SafeTensors
141
142---
143
144# Environmental Impact
145
146Training specifications:
147
148- Hardware type: NVIDIA GPU
149- Training duration: ~3–4 hours
150- Framework: PyTorch
151
152---
153
154# Author
155
156Abir Maheshwari
157Independent AI Researcher
158
159HuggingFace Profile:
160https://huggingface.co/abirmaheshwari
161
162---
163
164# Version
165
166ABIRHINv1
167
168Initial release version.
169
170---
171
172# Contact
173
174For questions, collaboration, or research inquiries:
175
176HuggingFace:
177https://huggingface.co/abirmaheshwari
178
179---