Views
No views yet
| Property | Value |
|---|---|
| Type | Decoder-only Transformer (GPT) |
| Parameters | ~10M |
| Layers | 6 Transformer blocks |
| Attention heads | 6 per block |
| Embedding dim | 384 |
| Context length | 256 tokens |
| Tokenizer | GPT-2 BPE (50,257 vocab) |
| Activation | GELU |
| Training steps | 5,000 |
| Final val loss | 2.97 |
flytech/python-codes-25k (first 8k examples)def fibonacci(n):1def fibonacci(n):
2 if n < 0:
3 print("Incorrect input")
4 elif n == 1:
5 return 0
6 elif n == 2:
7 return 1
8 else:
9 return fibonacci(n-1) + fibonacci(n-2)def binary_search(arr, target):1def binary_search(arr, target):
2 low = 0
3 high = len(arr) - 1
4 while low <= high:
5 mid = (low + high) // 2
6 if arr[mid] == target:
7 return mid
8 else:
9 low = mid + 1
10 return -1