Creating a 70b-parameter LLM from Scratch in Google Colab

Step-by-Step Instructions

1. Open Google Colab
   Go to → https://colab.research.google.com
   Sign in with your Google account (or create one).
2. Open Google Drive
   Go to → https://drive.google.com
   Make sure you are logged in with the same Google account.
3. Upload the Python script(s)
   • Download the training script (e.g., gpt_jit_modern_70b.py and any required files).
   • Upload them to your Google Drive → MyDrive folder (root).
4. Open a new Colab notebook
   In Colab, click File → Open notebook → Google Drive and navigate to your MyDrive folder.
5. Mount Google Drive in Colab Run this cell first:
   python

   1from google.colab import drive 2drive.mount('/content/drive') 3
6. %cd /content/drive/MyDrive

1. Create model

   python

   1python gpt_jit_modern_70b.py 2

2. See in terminal

• Creating 0.94B-parameter Llama-style model on cpu...
• Model weights checked: no NaN/Inf found.
• Model ready. Parameters: xxB
• Logits min/max: -4.722229480743408 4.230556488037109
• Any NaN in logits: False
• Test forward pass OK (without JIT) → Logits: torch.Size([1, 256, 50257])
• Tracing model for JIT export (length: 256)...
• Success! JIT model saved → /content/drive/MyDrive/models/gpt_modern_70b_class.script.pt (xx GB)
• Original state_dict saved for debugging → gpt_modern_1b_class.state_dict.pt

Files Generated

• After successful run, you will have:
• gpt_modern_1b_class.script.pt → JIT-traced model ready for fast inference (~xx GB)
• gpt_modern_1b_class.state_dict.pt → Original PyTorch state dict (useful for conversion to Hugging Face format)

Tips

• Use Colab Pro or Pro+ for faster training and higher RAM (124B+ models may require it).
• The script runs on CPU by default but can be modified to use GPU/TPU.
• To convert the .pt file to Hugging Face Transformers format, additional conversion scripts are usually needed.
• Enjoy building your own LLM from scratch for free!

Deep learning

• Edit file fine_tune1b_with_validation_no_torchscript.py
• Setup dataset path and number of EPOCHS
• Run the script
  python fine_tune1b_with_validation_no_torchscript.py

Test your model (LLM)

• Edit chatbot_state_dict_70b.py
• Setup TEMPERATURE = 0.7 and TOP_K = 50 or other values
• Setup correct path to trained model LAST_TRAINED_PATH = Path("models/gpt_last_modern_70b_class.state_dict.pt")
• and run chatbot to test you model
  python chatbot_state_dict_70b.py

After valudation model

• So you can add mode EPOCHS and contunue your trainings
• then run chatbot to check model again
• onece it done call me I will comvert it to ONNX format and make Web service with REST API on ONNX Runtime libary
• So About PyTorch script . You can use Pytorch script for AI classification task .
• Do not Jit for Chatbot task . Use just state dict PyTorch for GPT (Chatbot) tasks

install tokenizer before run

• mkdir -p tokenizer
• wget -O tokenizer/tokenizer.json https://huggingface.co/gpt2/resolve/main/tokenizer.json
• wget -O tokenizer/vocab.json https://huggingface.co/gpt2/resolve/main/vocab.json
• wget -O tokenizer/merges.txt https://huggingface.co/gpt2/resolve/main/merges.txt
• wget -O tokenizer/tokenizer_config.json https://huggingface.co/gpt2/resolve/main/tokenizer_config.json

JiRack RAG System

• It is microservice architecture with API Gateway and Service Discovery
• Framework Spring boot and Google embeddings model for JiRack RAG System with Chatbot and JiRach model deployment with docker scipt
• video https://www.youtube.com/watch?v=vHClQu76kMc
• RAG System https://bitbucket.org/cmsmanhattan/rag/src/main/

• Copyright Office
• From:
• cop-rc@loc.gov
• To:
• konstantin.grabko@yahoo.com
• Mon, Dec 15 at 7:25 AM
• THIS IS AN AUTOMATED EMAIL. PLEASE DO NOT REPLY.
• Thank you for submitting your registration claim using the Electronic Copyright Office (ECO) System.
• The following files were successfully uploaded for service request 1-15058193231
• File Name :jirack_gpt3_class_pytorch_models.zip
• File Size :20144 KB
• Date/Time :12/15/2025 7:32:34 AM
• [THREAD ID: 1-6X1C8AZ]
• United States Copyright Office

Welcom to CMS Manhattan

• Copyright (c) 2025 CMS Manhattan
• All rights reserved.
• Author: Konstantin Vladimirovich Grabko
• Email: grabko@cmsmanhattan.com
• Phone: +1(516)777-0945
• This file is part of a project authored by CMS Manhattan. You may use, distribute, and modify
• this code under the terms of the GNU GENERAL PUBLIC LICENSE, Version 3, 29 June 2007.
• Please read http://www.gnu.org/licenses/.