Large Language Model (LLM) is Linear Regression. This statement is not shared here without any evidence as This repository contains a reproducible
Notebook named as LLM_is_Linear_Regression.ipynb , therefore, anyone can run this notebook and verify that statement.
Benchmark (on Train set of main split of GSM8K dataset):
BLEU Score is 93%
Exact Match (EM) metrics is 100%
Artificial Intelligence (AI), Machine Learning, Deep Learning means Neural Network are evolved from Linear Regression so it may possible that
achieving LLM is possible by using Linear Regression as Logistic Regression which is backbone of classification is also evolved from
Linear Regression itself, therefore, This model is the attempt to find the better and simplest solution using the backbone of modern AI means
Linear Regression.
Solving Large Language model using Linear Regression in Machine Learning using library like scikit-learn do these five things:-
It changes the timeline to before Transformer Era or precisely before attention Era like who needs expensive GPU to run the LLMs
if they can do it using machine learning only in the personal computer.
It's Lightening fast so no comparison with existing Transformer based methods like Next word prediction and if one wants to compare then this method would be atleast 1000% times faster as believed.
Linear Regression is determinstic, Like we need AI which can be debugged and get fixed rather than probablistic for critical task like robotic surgery and code generation.
Generalisation happens when embedding comes in like previously also Word2Vec or Glove embeddings used for transfer learning and generalisation perspective so it can be done here also if required.
No hallucination and everytime same outcome because its deterministic.
Use below code to see the predictions.
from transformers import AutoModel
model = AutoModel.from_pretrained("Kratim-Budhimata/LLM_is_LinearRegresison", trust_remote_code=True, device_map="auto")
prompt=['Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?']
model.predict(prompt)
### Outcome should be: ['1/3rd circled = = <<48/2=24>>24 clips in may. natalia sold 48+24 = <<48+24=72>>72 clips altogether in april and may. #### 72']
This is true democratisation of AI and Only magic available in this notebook is considered Y axis means dependent variable as array of floats which gives luxury to generate prediction and later decision boundary logic did everything remaining.
So Text Generation is solved using this way, but it provides vision to solve image and video generation also by the same way because Fundamentally
Generation is happening at Y axis which is already a dependent variable so for example if its Text-to-Text or Text-to-Image or Text-to-video
generation, In all three scenarios, Y axis has text or image or video so its already dependent variable,
means you map Text or Image or Video as dependent variable, model does not discriminate to generate the outcome,
And if its Text-Image-to-Image Generation or Text-Video-to-Video generation then in that case also X axis would have two features (Text and Image, OR, Text and Video) which will get concatenated
before going into the model and Y axis already have Image or Video so these usecases also can be solved this way, therefore, Most Generative AI applications can be solved using this way.
As believed, people will say that Generative AI is Linear Regression application or use cases in near future.
Common Question:
If it is machine learning based then why integrated with Transformer
Answer: Modern AI Community has tech stack in transformer so if it can be integrated with existing tech then it will have less friction for developers,
otherwise reproducible notebook is here to implement from scratch for everyone.
Feedback and Queries: If you have any queries or feedback do reach out to us at-
