After we thought about how to go with LaaLM, we realized we'd be just spending compute.
We had originally thought to at least deliver it with some fine-tuning of Qwen 2.5 0.5B, but we have decided that LaaLM had enough models.
So we are announcing that we are officially stopping the LaaLM lineage here.
We know it's pretty early with just 2 released models being LaaLM-v1 and LaaLM-exp-v1, and a bunch of failed LaaLM-v2 snapshots that we lost (Like in actual storage sense we don't have acces to the copies.) but we can't see any novel or good perspective about this.
Go try prompting Claude or something like that to act as a Linux terminal, and it'll do much better than LaaLM.
It's actually pretty bad. The model LaaLM-exp-v1 was trained on -Qwen 2.5 3B- literally performed better than the actual LaaLM-exp-v1 when we gave it LaaLM-exp-v1's system prompt.
Or like maybe just install actual Linux.
We are thinking of another new model to continue our adventure in predicting computers with neural networks, but we do not want to say it here to keep some mystery.
And to make sure we don't announce something that won't make it.
Goodbye.
17 April 2026 Update
Just a quick reminder to you guys that the new model series we had been planning has been released as Code as a Language Model.
Our new project is now about having AI models work on made-up code languages and predict their output, even though the language is never actually something real.
Our first model CaaLM-v1 is on HuggingFace for you to checkout.
LaaLM-v2 (Deprecated. Anything you'll be reading from now on has been either lost or stopped.)
A small language model trained from scratch to be a Linux terminal. That's it.
What is this?
LaaLM-v2 is a ~100M parameter transformer trained from random weights to simulate bash commands. No pretrained base model, no finetuning - just straight training on terminal data.
Why? Because LaaLM-exp-v1 used a 3B pretrained model and knew way too much stuff it didn't need (like sudo, apt, networking). That's 30x more parameters than necessary. This version is built to only know what it needs to know.
These are currently what the model data looks like. The actual model may have different styles, but don't really expect a major difference.
Training
Training of LaaLM-v2 is a bit rough.
For the current status as of 19 February 2026, LaaLM had 3 training rounds with about 17.5k conversation examples.
But the model suffers from extreme hallucination, which we are trying to solve.
And experimentation ate a lot of budget from us, so it caused slowdowns.
For the current training hardware, we originally planned to use the Nvidia RTX PRO 6000, but due to software errors, we used an MI300X instead. Though we have plans of using TPUs also.
Although our new training code shows good improvements, we hope to see good results once the budget is resolved.
But the current versions, as we said, are pretty rough for now.
While we can't say we are very sure, we will try training LaaLM-v2 on the TPU v6e chips for our next tries. TPUs offer excellent price-to-performance for us, which we really need and with their fast interconnects and memory.
But it will take us a good amount of time to switch LaaLM-v2's codebase to TPUs and do the bug fixing steps after that.
We don't really know about our configuration, but our current planning seems that we will probably use a v6e-4 configuration for the training.
We will try to update when we get our first results out of TPUs.
28 February 2026 Update
We tried TPUs, but because of OOM errors that also didn't work.
While trying to think of anything, we suddenly remembered the Chinchilla Scaling Law, and after doing it on our datasets turns out we majorly miscalculated the parameter count by a huge margin.
We will try another training round today.
1 March 2026 Update
The new runs were not successful.
But we got real improvements. While the model still says unnecessary things, it now hallucinates much less and does some of the things correctly.
The run was on an L40S GPU. We don't know why we picked it, to be honest.
But we found out that the data generator is flawed.
Normally, LaaLM-exp-v1 was trained on scenarios where future commands actually got the things that happened from commands before.
So in the data generator, if the generated data did mkdir hello in a later command, there would be ls which would show hello. It was done by using a Linux simulator.
But here LaaLM-v2's data generator randomly generates data without caring about anything before.
So, on it instead, it may do cd hello, but then does touch hi and goes on randomly generating. No persistence which is literally what LaaLM-v2 is for.
We will first try using LaaLM-exp-v1's training dataset generator here to see if our theory is correct.
We will give an update when the results are out.
4 March 2026 Update
We haven't tried the new data generator because of budget problems.
But we thought in the meantime that Transformers are too heavy for what LaaLM does.
So we are going to use an LSTM-based architecture.
LSTMs are not designed for big models. But LaaLM is very simple, so an LSTM is better for it.
But we can assure you that LaaLM-v2 is close to release.
6 March 2026 Update
We have updated the codebase, but we have a problem.
Speed.
Speed is crucial to us because we need rapid experimentation. If we take too much time, the cloud bill will explode.
But that's the problem also.
We can't find the appropriate accelerator that is exactly what LaaLM needs.
We have been using an L40S, but we got well fed up with its slowness and ineffectiveness without FP8.
So we will use another accelerator. But there's not really any price-to-performance accelerator that can fit us.
For the data generator, we have a weird problem.
Technically, v2's data generator is actually better than exp-v1.
But for some reason, there's a leak somewhere we can't figure out that causes the model to just cheat.
Maybe it's our indicator tokens, or it's something subtle we can't figure out, but LaaLM-v2 definitely got us fed up with the whole LaaLM franchise.
We also have sad news for LaaLM today.
LaaLM-v2 will be the last model of the LaaLM series.
Our reason is that we have better projects to spend our compute on than a bash predictor that any other model can beat.
LaaLM has definitely been a fun experience for us, but we can't just spend precious compute for something this experimental and non-useful.
Maybe future models will keep coming, but we definitely recommend that you may stop expecting new models after LaaLM-v2. (Well this turned out to be correct.)
How to use it
(Code examples coming soon - model is still training and the API might change)
Basic idea:
Load the model
Start with a system message about the initial state
Send commands as user messages
Get outputs as assistant messages
Model remembers what you did (files you created, directories you made, etc.)
LaaLM-v1 was a T5 model (encoder-decoder). Worked okay, but couldn't really do conversations.
LaaLM-exp-v1 was Qwen 3B finetuned. It worked great, but it was 3B parameters! It knew about sudo, apt, networking, all this stuff it didn't need. Felt wasteful.
LaaLM-v2 is trained from scratch with just what it needs. ~100M parameters, no bloat. Every parameter is there for bash commands, nothing else.
Yeah, it's harder to train from scratch (no pretrained stopping behavior, needs way more data), but the purity is worth it.
What it doesn't do
This isn't a real terminal. It simulates bash behavior but doesn't actually run commands.
No networking (ssh, wget, curl), no package managers (apt, pip), no system admin stuff (sudo, systemctl), no scripting (loops, variables), no permissions (chmod, users).
We thought of basic bash, but we don't know if we can handle it.
