Views
No views yet
code-python-0.60-to-1.00 and web-0.50-to-1.00 subsets of AutoMathText. We also added 1M files from The Stack's Jupyter Notebooks, converted to script. They tend to have educational code interleaved with text.
We also included ultrachat formatted in the chat format of LlaMa models, so we don't have to instruction-tune the model after the pre-training. Additionally, we upsampled twice the data from these seed sources to help with commonsense and reasoning: stories, AutoMathText & KhanAcademy.1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3device = "cuda" # for GPU usage or "cpu" for CPU usage
4
5tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/cosmo-1b")
6model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/cosmo-1b").to(device)
7prompt = "Generate a story involving a dog, an astronaut and a baker"
8prompt= tokenizer.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False)
9inputs = tokenizer(prompt, return_tensors="pt").to(device)
10
11output = model.generate(**inputs, max_length=300, do_sample=True, temperature=0.6, top_p=0.95, repetition_penalty=1.2)
12print(tokenizer.decode(output[0]))<s><s> [INST] Generate a story involving a dog, an astronaut and a baker [/INST] Once upon a time, there was a sweet little terrier named Max who lived in the bustling city of New York. He loved nothing more than chasing after his owner, Dr. Sarah Johnson, as she worked tirelessly on her latest invention - a spaceship that would take humans to distant galaxies!
One day, Dr. Johnson decided it was time for her to leave Earth's atmosphere behind and embark on this exciting adventure with her loyal companion, Max. She knew he had always been fascinated by space travel, so she hoped he would be just as excited about the journey ahead.
As they boarded their rocket ship and blasted off into outer space, Max felt both nervous and thrilled at the same time. His ears perked up every time they passed clouds or saw stars twinkling far out from earth. But as days turned into weeks, Max started feeling homesick. The vast emptiness around him made him feel lonely and isolated.
Meanwhile back on planet Earth, Mr. Baker was busy baking cookies when suddenly, an idea popped into his head. Why not send some treats along with Dr. Johnson's family? It might make them all feel better knowing that someone else was also having fun exploring the universe.1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3device = "cuda" # for GPU usage or "cpu" for CPU usage
4
5tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/cosmo-1b")
6model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/cosmo-1b").to(device)
7prompt = "Photosynthesis is"
8
9inputs = tokenizer(prompt, return_tensors="pt").to(device)
10output = model.generate(**inputs, max_length=300, do_sample=True, temperature=0.6, top_p=0.95, repetition_penalty=1.2)
11print(tokenizer.decode(output[0]))<s> Photosynthesis is the process by which green plants, algae and some bacteria convert light energy into chemical energy in order to fuel their metabolic processes. The reaction takes place within specialized cells called chloroplasts. This article focuses on the electron transport chain (ETC), a critical part of photosystem II where most of the solar-driven electrons are passed through before being reduced to water.
