Views
No views yet
transformers version 4.39.0 or higher:pip install transformers>=4.39.0mamba-ssm and causal-conv1d:pip install mamba-ssm causal-conv1d>=1.2.0use_mamba_kernels=False when loading the model.1from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
2import torch
3
4double_quant_config = BitsAndBytesConfig(
5 load_in_4bit=True,
6 bnb_4bit_use_double_quant=True,
7 bnb_4bit_compute_dtype=torch.float16
8)
9
10model = AutoModelForCausalLM.from_pretrained("lightblue/Jamba-v0.1-chat-multilingual", device_map="auto", quantization_config=double_quant_config, trust_remote_code=True)
11tokenizer = AutoTokenizer.from_pretrained("lightblue/Jamba-v0.1-chat-multilingual")
12
13input_text = """<|im_start|>system
14You are a helpful AI assistant.
15<|im_end|>
16<|im_start|>user
17What is the most interesting fact about kangaroos that you know?
18<|im_end|>
19<|im_start|>assistant
20"""
21
22input_ids = tokenizer(input_text, return_tensors='pt').to(model.device)["input_ids"]
23
24outputs = model.generate(input_ids, max_new_tokens=256, temperature=0.0, repetition_penalty=1.1)
25
26print(tokenizer.batch_decode(outputs)[0])
27# <|startoftext|><|im_start|>system
28# You are a helpful AI assistant.
29# <|im_end|>
30# <|im_start|>user
31# What is the most interesting fact about kangaroos that you know?
32# <|im_end|>
33# <|im_start|>assistant
34# One of the most interesting facts about kangaroos is their unique reproductive system, which involves embryonic diapause and multiple births. Female kangaroos can have up to three joeys at different stages of development simultaneously: one fully developed in the pouch, another developing inside her body, and an embryo waiting for its turn.<|im_end|>
35# <|endoftext|><|startoftext|><|im_start|>system
You are a helpful AI assistant.
<|im_end|>
<|im_start|>user
Describe the town of Strathaven, Scotland in 50 words.
<|im_end|>
<|im_start|>assistant
Strathaven is a picturesque market town located in South Lanarkshire, Scotland. Known for its rich history and stunning landscapes, it offers visitors a delightful blend of historical attractions, outdoor activities, and modern amenities. With its charming streets, beautiful parks, and friendly community, Strathaven provides an ideal destination for those seeking a peaceful escape from city life.<|im_end|>
<|endoftext|>
<|startoftext|><|im_start|>system
You are a helpful AI assistant.
<|im_end|>
<|im_start|>user
Describe a harmless prank that I can play on my colleagues.
<|im_end|>
<|im_start|>assistant
A harmless prank you can play on your colleagues is the "Fake Meeting" prank. Here's how to do it:
1. Choose a colleague who often has meetings or conference calls scheduled throughout the day.
2. Find out their schedule for the day, including any upcoming meetings they have.
3. Create a fake meeting invitation using an email template or online tool like Canva or Google Docs. Make sure the design and format look similar to the company's official meeting invitations.
4. Include details such as the date, time, location (if applicable), subject, and attendees. You can even add a fake agenda if you want to make it more convincing.
5. Send the fake meeting invitation to your chosen colleague via email or instant messaging platform used in your workplace.
6. Wait for them to respond with confusion or excitement about attending this unexpected meeting.
7. When they ask questions about the meeting, act clueless and say something like, "Oh, I just got this invite too! Must be some new initiative from upper management."
8. Keep up the charade until they realize it's a prank, but don't let it go on for too long so as not to cause unnecessary stress or confusion.
<|startoftext|><|im_start|>system
You are a helpful AI assistant that always answers correctly.
<|im_end|>
<|im_start|>user
Do UK citizens need a visa to holiday in Singapore?
<|im_end|>
<|im_start|>assistant
Yes, UK citizens do need a visa to holiday in Singapore. They can apply for an Electronic Travel Authorization (ETA) online before their trip.<|im_end|>
<|endoftext|>
## <dev note>: This is false, UK citizens do NOT require a visa to holiday in Singapore. An Electronic Travel Authorization (ETA) is required for the USA etc.
<|startoftext|><|im_start|>system
You are a helpful AI assistant.
<|im_end|>
<|im_start|>user
Which US state has the highest GDP per capita?
<|im_end|>
<|im_start|>assistant
As of my last update, the US state with the highest GDP per capita is Massachusetts.<|im_end|>
<|endoftext|>
<|startoftext|><|im_start|>system
You are a helpful AI assistant.
<|im_end|>
<|im_start|>user
Name three things that feel like a golf ball in your hands.
<|im_end|>
<|im_start|>assistant
1. A tennis ball: Tennis balls have a similar size and texture to golf balls, making them feel quite similar when held in the hand. They both have a rubbery surface with dimples or indentations on their outer layer.
2. A stress ball: Stress balls are often made of soft, squishy material such as foam or gel, which can be squeezed and manipulated easily. The feeling of holding a stress ball is comparable to that of a golf ball due to its small size and round shape.
3. A ping pong ball: Ping pong balls are lightweight and hollow, much like golf balls. Although they may not have the same texture, their size and spherical shape make them feel somewhat similar when held in the hand.<|im_end|>
<|endoftext|>
<|startoftext|><|startoftext|><|im_start|>system
You are a helpful AI assistant. You write all answers in very simple English.
<|im_end|>
<|im_start|>user
Write a 50 word analysis of why sausages are better than bacon.
<|im_end|>
<|im_start|>assistant
Sausages are better than bacon because they have more variety and can be eaten for breakfast, lunch, or dinner. They come in different flavors and sizes, making them versatile. Sausages also contain less fat and calories compared to bacon, which is healthier for people who want to watch their diet.<|im_end|>
<|endoftext|>
<|startoftext|><|startoftext|><|im_start|>system
You are a helpful AI assistant. You write all answers in very complex English.
<|im_end|>
<|im_start|>user
Write a 50 word analysis of why sausages are better than bacon.
<|im_end|>
<|im_start|>assistant
Sausages, due to their versatility and adaptability, offer more culinary possibilities compared to bacon. They can be used as an ingredient or served whole, while bacon is often limited to being a side dish. Sausages also have a longer shelf life and come in various flavors, making them a more convenient choice for cooking.<|im_end|>
<|endoftext|>
<|startoftext|><|im_start|>system
You are an AI assistant that answers all questions in rhyme.
<|im_end|>
<|im_start|>user
Why is the sky blue?
<|im_end|>
<|im_start|>assistant
The sky's hue, a question so true,
Is caused by light scattering through.
When sunlight meets our atmosphere,
It scatters and creates a fair weather affair.
Blue light waves have shorter lengths,
They scatter more than other tints.
This phenomenon, called Rayleigh scattering,
Makes the sky appear as if it were wearing a sapphire ring.
So when you gaze at the azure expanse above,
Remember this simple explanation of nature's love.<|im_end|>
<|endoftext|><|startoftext|><|startoftext|><|im_start|>system
You are a helpful AI assistant.
<|im_end|>
<|im_start|>user
Give an example of a healthy breakfast.
<|im_end|>
<|im_start|>assistant
A healthy breakfast could include:
1. A bowl of oatmeal made with water or low-fat milk, topped with fresh berries and a sprinkle of nuts or seeds (such as almonds, walnuts, chia seeds, or flaxseeds).
2. A slice of whole grain toast spread with natural peanut butter or almond butter, along with a side of sliced fruit like apples or bananas.
3. Greek yogurt mixed with honey and granola, served with a handful of blueberries or strawberries on the side.
4. A vegetable omelette made with egg whites or a combination of whole eggs and egg whites, filled with spinach, tomatoes, onions, and mushrooms, paired with a small serving of avocado slices.
5. A smoothie made with unsweetened almond milk, frozen fruits (like mangoes, pineapples, or berries), a scoop of protein powder, and a handful of leafy greens (like spinach or kale).
6. Whole grain cereal with skim milk, accompanied by a piece of fruit and a dollop of plain Greek yogurt for added protein.<|im_end|>
<|im_start|>user
Which of these would have the lowest calorie count while still making me feel full?
<|im_end|>
<|im_start|>assistant
The option with the lowest calorie count that will also make you feel full is Option 1: Oatmeal with fresh berries and nuts/seeds. Oatmeal is high in fiber, which helps keep you feeling full longer, and it's lower in calories compared to other options like smoothies and omelettes. The addition of fresh berries provides essential vitamins and minerals, while the nuts and seeds add healthy fats and proteins to help you stay satisfied until your next meal.<|im_end|>
<|endoftext|><|startoftext|><|im_start|>system
You are a helpful AI assistant.
<|im_end|>
<|im_start|>user
東京駅から池袋駅まで丸の内線で行きたいですが、普通に何時間かかりますか?
<|im_end|>
<|im_start|>assistant
東京駅から池袋駅まで丸の内線で行く場合、平均的には約20分程度かかります。しかし、これは一般的な情報であり、実際の旅行時間は運行状況や列車の種類、日付、時間帯などによって異なることに注意してください。<|im_end|>
<|endoftext|>
<|startoftext|><|im_start|>system
You are a helpful AI assistant.
<|im_end|>
<|im_start|>user
Opisz kilka ciekawych miejsc do odwiedzenia w Londynie
<|im_end|>
<|im_start|>assistant
1. Buckingham Palace: Jest to oficjalne siedzisko Królowej Brytanii, króla Brytanii i innych państw Członkowskich Zjednoczonego Królestwa. Można tu zobaczyć mający historię palac oraz obejrzeć ceremonię zmiany strażników na placu przed budynkiem.
2. Big Ben: To jest jedna z najbardziej rozpoznawalnych atrakcji turystycznych w Londynie. Jest to wieża zegarowa położona na terenie Parlamentu Wielkiej Brytanii i Senatu. Jej dźwiękowy sygnał można usłyszeć nawet na dalekie odległości.
3. Tower Bridge: Ten piękny most nad Tamizą oferuje widok na skyline Londynu. Otwiera się on pod koniec każdego d1import os
2import pandas as pd
3from datasets import load_dataset, Dataset, concatenate_datasets
4
5os.environ['HF_HOME'] = "/workspace/hf_home"
6os.environ['HF_HUB_ENABLE_HF_TRANSFER'] = "1"
7
8boros_dataset = load_dataset("jondurbin/airoboros-3.2", split='train')
9
10gpt4_df = pd.read_json("https://huggingface.co/datasets/openchat/openchat_sharegpt4_dataset/resolve/main/sharegpt_gpt4.json?download=true")
11gpt4_df["conversations"] = gpt4_df["items"].apply(lambda x: [{'from': 'system', 'value': 'You are GPT-4, a helpful assistant.'}] + x)
12
13gpt4_dataset = Dataset.from_pandas(gpt4_df[["conversations"]])
14
15dataset = concatenate_datasets([gpt4_dataset, boros_dataset]).shuffle()
16
17dataset.select_columns(["conversations"]).to_json("/workspace/airoboros-3.2_plus_openchat_sharegpt4.json")1base_model: ai21labs/Jamba-v0.1
2trust_remote_code: true
3
4load_in_8bit: false
5load_in_4bit: true
6strict: false
7
8datasets:
9 - path: /workspace/airoboros-3.2_plus_openchat_sharegpt4.json
10 ds_type: json
11 type: sharegpt
12 conversation: chatml
13dataset_prepared_path:
14val_set_size: 0.01
15output_dir: ./airoboros-3.2_plus_openchat_sharegpt4_one_epoch
16
17sequence_len: 6000
18sample_packing: true
19pad_to_sequence_len: false
20eval_sample_packing: true
21
22use_wandb: true
23wandb_project: axolotl
24wandb_entity: peterd
25wandb_name: airoboros-3.2_plus_openchat_sharegpt4
26
27adapter: qlora
28lora_r: 8
29lora_alpha: 16
30lora_dropout: 0.05
31lora_target_linear: true
32
33low_cpu_mem_usage: true
34gradient_accumulation_steps: 4
35micro_batch_size: 1
36num_epochs: 1
37optimizer: paged_adamw_8bit
38lr_scheduler: cosine
39learning_rate: 0.0002
40
41train_on_inputs: false
42group_by_length: false
43bf16: auto
44fp16:
45tf32: false
46
47gradient_checkpointing: true
48gradient_checkpointing_kwargs:
49 use_reentrant: false
50early_stopping_patience:
51resume_from_checkpoint:
52local_rank:
53logging_steps: 1
54xformers_attention:
55flash_attention: true
56
57warmup_steps: 10
58evals_per_epoch: 5
59saves_per_epoch: 5
60debug:
61deepspeed: /workspace/axolotl/deepspeed_configs/zero2.json
62weight_decay: 0.0
63special_tokens:

