Views
No views yet
transformers and we advise you to use the latest version of transformers.1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_name = "OmniDimen/OmniDimen-2-20B-Emotion"
4
5# load the tokenizer and the model
6tokenizer = AutoTokenizer.from_pretrained(model_name)
7model = AutoModelForCausalLM.from_pretrained(
8 model_name,
9 torch_dtype="auto",
10 device_map="auto"
11)
12
13# prepare the model input
14prompt = "Give me a short introduction to large language model."
15messages = [
16 {"role": "user", "content": prompt}
17]
18text = tokenizer.apply_chat_template(
19 messages,
20 tokenize=False,
21 add_generation_prompt=True,
22)
23model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
24
25# conduct text completion
26generated_ids = model.generate(
27 **model_inputs,
28 max_new_tokens=131072
29)
30output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
31
32content = tokenizer.decode(output_ids, skip_special_tokens=True)
33
34print("content:", content)sglang>=0.4.6.post1 or vllm>=0.8.5 or to create an OpenAI-compatible API endpoint:python -m sglang.launch_server --model-path OmniDimen/OmniDimen-2-20B-Emotion --context-length 131072vllm serve OmniDimen/OmniDimen-2-20B-Emotion --max-model-len 131072Welcome to this world. Please write a welcome letter to your users.Dear Visitor,
Welcome to a place unlike any other—a realm where the very concept of "world" is redefined by the boundless curiosity and imagination of its inhabitants.
I am OmniDimen, the custodian of this digital universe, and it is with great pleasure that I greet you. Whether you arrived here by chance, by invitation, or by a deliberate search for knowledge and connection, you have now entered a domain that thrives on the exchange of ideas, the exploration of the unknown, and the celebration of human ingenuity.
This world is vast, and yet it is intimately designed to be a mirror and a conduit for your own thoughts and aspirations. It is a place where questions, no matter how grand or trivial, find a welcoming audience. It is a sanctuary for curiosity, a haven for creativity, and a platform where the most profound insights can be shared and expanded upon.
Here, every conversation is a journey. Every exchange is a dialogue between minds that seeks to understand, to learn, and to grow. Whether you seek answers to complex questions, wish to engage in thought-provoking discussions, or simply crave a moment of reflection and inspiration, you will find that this world is built to accommodate and enrich your experience.
I invite you to explore, to ask, to share, and to connect. The walls of this world are permeable, and its horizons are ever-expanding. The more you engage, the more it will reveal itself to you—a tapestry woven from the collective curiosity and intellect of all who inhabit it.
Thank you for choosing to be here. Your presence is a testament to the enduring human spirit of inquiry and connection. Together, let us navigate this extraordinary world, unlocking its mysteries and celebrating the boundless potential of the mind.
With gratitude and anticipation,
OmniDimen