Hare-1.1B-base is a pre-trained model developed by the LiteAI Team from China Telecom Guizhou Branch. We use a mix of high-quality open-source data and strategy-generated data as pre-train data. The model is only 1.1B in size and has performed well on the Open LLM Leaderboard.
We chose Mistral as the foundational architecture and reused its tokenizer, reducing the number of parameters by adjusting the hyperparameters of its model architecture. Consequently, our model can be directly applied to numerous open-source projects that support Mistral, such as vLLM.
Our model has a parameter count of only 1.1 billion, allowing us to deploy it on consumer-grade GPUs, mobile devices, and other cost-effective platforms.
We have explored efficient training at FP8 precision and have compiled a set of best practices, hoping to contribute as much as we can to LLM training in the open-source community. For best practices, please see our GitHub homepage.
We are currently developing and adapting for Chinese language support.
1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM
34device ="cuda"if torch.cuda.is_available()else"cpu"5model_path ="LiteAI-Team/Hare-1.1B-base"6tokenizer = AutoTokenizer.from_pretrained(model_path)7model = AutoModelForCausalLM.from_pretrained(model_path)8model.to(device)910prompt ="Write a poem based on the landscape of Guizhou:"11tokens = tokenizer(prompt, add_special_tokens=True, return_tensors='pt').to(device)12output = model.generate(**tokens,max_new_tokens=128)1314output_tokens = output[0].cpu().numpy()[tokens.input_ids.size()[1]:]15output_string = tokenizer.decode(output_tokens)16print(output_string)17>>"""The Guizhou landscape is a sight to behold,
18A place where nature's beauty is unmatched,
19A land of towering mountains and vast plains,
20A paradise for those who seek to explore.
2122The mountains rise high above the sky,
23A sight to beholder, a sight to see,
24The valleys stretch out as far as the eye can see,
25A landscape of endless beauty and grace."""
Install with vllm:
pip install vllm
python
1from vllm import LLM, SamplingParams
2from transformers import AutoTokenizer
34model_path ="LiteAI-Team/Hare-1.1B-base"5llm = LLM(model=model_path, trust_remote_code=True, tensor_parallel_size=4)67query ="Write a poem based on the landscape of Guizhou:"8sampling_params = SamplingParams(temperature=0.8, top_p=0.95, max_tokens=64)9outputs = llm.generate(query, sampling_params)10print(outputs)
Edge Deployment Demo 端侧部署
Our model has only 1.1 billion parameters, and after Int4 quantization, it occupies just 0.6GB of space, allowing for easy deployment on mobile devices, The Hare-1.1B-Chat model weights have been open-sourced.
Android：We chose MLC-LLM as the deployment framework and conducted deployment testing of the Chat model on the Redmi K40.
iOS & HarmonyOS：We will conduct deployment testing on the aforementioned devices in the future.
To fully leverage the advantages of deploying small models on edge devices, we referred to the work of Octopus-v2 and replaced Gemma-2B with Hare-1.1B-Tool, successfully enabling the invocation of Android system APIs and the orchestration of tool functionalities in composite scenarios on mobile devices.
Additionally, we conducted explorations and experiments addressing the issue of benchmark data leakage. For a detailed analysis, please refer to our paper.
This repository is open-sourced under the Apache-2.0 license.
The Hare series model weights are currently fully open only for academic research.
本仓库遵循Apache-2.0协议开源。
Hare系列模型权重目前仅对学术研究完全开放。
Statement 声明
Hare is a language model trained on a mix of open-source pre-training data and strategy-generated pre-training data. It lacks the ability to make value judgments and cannot understand or express personal opinions. The outputs of the model do not represent the views or positions of the LiteAI development team.
Therefore, the content generated using Hare may contain biased viewpoints and inaccuracies. Please use it at your discretion.
Similarly, we will not assume any responsibility for risks and issues arising from users deliberately using Hare to generate harmful content.
For modifications related to this repository, please contact: zhangly41 At(@) chinatelecom.cn.
Team contact information: chensq27 At(@) chinatelecom.cn. The LiteAI Team looks forward to collaborating with you.
@misc{zhang2024harehumanpriorskey,
title={HARE: HumAn pRiors, a key to small language model Efficiency},
author={Lingyun Zhang and Bin jin and Gaojian Ge and Lunhui Liu and Xuewen Shen and Mingyong Wu and Houqian Zhang and Yongneng Jiang and Shiqi Chen and Shi Pu},
year={2024},
eprint={2406.11410},
archivePrefix={arXiv},
primaryClass={cs.CL}
url={https://arxiv.org/abs/2406.11410},
}
