Views
No views yet
| Model | # Total Params | Context Length | Download Model | Download GGUF files |
|---|---|---|---|---|
| xLAM-1b-fc-r | 1.35B | 16384 | 🤗 Link | 🤗 Link |
| xLAM-7b-fc-r | 6.91B | 4096 | 🤗 Link | 🤗 Link |
fc series of models are optimized for function-calling capability, providing fast, accurate, and structured responses based on input queries and available APIs. These models are fine-tuned based on the deepseek-coder models and are designed to be small enough for deployment on personal devices like phones or computers.xLAM-1b-fc-r model, which is optimized for function-calling and can be easily deployed on personal devices.
xLAM-1b-fc-r with HuggingFace and vLLM.
We will first introduce the basic usage, and then walk through the provided tutorial and example scripts in the examples folder.
temperature=0.001 and top_p=1xLAM-7b-fc-r secures the 3rd place with an overall accuracy of 88.24% on the leaderboard, outperforming many strong models. Notably, our xLAM-1b-fc-r model is the only tiny model with less than 2B parameters on the leaderboard, but still achieves a competitive overall accuracy of 78.94% and outperforming GPT3-Turbo and many larger models.
Both models exhibit balanced performance across various categories, showing their strong function-calling capabilities despite their small sizes.xLAM-1b-fc-r model from Huggingface, please first install the transformers library:pip install transformers>=4.41.01import json
2import torch
3from transformers import AutoModelForCausalLM, AutoTokenizer
4
5torch.random.manual_seed(0)
6
7model_name = "Salesforce/xLAM-1b-fc-r"
8model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True)
9tokenizer = AutoTokenizer.from_pretrained(model_name)
10
11# Please use our provided instruction prompt for best performance
12task_instruction = """
13You are an expert in composing functions. You are given a question and a set of possible functions.
14Based on the question, you will need to make one or more function/tool calls to achieve the purpose.
15If none of the functions can be used, point it out and refuse to answer.
16If the given question lacks the parameters required by the function, also point it out.
17""".strip()
18
19format_instruction = """
20The output MUST strictly adhere to the following JSON format, and NO other text MUST be included.
21The example format is as follows. Please make sure the parameter type is correct. If no function call is needed, please make tool_calls an empty list '[]'.
22```
23{
24 "tool_calls": [
25 {"name": "func_name1", "arguments": {"argument1": "value1", "argument2": "value2"}},
26 ... (more tool calls as required)
27 ]
28}
29```
30""".strip()
31
32# Define the input query and available tools
33query = "What's the weather like in New York in fahrenheit?"
34
35get_weather_api = {
36 "name": "get_weather",
37 "description": "Get the current weather for a location",
38 "parameters": {
39 "type": "object",
40 "properties": {
41 "location": {
42 "type": "string",
43 "description": "The city and state, e.g. San Francisco, New York"
44 },
45 "unit": {
46 "type": "string",
47 "enum": ["celsius", "fahrenheit"],
48 "description": "The unit of temperature to return"
49 }
50 },
51 "required": ["location"]
52 }
53}
54
55search_api = {
56 "name": "search",
57 "description": "Search for information on the internet",
58 "parameters": {
59 "type": "object",
60 "properties": {
61 "query": {
62 "type": "string",
63 "description": "The search query, e.g. 'latest news on AI'"
64 }
65 },
66 "required": ["query"]
67 }
68}
69
70openai_format_tools = [get_weather_api, search_api]
71
72# Helper function to convert openai format tools to our more concise xLAM format
73def convert_to_xlam_tool(tools):
74 ''''''
75 if isinstance(tools, dict):
76 return {
77 "name": tools["name"],
78 "description": tools["description"],
79 "parameters": {k: v for k, v in tools["parameters"].get("properties", {}).items()}
80 }
81 elif isinstance(tools, list):
82 return [convert_to_xlam_tool(tool) for tool in tools]
83 else:
84 return tools
85
86# Helper function to build the input prompt for our model
87def build_prompt(task_instruction: str, format_instruction: str, tools: list, query: str):
88 prompt = f"[BEGIN OF TASK INSTRUCTION]\n{task_instruction}\n[END OF TASK INSTRUCTION]\n\n"
89 prompt += f"[BEGIN OF AVAILABLE TOOLS]\n{json.dumps(xlam_format_tools)}\n[END OF AVAILABLE TOOLS]\n\n"
90 prompt += f"[BEGIN OF FORMAT INSTRUCTION]\n{format_instruction}\n[END OF FORMAT INSTRUCTION]\n\n"
91 prompt += f"[BEGIN OF QUERY]\n{query}\n[END OF QUERY]\n\n"
92 return prompt
93
94# Build the input and start the inference
95xlam_format_tools = convert_to_xlam_tool(openai_format_tools)
96content = build_prompt(task_instruction, format_instruction, xlam_format_tools, query)
97
98messages=[
99 { 'role': 'user', 'content': content}
100]
101inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
102
103# tokenizer.eos_token_id is the id of <|EOT|> token
104outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
105print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)){"tool_calls": [{"name": "get_weather", "arguments": {"location": "New York", "unit": "fahrenheit"}}]}vllm and run inferences. First, install the required packages:pip install vllm openai argparse jinja2python test_prompt_template.py --model python -m vllm.entrypoints.openai.api_server --model Salesforce/xLAM-1b-fc-r --served-model-name xlam-1b-fc-r --dtype bfloat16 --port 8001python test_xlam_model_with_endpoint.py --model_name xlam-1b-fc-r --port 8001 [OPTIONS]--temperature: Default 0.3--top_p: Default 1.0--max_tokens: Default 512python test_xlam_model_with_vllm.py --model Salesforce/xLAM-1b-fc-r [OPTIONS]--dtype parameter when serving the model based on your GPU capacity.demo.ipynb file for a comprehensive description of the entire workflow, including how to execute APIs.xLAM-1b-fc-r is distributed under the CC-BY-NC-4.0 license, with additional terms specified in the Deepseek license.1@article{liu2024apigen,
2 title={APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets},
3 author={Liu, Zuxin and Hoang, Thai and Zhang, Jianguo and Zhu, Ming and Lan, Tian and Kokane, Shirley and Tan, Juntao and Yao, Weiran and Liu, Zhiwei and Feng, Yihao and others},
4 journal={arXiv preprint arXiv:2406.18518},
5 year={2024}
6}