You can deploy this model and start earning money today!

RichardErkhov/
argilla_-_Llama-3.2-1B-Instruct-APIGen-FC-v0.1-gguf

Name: argilla_-_Llama-3.2-1B-Instruct-APIGen-FC-v0.1-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3.2-1B-Instruct-APIGen-FC-v0.1 - GGUF

Model creator: https://huggingface.co/argilla/
Original model: https://huggingface.co/argilla/Llama-3.2-1B-Instruct-APIGen-FC-v0.1/

Name	Quant method	Size
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q2_K.gguf	Q2_K	0.54GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.IQ3_XS.gguf	IQ3_XS	0.58GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.IQ3_S.gguf	IQ3_S	0.6GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q3_K_S.gguf	Q3_K_S	0.6GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.IQ3_M.gguf	IQ3_M	0.61GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q3_K.gguf	Q3_K	0.64GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q3_K_M.gguf	Q3_K_M	0.64GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q3_K_L.gguf	Q3_K_L	0.68GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.IQ4_XS.gguf	IQ4_XS	0.7GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q4_0.gguf	Q4_0	0.72GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.IQ4_NL.gguf	IQ4_NL	0.72GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q4_K_S.gguf	Q4_K_S	0.72GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q4_K.gguf	Q4_K	0.75GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q4_K_M.gguf	Q4_K_M	0.75GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q4_1.gguf	Q4_1	0.77GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q5_0.gguf	Q5_0	0.83GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q5_K_S.gguf	Q5_K_S	0.83GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q5_K.gguf	Q5_K	0.85GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q5_K_M.gguf	Q5_K_M	0.85GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q5_1.gguf	Q5_1	0.89GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q6_K.gguf	Q6_K	0.95GB
Llama-3.2-1B-Instruct-APIGen-FC-v0.1.Q8_0.gguf	Q8_0	1.23GB

Original model description:

base_model: meta-llama/Llama-3.2-1B-Instruct datasets:

argilla-warehouse/apigen-synth-trl library_name: transformers model_name: Llama-3.2-1B-Instruct-APIGen-FC-v0.1 tags:
generated_from_trainer
trl
sft licence: license license: apache-2.0 language:
en

Model Card for Llama-3.2-1B-Instruct-APIGen-FC-v0.1

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on argilla-warehouse/apigen-synth-trl dataset, a version of argilla/Synth-APIGen-v0.1 ready to do SFT on top of it. It has been trained using TRL.

Quick start

This is a Fine tuned version of Llama-3.2-1B-Instruct model specific for Function Calling, to showcase how to fine tune a model on top of a dataset like argilla/Synth-APIGen-v0.1.

Helper functions for the prompt and output parsing

Click to see helper functions

python

from typing import Optional
import re
import json

from jinja2 import Template

SYSTEM_PROMPT = """
You are an expert in composing functions. You are given a question and a set of possible functions. 
Based on the question, you will need to make one or more function/tool calls to achieve the purpose. 
If none of the functions can be used, point it out and refuse to answer. 
If the given question lacks the parameters required by the function, also point it out.

The output MUST strictly adhere to the following format, and NO other text MUST be included.
The example format is as follows. Please make sure the parameter type is correct. If no function call is needed, please make the tool calls an empty list '[]'.
```
<tool_call>[
{"name": "func_name1", "arguments": {"argument1": "value1", "argument2": "value2"}},
... (more tool calls as required)
]</tool_call>
```
""".strip()

prompt = Template("""
You have access to the following tools:
<tools>{{ tools }}</tools>

Please answer the following query:
{{ query }}
""".lstrip())

def prepare_messages(
    query: str,
    tools: Optional[dict[str, any]] = None,
    conversation_history: Optional[list[dict[str, str]]] = None
) -> list[dict[str, str]]:
    """Prepare the system and user messages for the given query and tools.
    
    Args:
        query: The query to be answered.
        tools: The tools available to the user. Defaults to None, in which case if a
            list without content will be passed to the model.
        conversation_history: Exchange of messages, including the system_prompt from
            the first query. Defaults to None, the first message in a conversation.
    """
    if tools is None:
        tools = []

    if conversation_history:
        messages = conversation_history.copy()
        messages.append({"role": "user", "content": query})
    else:
        messages = [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": prompt.render(tools=json.dumps(tools), query=query)}
        ]

    return messages


def parse_response(text: str) -> str | dict[str, any]:
    """Parses a response from the model, returning either the
    parsed list with the tool calls parsed, or the
    model thought or response if couldn't generate one.

    Args:
        text: Response from the model.
    """
    pattern = r"<tool_call>(.*?)</tool_call>"
    matches = re.findall(pattern, text, re.DOTALL)
    if matches:
        return json.loads(matches[0])
    return text
    

Examples

The following examples show how to use the model with transformers, for different types of queries and depending on the availability of tools.

Example of simple function call:

python

import torch 
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "argilla-warehouse/Llama-3.2-1B-Instruct-APIGen-FC-v0.1"
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_name)


get_weather_api = {
    "name": "get_weather",
    "description": "Get the current weather for a location",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, New York"
            },
            "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
                "description": "The unit of temperature to return"
            }
        },
        "required": ["location"]
    }
}

search_api = {
    "name": "search",
    "description": "Search for information on the internet",
    "parameters": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "The search query, e.g. 'latest news on AI'"
            }
        },
        "required": ["query"]
    }
}

available_tools = [get_weather_api, search_api]

query = "What's the weather like in New York in fahrenheit?"

messages = prepare_messages(query, tools=available_tools)

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=False)

response = parse_response(result)
# [{'name': 'get_weather', 'arguments': {'location': 'New York', 'unit': 'fahrenheit'}}]

`Parallel` function call

Click here:

python

available_tools = [{"name": "spotify.play", "description": "Play specific tracks from a given artist for a specific time duration.", "parameters": {"type": "dict", "properties": {"artist": {"type": "string", "description": "The artist whose songs you want to play."}, "duration": {"type": "integer", "description": "The duration for which the songs should be played, in minutes."}}, "required": ["artist", "duration"]}}]
query = "Play songs from the artists Taylor Swift and Maroon 5, with a play time of 20 minutes and 15 minutes respectively, on Spotify."

messages = prepare_messages(query, tools=available_tools)

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=False)

response = parse_response(result)
# [{'name': 'spotify.play', 'arguments': {'artist': 'Taylor Swift', 'duration': 20}}, {'name': 'spotify.play', 'arguments': {'artist': 'Maroon 5', 'duration': 15}}]

`Multiple` function call

Click here:

python

available_tools = [{"name": "country_info.largest_city", "description": "Fetch the largest city of a specified country.", "parameters": {"type": "dict", "properties": {"country": {"type": "string", "description": "Name of the country."}}, "required": ["country"]}}, {"name": "country_info.capital", "description": "Fetch the capital city of a specified country.", "parameters": {"type": "dict", "properties": {"country": {"type": "string", "description": "Name of the country."}}, "required": ["country"]}}, {"name": "country_info.population", "description": "Fetch the current population of a specified country.", "parameters": {"type": "dict", "properties": {"country": {"type": "string", "description": "Name of the country."}}, "required": ["country"]}}]
query = "What is the capital of Brazil?"

messages = prepare_messages(query, tools=available_tools)

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=False)

response = parse_response(result)
# [{'name': 'country_info.capital', 'arguments': {'country': 'Brazil'}}]

`Parallel multiple` function call

Click here:

python

available_tools = [{"name": "math_toolkit.sum_of_multiples", "description": "Find the sum of all multiples of specified numbers within a specified range.", "parameters": {"type": "dict", "properties": {"lower_limit": {"type": "integer", "description": "The start of the range (inclusive)."}, "upper_limit": {"type": "integer", "description": "The end of the range (inclusive)."}, "multiples": {"type": "array", "items": {"type": "integer"}, "description": "The numbers to find multiples of."}}, "required": ["lower_limit", "upper_limit", "multiples"]}}, {"name": "math_toolkit.product_of_primes", "description": "Find the product of the first n prime numbers.", "parameters": {"type": "dict", "properties": {"count": {"type": "integer", "description": "The number of prime numbers to multiply together."}}, "required": ["count"]}}]
query = "Find the sum of all the multiples of 3 and 5 between 1 and 1000. Also find the product of the first five prime numbers."

messages = prepare_messages(query, tools=available_tools)

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=False)

response = parse_response(result)
# [{'name': 'math_toolkit.sum_of_multiples', 'arguments': {'lower_limit': 1, 'upper_limit': 1000, 'multiples': [3, 5]}}, {'name': 'math_toolkit.product_of_primes', 'arguments': {'count': 5}}]

`Multi-turn` function call

Click here:

python

get_weather_api = {
    "name": "get_weather",
    "description": "Get the current weather for a location",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, New York"
            },
            "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
                "description": "The unit of temperature to return"
            }
        },
        "required": ["location"]
    }
}

available_tools = [get_weather_api]

query = "What's the weather like in Madrid in celsius?"

messages = prepare_messages(query, tools=available_tools)

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=False)

response = parse_response(result)

# 2nd turn
conversation_history = messages.copy()
conversation_history.append({"role": "assistant", "content": json.dumps(response)})

new_query = "And in Edinburgh in celsius?"

new_messages = prepare_messages(new_query, tools=available_tools, conversation_history=conversation_history)

inputs = tokenizer.apply_chat_template(new_messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=False)

response = parse_response(result)
# [{'name': 'get_weather', 'arguments': {'location': 'Edinburgh', 'unit': 'celsius'}}]

`Irrelevance` function call (examples when some data is missing)

Click here:

Example response with no tools available

python

available_tools = []

query = "What's the weather like in New York in fahrenheit?"

messages = prepare_messages(query, tools=available_tools)

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)
response = parse_response(result)
# 'The query cannot be answered, no tools were provided.'

Example when a wrong tool is informed:

python

cut_number = {
  'type': 'function',
  'function': {
       'name': 'cut_number',
       'description': 'Returns the value `number` if it is greater than or equal to `threshold`, otherwise returns the value `threshold`.',
       'parameters': {
          'type': 'object',
          'properties': {'number': {'type': 'number', 'description': 'The number to compare.'}},
      'required': ['number']
      }
  }
}

available_tools = [cut_number]

query = "What's the weather like in New York in fahrenheit?"

messages = prepare_messages(query, tools=available_tools)
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)
response = parse_response(result)
# "The query cannot be answered with the provided tools. The query lacks the parameters required by the function. Please provide the parameters, and I'll be happy to assist."

Training procedure

Visualize in Weights & Biases

This model was trained with SFT. You can take a look at sft.slurm to see the training script, if you don't have access to a slurm cluster, it can be run jsut using the accelerate command. It took 13 minutes in a node with 8xH100.

To install the requirements, the following commands can be used:

bash

uv venv .venv --python 3.11
source .venv/bin/activate
git clone https://github.com/huggingface/trl.git
uv pip install .
uv pip install wandb
uv pip install deepspeed

And login to your WandB and Hugging Face accounts to push both logs and the final model.

Framework versions

TRL: 0.12.0.dev0
Transformers: 4.45.1
Pytorch: 2.4.1
Datasets: 3.0.1
Tokenizers: 0.20.0

Citations

Cite TRL as:

bibtex

@misc{vonwerra2022trl,
	title        = {{TRL: Transformer Reinforcement Learning}},
	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
	year         = 2020,
	journal      = {GitHub repository},
	publisher    = {GitHub},
	howpublished = {\url{https://github.com/huggingface/trl}}
}

RichardErkhov/argilla_-_Llama-3.2-1B-Instruct-APIGen-FC-v0.1-gguf

Original model description:

Model Card for Llama-3.2-1B-Instruct-APIGen-FC-v0.1

Quick start

Helper functions for the prompt and output parsing

Examples

Parallel function call

Multiple function call

Parallel multiple function call

Multi-turn function call

Irrelevance function call (examples when some data is missing)

Training procedure

Framework versions

Citations

RichardErkhov/
argilla_-_Llama-3.2-1B-Instruct-APIGen-FC-v0.1-gguf

`Parallel` function call

`Multiple` function call

`Parallel multiple` function call

`Multi-turn` function call

`Irrelevance` function call (examples when some data is missing)