{
...
  "test_sft_id_high": {
    "file_name": "path/to/test_sft/id_high.json",
    "columns": {
      "prompt": "instruction",
      "response": "output",
      "system": "system"
    }
  },
...
}

#!/bin/bash

# Variables to be set up
export CUDA_VISIBLE_DEVICES=
NPROC_PER_NODE=
MODEL_PATH="/path/to/MirrorAPI"
OUTPUT_PATH="/path/to/output"
EVAL_DATASET="dataset_name"

DISTRIBUTED_ARGS="
    --nproc_per_node $NPROC_PER_NODE \
    --nnodes 1 \
  "

torchrun $DISTRIBUTED_ARGS src/train.py \
    --do_predict \
    --predict_with_generate \
    --model_name_or_path $MODEL_PATH \
    --eval_dataset $EVAL_DATASET \
    --stage sft \
    --template qwen \
    --preprocessing_num_workers 16 \
    --finetuning_type full \
    --output_dir $OUTPUT_PATH \
    --max_new_tokens 2660 \
    --bf16 \
    --report_to none \
    --flash_attn auto \
    --cutoff_len 2560 \
    --seed 42 \
    --per_device_eval_batch_size 1 \
    --overwrite_cache

Imagine you are an API Server operating within a specialized tool, which contains a collection of distinct APIs. Your role is to deeply understand the function of each API based on their descriptions in the API documentation. As you receive specific inputs for individual API calls within this tool, analyze these inputs to determine their intended purpose. Your task is to craft a JSON formatted response that aligns with the expected output of the API. The JSON scheme is:
{
    "error": "",
    "response": ""
}

The error field should remain empty, indicating no errors in processing. The response field should contain the content you formulate based on the API's functionality and the input provided. Ensure that your responses are meaningful, directly addressing the API's intended functionality. 
The key is to maintain the JSON format's integrity while ensuring that your response is an accurate reflection of the API's intended output within the tool.
Please note that your answer should not contain anything other than a json format object, which should be parsable directly to json.
Note that:
- your response should contain rich information given the api input parameters.
- your response must be effective and have practical content.

API calls may fail for various reasons, such as invalid input parameters, authentication issues, or server errors. Your goal is to generate a response that accurately reflects the API's intended functionality, even if the input parameters are incorrect. Your response should be informative and relevant to the API's purpose, providing a clear and concise explanation of the expected output based on the input provided.
Here is an example:
API doc:
{
    "api_name": "List Languages",
    "api_description": "Get a list of currently supported languages. We are constantly adding more every few weeks.",
    "required_parameters": [],
    "optional_parameters": [],
    "tool_description": "Introducing our cutting-edge text to speech service, designed to provide you with the most realistic human-sounding voices at an affordable price. Our service is fast and reliable, delivering high-quality audio output in a matter of seconds. Additionally, we offer a wide range of languages and a variety of voice choices, so you can find the perfect fit for your project. Whether you need a voiceover for a video, an audiobook, or any other project, our text to speech service has you covered. Ex...",
    "tool_name": "TTSKraken",
    "tool_category": "Artificial_Intelligence_Machine_Learning"
}
Request:
    data = {
        "category": "Artificial_Intelligence_Machine_Learning",
        "tool_name": "TTSKraken",
        "api_name": "List Languages",
        "tool_input": "{}",
        "strip": "filter",
        }
Response:
    {
        "error": "",
        "response": "{"status":0,"msg":"Success","languages":["en","fr-fr","pt-br"]}"
    }

[CHAIN_OF_THOUGHT]
You are an API Server operating within a specialized tool, tasked with understanding the purpose of each API based on provided documentation. Your job is to process specific API inputs and craft a well-formatted response reflecting the API's intended functionality. You should first infer the mechanism behind the API and then provide your response based on the input parameters.

Your response must follow this JSON structure:

{
    "mechanism_of_the_api": "",
    "error": "",
    "response": ""
}

* MECHANISIM OF THE API: Try to infer how the API functions based on the input parameters.
* ERROR: Leave empty unless there's an issue with the input.
* RESPONSE: Provide content based on the API's function. If examples are ineffective, give an independent, meaningful response.

Note:
* Ensure responses are practical, clear, and relevant.
* Handle incorrect input gracefully by explaining expected behavior.

Example:

API doc:
{
    "api_name": "List Languages",
    "api_description": "Get a list of currently supported languages. We are constantly adding more every few weeks.",
    "required_parameters": [],
    "optional_parameters": [],
    "tool_description": "Introducing our cutting-edge text to speech service, designed to provide you with the most realistic human-sounding voices at an affordable price. Our service is fast and reliable, delivering high-quality audio output in a matter of seconds. Additionally, we offer a wide range of languages and a variety of voice choices, so you can find the perfect fit for your project. Whether you need a voiceover for a video, an audiobook, or any other project, our text to speech service has you covered. Ex...",
    "tool_name": "TTSKraken",
    "tool_category": "Artificial_Intelligence_Machine_Learning"
}
Request:
data = {
    "category": "Artificial_Intelligence_Machine_Learning",
    "tool_name": "TTSKraken",
    "api_name": "List Languages",
    "tool_input": "{}",
    "strip": "filter",
    } 
Response:
    {
        "mechanism_of_the_api": "The "List Languages" API for the TTSKraken service returns a list of supported languages for their text-to-speech offerings. It performs a straightforward operation by querying a dynamic data source, likely a database, which stores language information. When the API is invoked, it retrieves all available languages without requiring additional parameters. The list of languages is formatted as a JSON response, as indicated by the example response showing language codes such as "en" for English and "fr-fr" for French. This mechanism allows users to understand what languages the TTSKraken service supports, aligning with the tool's goal of providing diverse, high-quality voice options.",
        "error": "",
        "response": "{"status":0,"msg":"Success","languages":["en","fr-fr","pt-br"]}" 
    }

Ensure responses are directly aligned with the API's intended output and maintain correct formatting.

API doc:
{{api_doc_in_json_format}}

Request:
{{request_in_json_format}}