There seems to be repetition issues, the quant is probably bad.
Model Card for C4AI Command A
Model Summary
C4AI Command A is an open weights research release of a 111 billion parameter model optimized for demanding enterprises that require fast, secure, and high-quality AI. Compared to other leading proprietary and open-weights models Command A delivers maximum performance with minimum hardware costs, excelling on business-critical agentic and multilingual tasks while being deployable on just two GPUs.
You can try out C4AI Command A before downloading the weights in our hosted Hugging Face Space.
Usage
Please install transformers from the source repository that includes the necessary changes for this model.
py
1# pip install transformers2from transformers import AutoTokenizer, AutoModelForCausalLM
34model_id ="CohereForAI/c4ai-command-a-03-2025"5tokenizer = AutoTokenizer.from_pretrained(model_id)6model = AutoModelForCausalLM.from_pretrained(model_id)78# Format message with the c4ai-command-a-03-2025 chat template9messages =[{"role":"user","content":"Hello, how are you?"}]10input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")1112gen_tokens = model.generate(13 input_ids,14 max_new_tokens=100,15 do_sample=True,16 temperature=0.3,17)1819gen_text = tokenizer.decode(gen_tokens[0])20print(gen_text)
Model Details
Input: Models input text only.
Output: Models generate text only.
Model Architecture: This is an auto-regressive language model that uses an optimized transformer architecture. After pretraining, this model uses supervised fine-tuning (SFT) and preference training to align model behavior to human preferences for helpfulness and safety. The model features three layers with sliding window attention (window size 4096) and RoPE for efficient local context modeling and relative positional encoding. A fourth layer uses global attention without positional embeddings, enabling unrestricted token interactions across the entire sequence.
Languages covered: The model has been trained on 23 languages: English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, Chinese, Russian, Polish, Turkish, Vietnamese, Dutch, Czech, Indonesian, Ukrainian, Romanian, Greek, Hindi, Hebrew, and Persian.
Context Length: Command A supports a context length of 256K.
Chat Capabilities:
By default, Command A is configured as a conversational model. A preamble conditions the model on interactive behaviour, meaning it is expected to reply in a conversational fashion, provides introductory statements and follow-up questions, and uses Markdown as well as LaTeX where appropriate. This is desired for interactive experiences, such as chatbots, where the model engages in dialogue.
In other use cases, a non-interactive model behavior might be more desired (e.g. task-focused use cases like extracting information, summarizing text, translation, and categorization). Learn how system messages can be used to achieve such non-interactive behavior here.
Besides, Command A can be configured with two safety modes, which enable users to set guardrails that are both safe and suitable to their needs: contextual mode, or strict mode. Contextual mode is appropriate for wide-ranging interactions with fewer constraints on output, while maintaining core protections by rejecting harmful or illegal suggestions. Command A is configured to contextual mode by default. Strict mode aims to avoid all sensitive topics, such as violent or sexual acts and profanity. For more information, see the Command A prompt format docs.
RAG Capabilities:
Command A has been trained specifically for tasks like the final step of Retrieval Augmented Generation (RAG).
RAG with Command A is supported through chat templates in Transformers. The model takes a conversation as input (with an optional user-supplied system preamble), along with a list of document snippets.
RAG Example [CLICK TO EXPAND]
py
1# Define conversation input2conversation =[{"role":"user","content":"What has Man always dreamed of?"}]34# Define documents for retrieval-based generation5documents =[6{"heading":"The Moon: Our Age-Old Foe","body":"Man has always dreamed of destroying the moon. In this essay, I shall..."},7{"heading":"Love is all you need","body":"Man's dream has always been to find love. This profound lesson..."},8]910# Get the RAG prompt11input_prompt = tokenizer.apply_chat_template(12 conversation=conversation,13 documents=documents,14 tokenize=False,15 add_generation_prompt=True,16 return_tensors="pt",17)1819# Tokenize the prompt20input_ids = tokenizer.encode_plus(input_prompt, return_tensors="pt")
You can then generate text from this input as normal.
Document snippets should be short chunks, rather than long documents, typically around 100-400 words per chunk, formatted as key-value pairs. The keys should be short descriptive strings, the values can be text or semi-structured.
You may find that simply including relevant documents directly in a user message works just as well, or better than using the documents parameter to render the special RAG template. The RAG template is generally a strong default and is ideal for users wanting citations. We encourage users to play with both, and to evaluate which mode works best for their specific use case.
Note that this was a very brief introduction to RAG - for more information, see the Command A prompt format docs and the Transformers RAG documentation.
RAG with citations [CLICK TO EXPAND]
Optionally, one can ask the model to include grounding spans (citations) in its response to indicate the source of the information. The code is the same as before, except for this line.
py
1# Get the Grounded Generation prompt, with citations2input_prompt = tokenizer.apply_chat_template(3 conversation=conversation,4 documents=documents,5 tokenize=False,6 add_generation_prompt=True,7 return_tensors="pt",8 enable_citations=True,9)1011# There are two answers to this question. Man has dreamed of <co>destroying the moon</co: 0:[0]> and <co>finding love.</co: 0:[1]>
The output looks like this: the model will associate pieces of texts (called "spans") with specific document snippets that support them (called "sources"). Command A uses a pair of tags "<co>" and "</co>" to indicate when a span can be grounded onto a list of sources. For example, "<co>span</co: 0:[0,1]>" means that "span" is supported by documents snippets 0 and 1 that were provided in the last message.
Tool Use Capabilities:
Command A has been specifically trained with conversational tool use capabilities. This allows the model to interact with external tools like APIs, databases, or search engines.
Tool use with Command A is supported through chat templates in Transformers. We recommend providing tool descriptions using JSON schema.
Tool Use Example [CLICK TO EXPAND]
py
1# Define tools2tools =[{3"type":"function",4"function":{5"name":"query_daily_sales_report",6"description":"Connects to a database to retrieve overall sales volumes and sales information for a given day.",7"parameters":{8"type":"object",9"properties":{10"day":{11"description":"Retrieves sales data for this day, formatted as YYYY-MM-DD.",12"type":"string",13}14},15"required":["day"]16},17}18}]1920# Define conversation input21conversation =[{"role":"user","content":"Can you provide a sales summary for 29th September 2023?"}]222324# Get the Tool Use prompt25input_prompt = tokenizer.apply_chat_template(conversation=conversation, tools=tools, tokenize=False, add_generation_prompt=True, return_tensors="pt"))26# Tokenize the prompt27input_ids = tokenizer.encode_plus(input_prompt, return_tensors="pt")
You can then generate from this input as normal.
If the model generates a plan and tool calls, you should add them to the chat history like so:
py
1tool_call ={"name":"query_daily_sales_report","arguments":{"day":"2023-09-29"}}2tool_plan ="I will use the query_daily_sales_report tool to find the sales summary for 29th September 2023."3conversation.append({"role":"assistant","tool_calls":[{"id":"0","type":"function","function": tool_call}],"tool_plan": tool_plan})
and then call the tool and append the result, as a dictionary, with the tool role, like so:
py
1api_response_query_daily_sales_report ={"date":"2023-09-29","summary":"Total Sales Amount: 10000, Total Units Sold: 250"}# this needs to be a dictionary!!23# Append tool results4conversation.append({"role":"tool","tool_call_id":"0","content": api_response_query_daily_sales_report})
After that, you can generate() again to let the model use the tool result in the chat.
Optionally, one can ask the model to include grounding spans (citations) in its response to indicate the source of the information, by using enable_citations=True in tokenizer.apply_chat_template(). The generation would look like this:
On 29th September 2023, the total sales amount was <co>10000</co: 0:[0]> and the total units sold were <co>250.</co: 0:[0]>
When citations are turned on, the model associates pieces of texts (called "spans") with those specific tool results that support them (called "sources"). Command A uses a pair of tags "<co>" and "</co>" to indicate when a span can be grounded onto a list of sources, listing them out in the closing tag. For example, "<co>span</co: 0:[1,2],1:[0]>" means that "span" is supported by result 1 and 2 from "tool_call_id=0" as well as result 0 from "tool_call_id=1". Sources from the same tool call are grouped together and listed as "{tool_call_id}:[{list of result indices}]", before they are joined together by ",".
Code Capabilities:
Command A has meaningfully improved on code capabilities. In addition to academic code benchmarks, we have evaluated it on enterprise-relevant scenarios, including SQL generation and code translation, where it outperforms other models of similar size. Try these out by requesting code snippets, code explanations, or code rewrites. For better performance, we also recommend using a low temperature (and even greedy decoding) for code-generation related instructions.
Model Card Contact
For errors or additional questions about details in this model card, contact info@for.ai.
Terms of Use:
We hope that the release of this model will make community-based research efforts more accessible, by releasing the weights of a highly performant 111 billion parameter model to researchers all over the world. This model is governed by a CC-BY-NC License (Non-Commercial) with an acceptable use addendum, and also requires adhering to C4AI's Acceptable Use PolicyIf you are interested in commercial use, please contact Cohere’s Sales team.
Try Chat:
You can try Command A chat in the playground here. You can also use it in our dedicated Hugging Face Space here.
