FunctionGemma-270M-IT RAG
This is a fine-tuned derivative of google/functiongemma-270m-it, optimized for lightweight Retrieval-Augmented Generation (RAG) on mobile / edge / low-power devices. The fine-tune specializes the model to consistently emit a tool call to vector_search—with a well-formed, high-recall search query—when the user asks a natural-language question that should be answered from a document store.
It’s intended to be used as the “retrieval controller” in a local-first RAG pipeline:
User question → model generates vector_search(query=…) → system retrieves passages → (optional) downstream answer model composes final response.
Base model
-
Base: google/functiongemma-270m-it (Gemma 3 270M family), a small model tuned specifically for function calling. (
Google AI for Developers)
-
Interface & formatting: Uses FunctionGemma’s special control tokens for tool use (e.g.,
<start_function_call>…<end_function_call>) and the
<escape> delimiter for string fields. (
Google AI for Developers)
-
Context length (base): 32K total input context (and up to 32K output context per request, budget permitting). (
Hugging Face)
What’s new in this fine-tune
Primary behavioral change: When asked questions in natural language, the model reliably chooses to call:
-
vector_search
-
with a single string argument: a retrieval query designed to maximize recall and relevance for downstream passage ranking.
Example behavior (from your eval set):
- Prompt: “Can you compare the political systems of the Roman Republic and the Aztec Empire… succession and social mobility?”
Output: <start_function_call>call:vector_search{query:<escape>Roman Republic vs Aztec Empire political systems succession social mobility ...<escape>}<end_function_call> ✅
(Additional examples include VAR vs VAR review, journalism ethics across platforms, intrinsic vs extrinsic motivation, bench vs jury trial, Rodin image sources.)
Intended use
Designed for:
-
On-device or constrained deployments (mobile apps, embedded, low-cost CPU boxes) that need
fast, local routing to retrieval. FunctionGemma is explicitly positioned as a lightweight base for local-first agents and edge workflows. (
Google AI for Developers)
-
RAG systems where the most important skill is producing the right search query, not writing the final answer.
Not designed for:
-
Being the sole “answer model” for complex, high-stakes, or deeply reasoned tasks (it’s small; use it to retrieve, then answer with a stronger model if needed).
-
Multi-step tool plans out of the box (FunctionGemma’s training is strongest for single-turn / parallel calls; multi-step chaining isn’t its primary trained workflow). (
Google AI for Developers)
Tool contract
This fine-tune assumes a tool with the following conceptual signature:
-
Tool name: vector_search
-
Arguments:
query (string): a search query describing the user’s information need
-
Returns: passages/snippets (top-k) with metadata (titles/urls/ids), which are then fed into a downstream step.
Important formatting note: String values in tool blocks must be wrapped in
<escape>…<escape> to avoid parsing ambiguity. (
Google AI for Developers)
How to use (recommended pattern)
-
Run the model on the user question.
-
If the output contains a vector_search call, execute retrieval.
-
Feed retrieved passages to:
-
either the same model (if you accept lower-quality synthesis), or
-
a larger model for final answer generation.
If you are using the Hugging Face tooling, FunctionGemma models are typically used via chat templates that support tool definitions and function-call decoding. (
Hugging Face)
