Alibaba’s ZeroSearch Magic: Zero API Costs, Maximum Insight

What if your LLM could search like Google?

Most large language models (LLMs) struggle to answer questions about recent events or obscure facts. Traditional fixes like retrieval-augmented generation (RAG) rely on external search APIs, which rack up costs and introduce new points of failure. Enter ZeroSearch, Alibaba’s internal search simulation framework that eliminates API dependencies while boosting answer accuracy.

ZeroSearch trains LLMs to simulate a search engine internally. By generating both relevant and noisy documents during training, the model learns to retrieve and synthesize grounded answers using only its pretraining memory.

In this post, we’ll explore:

• Why traditional RAG and RL-based search incur heavy costs

• How ZeroSearch works under the hood

• What kind of performance gains it offers

• Limitations and future directions for internal tool simulation

Let's get started.

Before ZeroSearch: RAG, RL, and rising API costs#

LLMs only know what they learned up to their training cutoff date. This means that if you ask about anything that happened after that cutoff date, they can fill in gaps with made-up (hallucinated) or outdated information, which undermines trust in real-world use.

Retrieval-augmented generation (RAG) tackles this by letting models use external knowledge to ground their answers.

• Early RAG work relied on carefully crafted prompts to break questions into queries, decompose complex requests, and stitch together information across multiple steps. That design requires relentless prompt tuning and strong reasoning by the model.

• Later methods used supervised fine tuning (SFT) to strengthen smaller models and even applied Monte Carlo tree search (MCTS)Monte Carlo Tree Search is an algorithm that builds a decision tree through iterative random simulations and selects actions based on their estimated outcomes. at inference to explore more answer paths. Those techniques boosted accuracy but came with heavy computational overhead.

Reinforcement learning (RL) has also emerged as a way to teach models when and how to fetch information based purely on reward signals. Some approaches hook into live search engines during training to mimic real web queries. In that setup, the quality of returned documents can be wildly unpredictable, and the cost of making hundreds of thousands of API calls quickly becomes a financial roadblock that limits scale.

How does ZeroSearch work?#

ZeroSearchhttps://github.com/Alibaba-NLP/ZeroSearch turns an LLM’s built-in knowledge into its search tool, no external engines required. It starts with a lightweight supervised fine-tuning step that teaches the model (separate from the reference model) to generate spot-on and deliberately noisy document snippets simply by tweaking the prompt.

A curriculum rolloutA phased training method that gradually injects increasing amounts of noise into simulated search snippets to sharpen an LLM’s retrieval and reasoning skills without using real APIs. then gradually ramps the noise level, so the policy model (the main LLM) masters clear, simple retrievals before tackling tougher, messier scenarios. Everything runs on GPU clusters, and throwing more GPUs at the problem directly speeds up document generation, keeping API bills at zero while scaling efficiently. 

Let’s break down the steps:

1. Search engine#

A separate LLM is fine-tuned to act as a search engine simulator. First, a training dataset is generated using a three-step template—think, search, answer—where the model produces candidate documents labeled positive or negative based on retrieval accuracy. After collecting a large set of these query–document examples, including both relevant and noisy results, the LLM is fine-tuned to enhance its ability to generate realistic search-style outputs. Here is the training template used during supervised fine-tuning and inference:

Fine-tuning for realistic search results#

The model is given two slightly different prompts, one to produce relevant documents and one to generate noisy documents. It can produce each type on demand by tweaking just a few words. Each prompt also embeds example input–output pairs to strengthen the LLM’s retrieval knowledge. Here is the template for the search simulation:

By the end of this stage, the LLM is ready to generate good and noisy search results and be used by the main LLM.

2. Curriculum rollout#

With the simulation LLM calibrated to produce both clean and noisy results on demand, the policy model is ready to learn how to use those documents effectively. The main LLM is trained via reinforcement learning. Early in training, it receives high-quality documents generated by the simulation LLM. A curriculum rollout then gradually increases the probability of noisy documents according to:

$\pi_i = p_s + \frac{b^{i/m} - 1}{b - 1}\,(p_e - p_s)$

Where $p_s$ and $p_e$ denote the initial and final noise probabilities, $i$ is the current training step, $m$ is the total number of steps, and $b$ is the exponential base with 4 as the default value. The policy model is rewarded based on the F1 score between its predicted answer and the ground truth. While F1 is a simple harmonic mean of precision and recall, its role here is critical as it provides a scalar reward that guides the reinforcement learning process. The reward helps the model learn to retrieve and synthesize relevant documents internally, maximizing answer accuracy over time.

Sample of relevant and noisy document outputs#

Below is a sample of documents generated by the simulation LLM in response to the query. The table contrasts the useful outputs, which contain relevant information about Óskar Jónasson’s birth date, with the noisy outputs, which are off-topic or irrelevant.

Query details

Retrieved documents

ZeroSearch vs. Baselines: Benchmark results#

Across seven diverse QA benchmarks, covering in-domain tasks such as NQ and HotpotQA and out-of-domain challenges like TriviaQA and Bamboogle, ZeroSearch consistently outperforms every baseline it was tested against, demonstrating its robustness across varied scenarios.

The following table compares ZeroSearch (base and instruction-tuned) with several baseline methods across seven QA datasets for Qwen-2.5-7B-Base/Instruct, showing that ZeroSearch (instruction-tuned) achieves the highest average score of 40.54 and leads on five out of seven tasks.

ZeroSearch still pulls ahead, underscoring its promise as a cost-free alternative for live retrieval in large-scale reinforcement learning.

ZeroSearch’s advantages apply across different model families and sizes. It delivers strong results for both base and instruction-tuned variants and shows further improvements as model size increases, illustrating its broad applicability and scalability. Performance of simulated search engines using different LLM configurations is shown in the table below:

From the table, it’s clear that SFT-7B matches Google Search’s performance, and SFT-14B even exceeds it, proving that a well-tuned LLM can stand in for a live engine.

Bigger simulation LLMs only improve, offering more accurate retrievals and stronger curriculum learning as size increases.

The moment of truth: GPU vs. API costs#

ZeroSearch swaps commercial API fees for GPU infrastructure costs by running simulation LLMs on GPU servers. Using Qwen-2.5-7B with a batch size of 64, five rollout repetitions, and 200 training steps (approximately 12 hours and 64 000 search requests), the table below shows that:

• One A100 GPU for SFT-3B costs $17.7

• Two GPUs for SFT-7B cost $35.4

• Four GPUs for SFT-14B cost $70.8

...Compared with $586.7 in Google Search API charges

However, GPU costs are far lower than API costs, utilization peaks during rollouts and drops during policy updates. Sharing a simulation server across multiple RL jobs can reduce idle time and cut costs. Offering simulation LLMs of various sizes lets teams balance performance and resource use to fit their needs.

Limits of simulated search#

ZeroSearch depends entirely on the simulation LLM’s pretraining memory. Any request for information introduced after the model’s cutoff date or for highly specialized topics not seen in its training corpus will exceed its capabilities. For example, asking “Who won the 2025 Booker Prize?” may produce a plausible but incorrect answer or revert to previous winners instead of the actual recipient. Typical failure scenarios include breaking news events, ultra-niche technical findings, real-time financial data, and live sports scores. When up-to-the-minute accuracy or truly novel facts are required, a live search engine or external data source remains necessary.

Isn’t the knowledge cutoff still the ultimate limit, even with simulated search?

Large language models only know what they saw during pretraining, leaving them prone to hallucinations or stale facts when asked about anything beyond their cutoff date. ZeroSearch taps into that frozen knowledge more effectively by training the model to retrieve from its memory, surfacing relevant passages to ground its answers. This internal retrieval sharpens reasoning, reduces made-up content, and delivers Google-grade performance without live lookup fees. Even though it cannot fetch new information, ZeroSearch makes the most of what the model already knows, boosting reliability and accuracy across various tasks.

What lies beyond simulating search engines?#

Simulating a search engine is only the beginning. This approach can internalize services such as code execution, translation, summarization, and structured data extraction.

For example:

• An LLM fine-tuned as an internal Python interpreter could debug and optimize scripts.

• Another could act as a multilingual translation engine, converting text in dozens of languages offline.

• A summarization module could condense lengthy meeting transcripts into concise briefs.

• Knowledge graph generators could map complex relationships in legal or medical documents.

Chaining these simulated tools, the model could orchestrate workflows from data analysis to personalized tutoring without external dependencies or unpredictable costs.

A future in simulated search#

ZeroSearch transforms a model’s frozen knowledge into a dynamic in-house retrieval engine, cutting API costs and boosting answer quality. It delivers robust, scalable performance across diverse tasks by blending supervised fine-tuning, curriculum rollout, and reinforcement learning. ZeroSearch counts every byte, empowering LLMs to act like self-sufficient research assistants even with a static knowledge base.

Simulated search is just one part of the fine-tuning frontier. If you're ready to train models that reason better, retrieve smarter, and adapt faster, check out our hands-on course below.