Benchmark
Siftdog vs Tavily: search benchmark
- Facts: on par. Both found 25 of 25 answers in advanced mode.
- Official docs: Siftdog found them more often, 9 of 10 vs Tavily's 7 (basic) and 3 (advanced).
- Speed: similar. Tavily was faster in basic mode, Siftdog in advanced.
- Cost: $0 per search, vs $8–16 per 1,000 on Tavily.
Results
| Siftdog basic | Tavily basic | Siftdog advanced | Tavily advanced | |
|---|---|---|---|---|
| Factual answer found (of 25) | 24 | 25 | 25 | 25 |
| Official docs returned (of 10) | 9 | 7 | 9 | 3 |
| Official docs ranked first (of 10) | 3 | 1 | 7 | 1 |
| Median latency | 1.61s | 1.25s | 2.19s | 2.81s |
| 95th percentile latency | 3.03s | 3.09s | 4.55s | 5.02s |
| Errors (of 45) | 0 | 0 | 0 | 0 |
| Cost per 1,000 searches | $0 | $8 | $0 | $16 |
Highlighted: better within each mode. Siftdog's cost excludes its server. Tavily's is pay-as-you-go ($0.008 per credit).
Method
- Both called through the official
tavily-pythonclient with identical arguments; only the base URL differed. - 45 queries: 25 factual questions with a known answer, 10 technical questions with an official docs site, 5 news, 5 open-ended. 5 results each, basic and advanced.
Caveats
- Some of Siftdog's search engines (Google, Google CSE, Brave) were rate-limiting the test machine, so these numbers aren't Siftdog at full strength.
- Small sample: a difference of one or two queries isn't significant.
- Siftdog ran locally; Tavily over the internet.
- A single run on 25 September 2026. Results change over time.
Reproduce it
pip install -r bench/requirements.txt
docker compose up -d
TAVILY_API_KEY=tvly-... python bench/run.py --delay 6
python bench/report.py bench/results/<timestamp>Code and queries: bench/. A full run uses about 135 Tavily credits. Not affiliated with Tavily; Tavily is a trademark of its owner.