Blog
Guides and results
Guides and results for Siftdog, the open-source, self-hosted Tavily alternative.
Generate Answers from Search Results with Siftdog include_answer
Configure Siftdog to write search answers with Claude, or skip answer generation and send search results to your own LLM.
Recent news search for LLM apps with Siftdog
Use Siftdog’s news topic and time range to find recent coverage for an LLM app, and understand what upstream search engines can affect.
Restrict AI Agent Web Search to Trusted Domains
Use Siftdog’s include_domains and exclude_domains search filters to narrow coding-agent results to documentation or keep selected sites out.
Tavily search_depth Basic vs Advanced, Explained
Understand what basic and advanced search_depth do in Siftdog, how chunks_per_source fits in, and when the extra page-fetching time may be worthwhile.
Migrating from Tavily to a Self-Hosted API: What Changes
Point Tavily-compatible code at Siftdog, then check ranking differences, ignored parameters, and endpoints you’ll need to replace.
Add web search to Claude Desktop with a self-hosted MCP server
Connect Claude Desktop to Siftdog over stdio for web search, extraction, crawling, and site maps. Search needs SearXNG; the other tools do not.
Crawl a documentation site into Markdown with Siftdog
Map a documentation site, then crawl selected paths into Markdown with Siftdog’s depth and page limits.
Turn web pages into clean text for RAG with Siftdog
Extract clean Markdown or text from known URLs with Siftdog, then send it to your own chunking, embedding, and retrieval pipeline.
Firecrawl vs Tavily: scraping, search, pricing, and self-hosting
Compare Firecrawl and Tavily by what they do, how their API credits work, and when a self-hosted Tavily-compatible option like Siftdog makes sense.
Tavily API pricing vs self-hosting: when does Siftdog cost less?
Compare Tavily’s per-credit search pricing with the infrastructure and operations cost of running Siftdog yourself.
SearXNG JSON API: Enable JSON and Handle Rate Limits
Enable SearXNG's JSON output, test the API, and troubleshoot upstream rate limits when using it with Siftdog.
Add web search to Ollama with self-hosted Siftdog
Give a local Ollama model web search through Siftdog, with no third-party search or LLM API key.
Add self-hosted web search to Claude Code with MCP
Give Claude Code web search, page extraction and crawling from your own machine with Siftdog's MCP server. One command, no search API key.
Add web search to Cursor with an MCP server
Connect Cursor to Siftdog for self-hosted web search, page extraction and crawling. Set it up locally or connect to a running server.
How to self-host a Tavily alternative
Run your own Tavily-compatible web search API with Siftdog and Docker Compose: search, extract, crawl and map, no search API key and no per-search fees.
Siftdog vs Firecrawl vs crw: self-hosted web search for AI agents
An honest comparison of three self-hostable web search and crawling APIs for AI agents: licenses, APIs, JavaScript rendering, MCP support and when to pick each.
Use Siftdog with LangChain and the Tavily SDK
Keep your Tavily code and point it at a self-hosted Siftdog server: tavily-python (sync and async) and langchain-tavily, with one extra argument.