SearchClaw - An AI deep research intelligent agent launched by Renmin University of China

What is SearchClaw?

SearchClaw is a self-hosted AI deep research agent launched by RUC-NLPIR (Information Retrieval Laboratory, Renmin University of China), equipped with a web-based interactive interface. After a user submits a question, the system automatically performs multiple rounds of web searches, page crawling, paper retrieval, and content synthesis, ultimately generating a detailed report with citation links. Built on FastAPI, the project ensures research quality through a "tools + hooks" architecture, supports various LLM providers and persistent memory, and is suitable for researchers and analysts to conduct traceable, automated research.

SearchClaw's main functions

• Autonomous multi-round research cycleIt automatically iterates through searching, fetching, reading, and referencing until the answer meets the criteria.
• Multi-source information retrievalIntegrated web search (Google/Serper), academic search (Semantic Scholar/DBLP/arXiv), news search (NewsAPI/Google News RSS), and WeChat public account article search.
• Deep browser integrationRender JavaScript pages via Playwright/CDP and support crawling in login state.
• Intelligent follow-up questions and clarificationDuring the research process, follow-up questions can be raised with users to clarify their needs.
• Research Plan BreakdownIt automatically breaks down complex queries into traceable subtasks and executes them step by step.
• Quality access control systemThe built-in hook checks the number of citations, source diversity, and answer completeness; if these criteria are not met, the investigation continues.
• Context compression managementThe two-stage compression mechanism ensures that long sessions do not exceed the context window limit.
• Cross-session persistent memoryAutomatically saves and reuses high-quality sources, user preferences, and key facts from past research.

How to use SearchClaw

• Clone the repository and install dependencies:git clone https://github.com/RUC-NLPIR/SearchClaw.git && pip install -e .
• Configure API keys (LLM and search source):set up ANTHROPIC_API_KEY or OPENAI_API_KEYRecommended configuration SERPER_API_KEY and JINA_API_KEY Improve search quality.
• Start service:python -m src.mainBrowser access http://localhost:8000.
• Return resultsInput your research question, and the system will return a comprehensive report with references in real time via WebSocket.

Key information and usage requirements of SearchClaw

• Environmental requirementsPython 3.11+, supports Linux/macOS/Windows.
• LLM supportIt supports mainstream models such as Anthropic, OpenAI, Google Gemini, xAI, Alibaba Tongyi, ByteDance, Zhipu GLM, and Lunar Dark Side, as well as local vLLM/Ollama endpoints via litellm routing.
• Search dependencyAutomatically downgrades to DuckDuckGo fetching when no Serper is available, and falls back to direct HTTP requests when no Jina is available.
• Optional browser:implement pip install -e '.[browser]' && playwright install chromium Enable JS rendering support.
• Security ConfigurationRemote deployment via SEARCH_CLAW_API_KEY Set an access password to prevent unauthorized use.
• Persistent storageMemory and session data are stored in JSON format on the local file system by default.

SearchClaw's core advantages

• Verifiable citation quality: Mandating multi-source citations and passing access control checks reduces the risk of hallucinations and is superior to the fragmented information splicing of single-round RAGs.
• Autonomous planning and error correctionIt automatically breaks down complex tasks and self-sustains research when quality is substandard, without requiring manual intervention in the iterative process.
• Multi-source heterogeneous integrationIt is the only open-source research tool that simultaneously covers English academic repositories, Chinese WeChat public accounts, and real-time news, making it suitable for cross-border research scenarios.
• Fully self-hostedData remains on-premises and supports local LLM endpoints, meeting privacy compliance requirements for sensitive research data.

SearchClaw's project address

• GitHub repositoryhttps://github.com/RUC-NLPIR/SearchClaw

SearchClaw's Competitive Product Comparison

Application Scenarios of SearchClaw

• Literature ReviewThe tool can automatically retrieve papers from Semantic Scholar and arXiv and generate cited summaries of advancements in the field.
• Market competitor monitoringAutomatically tracks news sources and WeChat official accounts, analyzes competitor activities, and outputs timestamped reference data.
• Policy and Regulation ResearchThe tool can crawl government websites and think tank reports, and integrate multilingual sources to form a compliance analysis report.
• Technology selection evaluationIt allows you to search technical blogs, GitHub discussions, and academic papers to compare community evaluations and performance data of different frameworks.