DeepDoc - an open-source deep research tool focused on local resource research.
DeepDoc is an open-source deep research tool focused on in-depth research of local knowledge bases. The tool uses a research-based workflow to extract text from local resources (such as PDF, DOCX, JPG, TXT, etc.), segment the data stored in...
What is DeepDoc?
DeepDoc is an open-source deep research tool focused on in-depth research of local knowledge bases. Through a research-based workflow, it extracts text from local resources (such as PDFs, DOCXs, JPGs, TXTs, etc.), segments and stores it in a vector database for semantic similarity searches. Users can query and generate content structures based on instructions, providing feedback to optimize the structure. DeepDoc generates clear Markdown reports. The tool is suitable for scenarios requiring rapid extraction of insights from local files without manually browsing large amounts of data.
DeepDoc's main functions
- Local resource researchSupports multiple local file formats (PDF, DOCX, JPG, TXT, etc.), extracts and segments text content for easier subsequent processing.
- Semantic similarity searchBy embedding text blocks into a vector database, efficient semantic similarity search can be achieved, quickly locating relevant content.
- Research-based workflowIt generates content structure based on user instructions, supports feedback optimization, and improves the accuracy of research.
- Multi-step research processThrough steps such as knowledge generation, query creation, and search optimization, high-quality report content is gradually generated.
- Structured report generationOutputs clear Markdown format reports for easy viewing and use by users.
DeepDoc's technical principles
- Text extraction and segmentationThis technique uses Optical Character Recognition (OCR) to extract text from image files (such as JPGs). The extracted text is then segmented into page-level blocks for easier subsequent processing.
- Vector database storageThe segmented text blocks are embedded into a vector space and stored in a vector database (such as Qdrant). This enables the system to perform semantic similarity searches efficiently and quickly find the text blocks most relevant to the user's query.
- Multi-step research processFor each report section, the research agent generates knowledge and creates research queries. The search agent runs on local data, finding the most relevant text blocks to the queries. The reflex agent optimizes the search results, ensuring the generated content is accurate and useful. Finally, the content of each section is compiled into a complete report.
DeepDoc's project address
- GitHub repositoryhttps://github.com/Datalore-ai/deepdoc
Application scenarios of DeepDoc
- academic researchResearchers can quickly sort through and analyze large amounts of literature to generate structured research reports, saving time spent manually organizing literature.
- Enterprise knowledge managementThe system allows companies to deeply mine massive amounts of internal documents, reports, and project materials to quickly extract key information and support decision-making.
- Legal document analysisLegal professionals conduct in-depth analysis of a large number of legal documents, cases, contracts, etc., to quickly locate relevant clauses and cases, thereby improving work efficiency.
- Market ResearchMarket researchers conduct in-depth analysis of collected market research reports, consumer feedback, competitor information, etc., and quickly generate market research reports.
- Personal knowledge managementIndividual users can deeply organize and analyze their personal notes, learning materials, project documents, etc., quickly extract key information, and improve learning and work efficiency.