#data-extraction Repositories
131 repositories total
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Open-source API interface server for crawling platforms like Douyin (a_bogus latest version), Kuaishou, Bilibili, Xiaohongshu, Taobao, JD.com, Weibo, etc. One-click quick deployment with Docker.
Heapdump sensitive information query tool, for example, for finding plaintext passwords, AK, SK, etc. in spring heapdumps
Effortlessly extract data with our AI scraper API. Simplify data extraction, get clean JSON outputs and adapt to page changes. Try it free today!
A beginner-friendly yet powerful Python toolkit for financial analysis and automation โ built to make modern investing accessible to everyone
Flyscrape is a command-line web scraping tool designed for those without advanced programming skills.
Dianping Crawler (crawlable on the whole site, solves dynamic font encryption, not OCR). Continuous update
๐ฅ AI-powered data enrichment tool that transforms emails into rich datasets with company profiles, funding data, tech stacks, and more using Firecrawl and multi-agent AI
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
The fastest PDF library for Python and Rust. Text extraction, image extraction, markdown conversion, PDF creation & editing. 0.8ms mean, 5ร faster than industry leaders, 100% pass rate on 3,830 PDFs. MIT/Apache-2.0.
โ๏ธ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites
A Rust library to extract useful data from HTML documents, suitable for web scraping.
Ranking Google Scholar search results based on the number of citations
Used to export vocabulary databases from MoMo Vokabeltrainer and generate custom vocabulary databases suitable for List Vokabeltrainer, BuBei Vokabeltrainer, Oulu Dictionary, etc.
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
A Python 3 compatible version of goose http://goose3.readthedocs.io/en/latest/index.html
2019CCF-BDCI Competition Winner of the Best Innovation Exploration Award Champion of the OCR-Based ID Card Element Extraction Challenge Tianchen Poxiao Team Challenge Source Code
Access your meal tracking data stored in MyFitnessPal programatically
Data management framework for Python that provides functionality to describe, extract, validate, and transform tabular data
A chatbot/GraphRAG framework that creates multi-llm-agents from social platform user comments and let them debate on specific topics.
A browser extension: mainly used for taking notes in WeChat Read, it is quite helpful for readers who frequently use Markdown for notes.