Git Stars Git Stars

#data-extraction Repositories

131
Total Repositories
729,329
Total Stars
5,567
Avg Stars/Repo
51
hi-primus/optimus

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

hi-primus
โญ 1,539 Python ๐Ÿ‘ค hi-primus Organization ๐Ÿ• Updated 2024-12-02 ๐Ÿ“… Created 2017-07-13
52
chinoogawa/fbht

Facebook Hacking Tool

chinoogawa
โญ 1,510 Python ๐Ÿ‘ค chinoogawa User ๐Ÿ• Updated 2024-05-09 ๐Ÿ“… Created 2013-12-16
53
ShilongLee/Crawler

Open-source API interface server for crawling platforms like Douyin (a_bogus latest version), Kuaishou, Bilibili, Xiaohongshu, Taobao, JD.com, Weibo, etc. One-click quick deployment with Docker.

ShilongLee
โญ 1,490 JavaScript ๐Ÿ‘ค ShilongLee User ๐Ÿ• Updated 2026-10-19 ๐Ÿ“… Created 2024-05-24
54
HolographicHat/Yae

Faster, more accurate Genshin Impact data export tool

HolographicHat
โญ 1,471 C# ๐Ÿ‘ค HolographicHat User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2022-03-21
55
Miserlou/SoundScrape

SoundCloud (and Bandcamp and Mixcloud) downloader in Python.

Miserlou
โญ 1,436 Python ๐Ÿ‘ค Miserlou User ๐Ÿ• Updated 2024-06-23 ๐Ÿ“… Created 2013-12-29
56
wyzxxz/heapdump_tool

Heapdump sensitive information query tool, for example, for finding plaintext passwords, AK, SK, etc. in spring heapdumps

wyzxxz
โญ 1,430 ๐Ÿ‘ค wyzxxz User ๐Ÿ• Updated 2024-05-21 ๐Ÿ“… Created 2021-08-06
57
ScrapingBee/ai-web-scraper

Effortlessly extract data with our AI scraper API. Simplify data extraction, get clean JSON outputs and adapt to page changes. Try it free today!

ScrapingBee
โญ 1,424 ๐Ÿ‘ค ScrapingBee Organization ๐Ÿ• Updated 2026-03-13 ๐Ÿ“… Created 2026-03-12
58
thinh-vu/vnstock

A beginner-friendly yet powerful Python toolkit for financial analysis and automation โ€” built to make modern investing accessible to everyone

thinh-vu
โญ 1,422 ๐Ÿ‘ค thinh-vu User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2022-02-27
59
facundoolano/app-store-scraper

scrape data from the itunes app store

facundoolano
โญ 1,396 JavaScript ๐Ÿ‘ค facundoolano User ๐Ÿ• Updated 2025-07-27 ๐Ÿ“… Created 2016-05-31
60
raznem/parsera

Lightweight library for scraping web-sites with LLMs

raznem
โญ 1,372 Python ๐Ÿ‘ค raznem User ๐Ÿ• Updated 2025-12-17 ๐Ÿ“… Created 2024-08-12
61
philippta/flyscrape

Flyscrape is a command-line web scraping tool designed for those without advanced programming skills.

philippta
โญ 1,347 Go ๐Ÿ‘ค philippta User ๐Ÿ• Updated 2025-11-24 ๐Ÿ“… Created 2023-08-28
62
mvdbos/php-spider

A configurable and extensible PHP web spider

mvdbos
โญ 1,338 PHP ๐Ÿ‘ค mvdbos User ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2013-02-25
63
Sniper970119/dianping_spider

Dianping Crawler (crawlable on the whole site, solves dynamic font encryption, not OCR). Continuous update

Sniper970119
โญ 1,329 Python ๐Ÿ‘ค Sniper970119 User ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2021-01-22
64
jbarrow/commonforms

CommonForms โ€” open models to auto-detect PDF form fields

jbarrow
โญ 1,309 Python ๐Ÿ‘ค jbarrow User ๐Ÿ• Updated 2026-06-17 ๐Ÿ“… Created 2025-09-17
65
firecrawl/fire-enrich

๐Ÿ”ฅ AI-powered data enrichment tool that transforms emails into rich datasets with company profiles, funding data, tech stacks, and more using Firecrawl and multi-agent AI

firecrawl
โญ 1,308 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2025-06-06
66
fastcrw/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

fastcrw
โญ 1,284 Rust ๐Ÿ‘ค fastcrw Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-03-02
67
raawaa/jav-scrapy

The hard laborer for batch fetching AV magnet links or covers

raawaa
โญ 1,275 JavaScript ๐Ÿ‘ค raawaa User ๐Ÿ• Updated 2026-10-17 ๐Ÿ“… Created 2015-10-31
68
google/textfsm

Python module for parsing semi-structured text into python tables.

google
โญ 1,258 Python ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2015-06-23
69
KillzXGaming/Switch-Toolbox

A tool to edit many video game file formats

KillzXGaming
โญ 1,249 C# ๐Ÿ‘ค KillzXGaming User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2018-11-12
โญ 1,228 ๐Ÿ‘ค Sak32009 User ๐Ÿ• Updated 2026-08-30 ๐Ÿ“… Created 2016-06-15
71
yfedoseev/pdf_oxide

The fastest PDF library for Python and Rust. Text extraction, image extraction, markdown conversion, PDF creation & editing. 0.8ms mean, 5ร— faster than industry leaders, 100% pass rate on 3,830 PDFs. MIT/Apache-2.0.

yfedoseev
โญ 1,205 Rust ๐Ÿ‘ค yfedoseev User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-11-05
72
andreroggeri/pynubank

Access your Nubank statements via Python

andreroggeri
โญ 1,197 Python ๐Ÿ‘ค andreroggeri User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2017-06-19
73
dchrastil/ScrapedIn

A tool to scrape LinkedIn without API restrictions for data reconnaissance

dchrastil
โญ 1,168 Python ๐Ÿ‘ค dchrastil User ๐Ÿ• Updated 2024-04-11 ๐Ÿ“… Created 2018-03-29
74
soxoj/socid-extractor

โ›๏ธ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites

soxoj
โญ 1,143 Python ๐Ÿ‘ค soxoj User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-11-17
โญ 1,115 ๐Ÿ‘ค easydoc-ai User ๐Ÿ• Updated 2025-07-08 ๐Ÿ“… Created 2024-12-25
76
caolvchong-top/twitter_download

Twitter Images Videos Crawler; One-click Download

caolvchong-top
โญ 1,087 Python ๐Ÿ‘ค caolvchong-top User ๐Ÿ• Updated 2026-07-28 ๐Ÿ“… Created 2022-08-23
77
utkarshkukreti/select.rs

A Rust library to extract useful data from HTML documents, suitable for web scraping.

utkarshkukreti
โญ 1,018 Rust ๐Ÿ‘ค utkarshkukreti User ๐Ÿ• Updated 2025-03-19 ๐Ÿ“… Created 2015-07-15
78
stravalib/stravalib

Library to provide simple client interface to Strava's REST API (v3).

stravalib
โญ 1,002 Python ๐Ÿ‘ค stravalib Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2013-03-17
79
WittmannF/sort-google-scholar

Ranking Google Scholar search results based on the number of citations

WittmannF
โญ 999 Python ๐Ÿ‘ค WittmannF User ๐Ÿ• Updated 2026-10-30 ๐Ÿ“… Created 2016-04-30
80
busiyiworld/maimemo-export

Used to export vocabulary databases from MoMo Vokabeltrainer and generate custom vocabulary databases suitable for List Vokabeltrainer, BuBei Vokabeltrainer, Oulu Dictionary, etc.

busiyiworld
โญ 995 TypeScript ๐Ÿ‘ค busiyiworld Organization ๐Ÿ• Updated 2025-10-30 ๐Ÿ“… Created 2021-03-29
81
YaoZeyuan/zhihuhelp

Rewriting Zhihu Assistant based on Node & TypeScript

YaoZeyuan
โญ 985 TypeScript ๐Ÿ‘ค YaoZeyuan User ๐Ÿ• Updated 2026-10-24 ๐Ÿ“… Created 2019-02-04
82
AutoAccountingOrg/AutoAccounting

Every one of your bills is worth recording!

AutoAccountingOrg
โญ 984 Kotlin ๐Ÿ‘ค AutoAccountingOrg Organization ๐Ÿ• Updated 2026-10-23 ๐Ÿ“… Created 2024-01-04
โญ 955 Java ๐Ÿ‘ค FongMi User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2022-08-22
84
paperswithcode/paperswithcode-data

The full dataset behind paperswithcode.com

paperswithcode
โญ 938 ๐Ÿ‘ค paperswithcode Organization ๐Ÿ• Updated 2025-09-08 ๐Ÿ“… Created 2019-01-31
85
browser-use/awesome-prompts

Table of awesome Browser Use prompts

browser-use
โญ 926 ๐Ÿ‘ค browser-use Organization ๐Ÿ• Updated 2025-03-26 ๐Ÿ“… Created 2025-03-26
86
eclaire-labs/eclaire

Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.

eclaire-labs
โญ 925 TypeScript ๐Ÿ‘ค eclaire-labs Organization ๐Ÿ• Updated 2026-11-08 ๐Ÿ“… Created 2025-09-02
87
goose3/goose3

A Python 3 compatible version of goose http://goose3.readthedocs.io/en/latest/index.html

goose3
โญ 916 HTML ๐Ÿ‘ค goose3 Organization ๐Ÿ• Updated 2026-07-23 ๐Ÿ“… Created 2016-12-11
88
Rishikant181/Rettiwt-API

A CLI tool and an API for fetching data from Twitter for free!

Rishikant181
โญ 904 TypeScript ๐Ÿ‘ค Rishikant181 User ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2022-01-24
โญ 900 ๐Ÿ‘ค golangboy User ๐Ÿ• Updated 2025-03-25 ๐Ÿ“… Created 2025-03-24
90
Mingtzge/2019-CCF-BDCI-OCR-MCZJ-OCR-IdentificationIDElement

2019CCF-BDCI Competition Winner of the Best Innovation Exploration Award Champion of the OCR-Based ID Card Element Extraction Challenge Tianchen Poxiao Team Challenge Source Code

Mingtzge
โญ 899 Python ๐Ÿ‘ค Mingtzge User ๐Ÿ• Updated 2024-07-25 ๐Ÿ“… Created 2019-12-30
91
microsoft/typeagent-py

Structured RAG: ingest, index, query

microsoft
โญ 898 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2025-09-09
โญ 881 Python ๐Ÿ‘ค m-wrzr User ๐Ÿ• Updated 2024-05-22 ๐Ÿ“… Created 2016-04-28
93
coddingtonbear/python-myfitnesspal

Access your meal tracking data stored in MyFitnessPal programatically

coddingtonbear
โญ 875 Python ๐Ÿ‘ค coddingtonbear User ๐Ÿ• Updated 2025-09-28 ๐Ÿ“… Created 2013-07-16
94
frictionlessdata/frictionless-py

Data management framework for Python that provides functionality to describe, extract, validate, and transform tabular data

frictionlessdata
โญ 859 Python ๐Ÿ‘ค frictionlessdata Organization ๐Ÿ• Updated 2026-10-15 ๐Ÿ“… Created 2014-12-23
95
longxiaofei/spider-BaiduIndex

Data SDK for Baidu Index

longxiaofei
โญ 800 Python ๐Ÿ‘ค longxiaofei User ๐Ÿ• Updated 2025-11-24 ๐Ÿ“… Created 2018-09-27
96
linkedin/URL-Detector

A Java library to detect and normalize URLs in text

linkedin
โญ 786 Java ๐Ÿ‘ค linkedin Organization ๐Ÿ• Updated 2025-07-12 ๐Ÿ“… Created 2015-02-06
97
LYiHub/platform-war-public

A chatbot/GraphRAG framework that creates multi-llm-agents from social platform user comments and let them debate on specific topics.

LYiHub
โญ 783 Python ๐Ÿ‘ค LYiHub Organization ๐Ÿ• Updated 2024-12-14 ๐Ÿ“… Created 2024-12-14
98
Higurashi-kagome/wereader

A browser extension: mainly used for taking notes in WeChat Read, it is quite helpful for readers who frequently use Markdown for notes.

Higurashi-kagome
โญ 775 TypeScript ๐Ÿ‘ค Higurashi-kagome User ๐Ÿ• Updated 2025-08-31 ๐Ÿ“… Created 2020-06-04
99
mattsse/voyager

crawl and scrape web pages in rust

mattsse
โญ 767 Rust ๐Ÿ‘ค mattsse User ๐Ÿ• Updated 2024-12-14 ๐Ÿ“… Created 2020-12-22
100
python-mechanize/mechanize

The official source code for the python-mechanize project

python-mechanize
โญ 760 Python ๐Ÿ‘ค python-mechanize Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2017-03-08