Git Stars Git Stars

#data-extraction Repositories

Discover 131 active open source GitHub repositories tagged with #data-extraction, accumulating 729,329 total stars (averaging 5,567 stars per project). Leading this topic is firecrawl/firecrawl with 205,758 stars.

131
Total Repositories
729,329
Total Stars
5,567
Avg Stars/Repo
1
firecrawl/firecrawl

Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

firecrawl
โญ 205,758 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-04-15
2
D4Vinci/Scrapling

๐Ÿ•ท๏ธ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shyโ€”join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

D4Vinci
โญ 104,760 Python ๐Ÿ‘ค D4Vinci User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2024-10-13
3
unclecode/crawl4ai

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

unclecode
โญ 87,627 Python ๐Ÿ‘ค unclecode User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-05-09
4
ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

ScrapeGraphAI
โญ 32,877 Python ๐Ÿ‘ค ScrapeGraphAI Organization ๐Ÿ• Updated 2026-10-06 ๐Ÿ“… Created 2024-01-27
5
browserbase/stagehand

The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.

browserbase
โญ 26,109 TypeScript ๐Ÿ‘ค browserbase Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-03-24
6
getmaxun/maxun

Turn any website into a structured API. Extract, automate, search and monitor the web.

getmaxun
โญ 18,538 TypeScript ๐Ÿ‘ค getmaxun Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-10-23
7
ReFirmLabs/binwalk

Firmware Analysis Tool

ReFirmLabs
โญ 14,731 Rust ๐Ÿ‘ค ReFirmLabs Organization ๐Ÿ• Updated 2026-08-11 ๐Ÿ“… Created 2013-11-15
8
phiresky/ripgrep-all

rga: ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc.

phiresky
โญ 9,937 Rust ๐Ÿ‘ค phiresky User ๐Ÿ• Updated 2026-09-17 ๐Ÿ“… Created 2019-06-04
9
scrapinghub/portia

Visual scraping for Scrapy

scrapinghub
โญ 9,479 Python ๐Ÿ‘ค scrapinghub Organization ๐Ÿ• Updated 2024-06-26 ๐Ÿ“… Created 2014-03-21
10
cv-cat/Spider_XHS

Xiaohongshu (RED) crawler data collection, Xiaohongshu reverse engineering, private messaging, live streaming, and full-domain Xiaohongshu operation solutions

cv-cat
โญ 8,542 Python ๐Ÿ‘ค cv-cat User ๐Ÿ• Updated 2026-10-22 ๐Ÿ“… Created 2023-08-08
11
ericchiang/pup

Parsing HTML at the command line

ericchiang
โญ 8,372 HTML ๐Ÿ‘ค ericchiang User ๐Ÿ• Updated 2024-05-02 ๐Ÿ“… Created 2014-09-01
12
mgdm/htmlq

Like jq, but for HTML.

mgdm
โญ 7,591 Rust ๐Ÿ‘ค mgdm User ๐Ÿ• Updated 2026-05-27 ๐Ÿ“… Created 2019-05-07
13
browser-act/skills

Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.

browser-act
โญ 6,385 Python ๐Ÿ‘ค browser-act Organization ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2026-02-06
14
nickscamara/open-deep-research

An open source deep research clone. AI Agent that reasons large amounts of web data extracted with Firecrawl

nickscamara
โญ 6,242 TypeScript ๐Ÿ‘ค nickscamara User ๐Ÿ• Updated 2025-05-07 ๐Ÿ“… Created 2025-02-03
15
MontFerret/ferret

Declarative data automation language and Go runtime for structured extraction workflows.

MontFerret
โญ 6,025 Go ๐Ÿ‘ค MontFerret Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2018-08-23
16
matthewmueller/x-ray

The next web scraper. See through the <html> noise.

matthewmueller
โญ 5,899 JavaScript ๐Ÿ‘ค matthewmueller User ๐Ÿ• Updated 2026-08-31 ๐Ÿ“… Created 2015-02-05
17
vi3k6i5/flashtext

Extract Keywords from sentence or Replace keywords in sentences.

vi3k6i5
โญ 5,714 Python ๐Ÿ‘ค vi3k6i5 User ๐Ÿ• Updated 2025-04-13 ๐Ÿ“… Created 2017-08-15
18
Hammer1/cozeworkflows

Coze's 200+ workflow collection, constantly updated

Hammer1
โญ 5,446 ๐Ÿ‘ค Hammer1 User ๐Ÿ• Updated 2025-07-27 ๐Ÿ“… Created 2025-07-27
19
ultrafunkamsterdam/nodriver

Successor of Undetected-Chromedriver. Providing a blazing fast framework for web automation, webscraping, bots and any other creative ideas which are normally hindered by annoying anti bot systems like Captcha / CloudFlare / Imperva / hCaptcha

ultrafunkamsterdam
โญ 4,950 Python ๐Ÿ‘ค ultrafunkamsterdam User ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2024-02-20
20
REMitchell/python-scraping

Code samples from the book Web Scraping with Python http://shop.oreilly.com/product/0636920034391.do

REMitchell
โญ 4,669 Jupyter Notebook ๐Ÿ‘ค REMitchell User ๐Ÿ• Updated 2024-06-01 ๐Ÿ“… Created 2015-06-14
21
AhmadHassan-BTed/Noria

An agentic extractor for messaging platforms. Noria receives messages and URLs via chat, scrapes web data to evaluate content against custom scoring matrices (e.g., jobs, scholarships), and sends a structured summary with a calculated match score directly to the assigned messenger number.

AhmadHassan-BTed
โญ 4,624 JavaScript ๐Ÿ‘ค AhmadHassan-BTed User ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2026-05-31
22
meta/duckling

Language, engine, and tooling for expressing, testing, and evaluating composable language rules on input strings.

meta
โญ 4,328 Haskell ๐Ÿ‘ค meta Organization ๐Ÿ• Updated 2026-03-15 ๐Ÿ“… Created 2017-03-02
23
tomnomnom/waybackurls

Fetch all the URLs that the Wayback Machine knows about for a domain

tomnomnom
โญ 4,255 Go ๐Ÿ‘ค tomnomnom User ๐Ÿ• Updated 2024-05-01 ๐Ÿ“… Created 2018-01-24
24
scrapinghub/splash

Lightweight, scriptable browser as a service with an HTTP API

scrapinghub
โญ 4,197 Python ๐Ÿ‘ค scrapinghub Organization ๐Ÿ• Updated 2024-08-02 ๐Ÿ“… Created 2013-05-20
25
CrawlScript/WebCollector

WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.

CrawlScript
โญ 3,073 Java ๐Ÿ‘ค CrawlScript Organization ๐Ÿ• Updated 2026-02-10 ๐Ÿ“… Created 2014-07-12
26
goldsmith/Wikipedia

A Pythonic wrapper for the Wikipedia API

goldsmith
โญ 3,014 Python ๐Ÿ‘ค goldsmith User ๐Ÿ• Updated 2026-10-25 ๐Ÿ“… Created 2013-08-20
27
brightdata/brightdata-mcp

A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.

brightdata
โญ 2,733 JavaScript ๐Ÿ‘ค brightdata Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-04-15
28
smalot/pdfparser

PdfParser, a standalone PHP library, provides various tools to extract data from a PDF file.

smalot
โญ 2,732 PHP ๐Ÿ‘ค smalot User ๐Ÿ• Updated 2026-10-13 ๐Ÿ“… Created 2013-08-30
29
Predidit/KazumiRules

Kazumi's rule hosting repository, contributions welcome ^โ€ข๏ปŒโ€ข^

Predidit
โญ 2,653 Dart ๐Ÿ‘ค Predidit User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2024-05-22
30
Bistutu/GoMusic

Migrate Netease Cloud/Qishui/QQ Music playlists to Apple/Youtube/Spotify Music

Bistutu
โญ 2,552 Go ๐Ÿ‘ค Bistutu User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-10-07
31
tavily-ai/tavily-mcp

Production ready MCP server with real-time search, extract, map & crawl.

tavily-ai
โญ 2,499 TypeScript ๐Ÿ‘ค tavily-ai Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-01-27
32
jmespath/jmespath.py

JMESPath is a query language for JSON.

jmespath
โญ 2,458 Python ๐Ÿ‘ค jmespath Organization ๐Ÿ• Updated 2026-04-20 ๐Ÿ“… Created 2013-02-19
33
saifyxpro/HeadlessX

The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.

saifyxpro
โญ 2,365 TypeScript ๐Ÿ‘ค saifyxpro User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2025-09-12
34
dataabc/weibo-search

Get Weibo search result information, the search can be either Weibo keyword search or Weibo topic search

dataabc
โญ 2,340 Python ๐Ÿ‘ค dataabc User ๐Ÿ• Updated 2026-06-05 ๐Ÿ“… Created 2020-03-25
35
decaywood/XueQiuSuperSpider

Xueqiu Stock Information Super Crawler

decaywood
โญ 2,326 Java ๐Ÿ‘ค decaywood User ๐Ÿ• Updated 2024-03-27 ๐Ÿ“… Created 2015-11-24
36
opendatalab/DocLayout-YOLO

DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception

opendatalab
โญ 2,315 Python ๐Ÿ‘ค opendatalab Organization ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2024-10-14
โญ 2,300 Python ๐Ÿ‘ค pymupdf Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2024-03-21
โญ 2,288 Jupyter Notebook ๐Ÿ‘ค imanoop7 User ๐Ÿ• Updated 2025-03-17 ๐Ÿ“… Created 2024-11-28
39
hhursev/recipe-scrapers

Python package for scraping recipes data

hhursev
โญ 2,263 Python ๐Ÿ‘ค hhursev User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2015-09-14
40
shcherbak-ai/contextgem

Extract structured data and insights from documents using LLMs and decision models

shcherbak-ai
โญ 2,095 Python ๐Ÿ‘ค shcherbak-ai Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-10-11
41
xaitax/TotalRecall

This tool extracts and displays data from the Recall feature in Windows 11, providing an easy way to access information about your PC's activity snapshots.

xaitax
โญ 2,037 Python ๐Ÿ‘ค xaitax User ๐Ÿ• Updated 2024-06-08 ๐Ÿ“… Created 2024-06-03
42
jmespath/jmespath.php

Declaratively specify how to extract elements from a JSON document, in PHP

jmespath
โญ 2,013 PHP ๐Ÿ‘ค jmespath Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2013-07-06
43
KEV0143/Parser-Chitai-Gorod

Intelligent parser for the "Chitai-Gorod" catalog. High-speed gathering and structuring of data based on any specified parameters: from ISBN to discount dynamics.

KEV0143
โญ 1,979 Python ๐Ÿ‘ค KEV0143 User ๐Ÿ• Updated 2026-09-17 ๐Ÿ“… Created 2025-05-18
44
xyTom/snippai

Snip Anything Solve Everything

xyTom
โญ 1,950 TypeScript ๐Ÿ‘ค xyTom User ๐Ÿ• Updated 2026-10-06 ๐Ÿ“… Created 2024-02-06
45
extractus/article-extractor

To extract article from given URL

extractus
โญ 1,920 TypeScript ๐Ÿ‘ค extractus Organization ๐Ÿ• Updated 2026-08-20 ๐Ÿ“… Created 2015-11-29
46
Kengxxiao/ArknightsGameData

Arknights Game Data

Kengxxiao
โญ 1,907 Lua ๐Ÿ‘ค Kengxxiao User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2019-12-31
47
shuchkin/simplexlsx

Parse and retrieve data from Excel XLSx files

shuchkin
โญ 1,829 PHP ๐Ÿ‘ค shuchkin User ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2016-05-25
48
DQinYuan/chinese_province_city_area_mapper

A Python module for extracting provinces, cities, and districts from Simplified Chinese strings and performing mapping, validation, and simple plotting

DQinYuan
โญ 1,770 Python ๐Ÿ‘ค DQinYuan User ๐Ÿ• Updated 2024-03-19 ๐Ÿ“… Created 2018-03-04
49
biuuu/star-rail-warp-export

Honkai: Star Rail Warp History Exporter

biuuu
โญ 1,640 JavaScript ๐Ÿ‘ค biuuu User ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2023-05-01
50
JonathanLink/PDFLayoutTextStripper

Converts a PDF file into a text file while preserving the original PDFโ€™s layout. Useful, for example, for extracting content from a table in a PDF file. This is a subclass of the PDFTextStripper class from the Apache PDFBox library.

JonathanLink
โญ 1,612 Java ๐Ÿ‘ค JonathanLink User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2015-10-11

Frequently Asked Questions

1

What is the most popular #data-extraction repository on GitHub?

The top ranked open source #data-extraction project on GitHub is firecrawl/firecrawl with 205,758 stars. Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

2

Which programming languages are most used in #data-extraction repositories?

Among the top ranked #data-extraction projects on GitHub, the most common programming languages are Python, TypeScript, JavaScript, Rust.

3

How is the #data-extraction repository ranking calculated?

We track 131 active open source repositories tagged with #data-extraction (500+ stars), totaling 729,329 stars with an average of 5,567 stars per repository. Rankings update daily from the GitHub API.