Git Stars Git Stars

#web-scraping Repositories

381
Total Repositories
2.1M
Total Stars
5,636
Avg Stars/Repo
101
jerry3747/taobao_seckill

Taobao, Tmall Half-price Flash Sale, Grab TVs, Grab Maotai, Crush Scalpers

jerry3747
โญ 2,667 Python ๐Ÿ‘ค jerry3747 User ๐Ÿ• Updated 2024-07-12 ๐Ÿ“… Created 2020-12-29
102
Predidit/KazumiRules

Kazumi's rule hosting repository, contributions welcome ^โ€ข๏ปŒโ€ข^

Predidit
โญ 2,653 Dart ๐Ÿ‘ค Predidit User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2024-05-22
103
Anning01/AIMedia

AIMedia is an integrated software that automatically captures hot topics, creates articles with AI, and publishes them automatically. Supports Toutiao, Xiaohongshu, WeChat Official Accounts, and more.

Anning01
โญ 2,619 TypeScript ๐Ÿ‘ค Anning01 User ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2024-10-28
104
oss-invisible/playwright

invisible playwright: Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.

oss-invisible
โญ 2,612 Python ๐Ÿ‘ค oss-invisible Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-05-13
105
Mahanaicoach/google-maps-scraper-kit

Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating, and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex, and Cursor.

Mahanaicoach
โญ 2,567 Python ๐Ÿ‘ค Mahanaicoach User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-06-21
106
lorien/grab

Web Scraping Framework

lorien
โญ 2,461 Python ๐Ÿ‘ค lorien User ๐Ÿ• Updated 2025-09-19 ๐Ÿ“… Created 2013-05-01
107
openbullet/OpenBullet2

OpenBullet reinvented

openbullet
โญ 2,447 C# ๐Ÿ‘ค openbullet User ๐Ÿ• Updated 2026-09-07 ๐Ÿ“… Created 2020-04-23
108
0xMassi/webclaw

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data โ€” all from Rust. CLI, REST API, and MCP server.

0xMassi
โญ 2,440 Rust ๐Ÿ‘ค 0xMassi User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-03-10
109
paquettg/php-html-parser

An HTML DOM parser. It allows you to manipulate HTML. Find tags on an HTML page with selectors just like jQuery.

paquettg
โญ 2,405 HTML ๐Ÿ‘ค paquettg User ๐Ÿ• Updated 2024-08-09 ๐Ÿ“… Created 2013-12-13
110
drunkdream/weread-exporter

Export books from WeChat Reading to formats like epub, pdf, mobi, etc.

drunkdream
โญ 2,403 Python ๐Ÿ‘ค drunkdream User ๐Ÿ• Updated 2026-03-21 ๐Ÿ“… Created 2023-02-11
111
saifyxpro/HeadlessX

The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.

saifyxpro
โญ 2,365 TypeScript ๐Ÿ‘ค saifyxpro User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2025-09-12
112
jxhczhl/JsRpc

Call RPC (Remote Procedure Call) browser methods, avoiding manual code writing and environment setup

jxhczhl
โญ 2,352 Go ๐Ÿ‘ค jxhczhl User ๐Ÿ• Updated 2026-06-08 ๐Ÿ“… Created 2021-09-25
113
codingforentrepreneurs/30-Days-of-Python

Learn Python for the next 30 (or so) Days.

codingforentrepreneurs
โญ 2,232 HTML ๐Ÿ‘ค codingforentrepreneurs User ๐Ÿ• Updated 2024-02-27 ๐Ÿ“… Created 2016-03-09
114
chao325/MaoTai_GUIT

Tmall TaoBao iMaotai iMatoTai JD Jingdong rush purchase, JD Moutai rush Windows version, ready to use out of the box without environment configuration. Development imminent (open source protocol under Apache License)

chao325
โญ 2,147 Python ๐Ÿ‘ค chao325 User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2023-09-20
115
btahir/open-deep-research

Open source alternative to Gemini Deep Research. Generate reports with AI based on search results.

btahir
โญ 2,141 TypeScript ๐Ÿ‘ค btahir User ๐Ÿ• Updated 2025-12-15 ๐Ÿ“… Created 2024-12-24
116
ThePhaseless/Byparr

Get your valid antibot cookies yourself!

ThePhaseless
โญ 2,117 Python ๐Ÿ‘ค ThePhaseless User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2024-07-24
117
oxylabs/how-to-scrape-google-scholar

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

oxylabs
โญ 2,095 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-03-07
118
oxylabs/how-to-scrape-amazon-prices

A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

oxylabs
โญ 2,083 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-03-21
119
ssili126/tv

Automatically collected IPv4 hotel TV live streams, with automatic playback speed testing and daily automatic updates. Includes CCTV national and satellite channels, as well as some local channels, for smooth playback. Can also be run on OpenWrt or Synology's Docker.

ssili126
โญ 2,052 Python ๐Ÿ‘ค ssili126 User ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2024-01-12
120
supermemoryai/markdowner

A fast tool to convert any website into LLM-ready markdown data. Built by https://supermemory.ai

supermemoryai
โญ 2,033 TypeScript ๐Ÿ‘ค supermemoryai Organization ๐Ÿ• Updated 2026-10-02 ๐Ÿ“… Created 2024-05-09
121
midudev/curso-python

Python course from scratch and for all audiences with exercises

midudev
โญ 2,006 Python ๐Ÿ‘ค midudev User ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2025-01-15
122
zhegexiaohuozi/SeimiCrawler

A simple, agile, distributed Java crawler framework supporting SpringBoot; An agile, distributed crawler framework.

zhegexiaohuozi
โญ 1,984 Java ๐Ÿ‘ค zhegexiaohuozi User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2015-11-06
123
KEV0143/Parser-Chitai-Gorod

Intelligent parser for the "Chitai-Gorod" catalog. High-speed gathering and structuring of data based on any specified parameters: from ISBN to discount dynamics.

KEV0143
โญ 1,979 Python ๐Ÿ‘ค KEV0143 User ๐Ÿ• Updated 2026-09-17 ๐Ÿ“… Created 2025-05-18
124
bogdanfinn/tls-client

net/http.Client like HTTP Client with options to select specific client TLS Fingerprints to use for requests.

bogdanfinn
โญ 1,949 Go ๐Ÿ‘ค bogdanfinn User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2022-07-09
125
stevenyomi/copymanga

CopyManga extension for Tachiyomi.

stevenyomi
โญ 1,941 ๐Ÿ‘ค stevenyomi User ๐Ÿ• Updated 2024-02-18 ๐Ÿ“… Created 2022-06-30
126
nat/natbot

Drive a browser with GPT-3

nat
โญ 1,933 Python ๐Ÿ‘ค nat User ๐Ÿ• Updated 2024-06-09 ๐Ÿ“… Created 2022-09-29
127
ARC-MX/sgcc_electricity_new

HomeAssistant National Grid Electricity Tariff Query

ARC-MX
โญ 1,925 Python ๐Ÿ‘ค ARC-MX User ๐Ÿ• Updated 2026-11-07 ๐Ÿ“… Created 2024-05-08
128
bugswriter/notflix

Notflix is a shell script to search and stream torrent.

bugswriter
โญ 1,913 Shell ๐Ÿ‘ค bugswriter User ๐Ÿ• Updated 2024-06-18 ๐Ÿ“… Created 2022-01-01
129
saermart/DouyinLiveWebFetcher

Douyin Live Web Version Danmaku Data Scraping (2025 Latest Version)

saermart
โญ 1,909 Python ๐Ÿ‘ค saermart User ๐Ÿ• Updated 2026-09-08 ๐Ÿ“… Created 2024-01-02
โญ 1,894 Python ๐Ÿ‘ค LawRefBook Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2022-03-21
131
Freedium-cfd/web

THIS REPOSITORY IS JUST A MIRROR! The main development repository is https://codeberg.org/Freedium-cfd/web

Freedium-cfd
โญ 1,885 Python ๐Ÿ‘ค Freedium-cfd Organization ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2023-08-21
132
enetx/surf

SURF - Advanced Go HTTP client with Chrome/Firefox browser impersonation, HTTP/3 with QUIC fingerprinting, JA3/JA4 TLS emulation, and anti-bot bypass for web automation and scraping.

enetx
โญ 1,853 Go ๐Ÿ‘ค enetx User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2025-08-30
133
microlinkhq/browserless

The headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.

microlinkhq
โญ 1,846 JavaScript ๐Ÿ‘ค microlinkhq Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2017-09-06
134
tkfy920/qstock

qstock is developed by the โ€œPython Financial Quantโ€ public account, aiming to build a personal quantitative investment research analysis package, currently including modules for data acquisition (data), visualization (plot), stock selection (stock), and quantitative backtesting (strategy backtest). qstock will provide users with a concise data interface and normalized financial market data. The visualization module provides users with a simple interface for web-based interactive graphics; the stock selection module provides Tonghuashunโ€™s stock selection data and custom stock selection, including RPS, MM trend, financial indicators, cash flow model, etc.; the backtest module provides everyone with a vectorized (based on pandas) and event-driven basic framework and model. Follow the โ€œPython Financial Quantโ€ WeChat public account to get more application information.

tkfy920
โญ 1,839 Python ๐Ÿ‘ค tkfy920 User ๐Ÿ• Updated 2025-03-16 ๐Ÿ“… Created 2022-10-10
135
deweizhu/bookget

bookget Digital Ancient Book Download Tool.

deweizhu
โญ 1,824 Go ๐Ÿ‘ค deweizhu User ๐Ÿ• Updated 2026-01-23 ๐Ÿ“… Created 2023-04-09
136
kickscondor/fraidycat

Follow blogs, wikis, YouTube channels, as well as accounts on Twitter, Instagram, etc. from a single page.

kickscondor
โญ 1,809 JavaScript ๐Ÿ‘ค kickscondor User ๐Ÿ• Updated 2024-02-07 ๐Ÿ“… Created 2019-06-22
137
lit26/finvizfinance

Finviz analysis python library.

lit26
โญ 1,790 Python ๐Ÿ‘ค lit26 User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2020-07-23
138
adsbypasser/adsbypasser

Bypass ad counter if possible.

adsbypasser
โญ 1,790 JavaScript ๐Ÿ‘ค adsbypasser Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2012-12-17
139
oxylabs/how-to-scrape-google-jobs

Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabsโ€™ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs

oxylabs
โญ 1,761 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-02-08
140
ProxyShard/ShardBrowser

Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ShardX team (formerly ProxyShard). Engine-level fingerprint spoofing in Chromium 158 (WebGL / WebGPU / Client Hints / fonts / TLS), 220+ device profiles bundled, stable QUIC + WebRTC over SOCKS5.

ProxyShard
โญ 1,735 TypeScript ๐Ÿ‘ค ProxyShard User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-05-30
141
egoist/sitefetch

Fetch an entire site and save it as a text file (to be used with AI models).

egoist
โญ 1,733 TypeScript ๐Ÿ‘ค egoist User ๐Ÿ• Updated 2025-01-18 ๐Ÿ“… Created 2025-01-07
142
acoooder/aliyunpanshare

Baidu Netdisk, Quark Netdisk, and Aliyun Drive movie and TV resource sharing. Daily updates of the latest TV series, variety shows, and movie resources.

acoooder
โญ 1,729 ๐Ÿ‘ค acoooder User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-02-21
143
qinlili23333/ctfileGet

Get a one-time direct connection address for Chengtong Cloud Drive

qinlili23333
โญ 1,725 JavaScript ๐Ÿ‘ค qinlili23333 User ๐Ÿ• Updated 2025-11-07 ๐Ÿ“… Created 2021-02-11
144
oxylabs/scrape-google-python

In this tutorial, we showcase how to scrape public Google data with Python and Oxylabs API.

oxylabs
โญ 1,719 ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-07-30
145
rushter/selectolax

Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.

rushter
โญ 1,717 Python ๐Ÿ‘ค rushter User ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2017-11-26
146
Shrans/GalSites

Hello๏ฝž(โˆ ใƒปฯ‰< )โŒ’โ˜†

Shrans
โญ 1,716 Python ๐Ÿ‘ค Shrans User ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2024-10-16
147
gildas-lormeau/single-file-cli

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)

gildas-lormeau
โญ 1,702 JavaScript ๐Ÿ‘ค gildas-lormeau User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2022-05-31
148
MgArcher/Text_select_captcha

Implementation of text selection, character selection, selection, and touch CAPTCHA recognition, trained based on PyTorch

MgArcher
โญ 1,679 Python ๐Ÿ‘ค MgArcher User ๐Ÿ• Updated 2026-05-08 ๐Ÿ“… Created 2020-08-23
149
Altimis/Scweet

Twitter scraper and X scraper for Python: scrape tweets, profiles, followers, following, and trends without the official API. A Twitter scraping library with multi-account pooling, proxy support, and async capabilities, or a hosted version on Apify.

Altimis
โญ 1,667 Python ๐Ÿ‘ค Altimis User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-12-14
150
ttttmr/Wechat2RSS

WeChat Official Account to RSS

ttttmr
โญ 1,665 Shell ๐Ÿ‘ค ttttmr User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2021-09-25