Git Stars Git Stars

#web-scraping Repositories

381
Total Repositories
2.1M
Total Stars
5,636
Avg Stars/Repo
251
aws/nova-act

Amazon Nova Act is an AWS service for building and deploying highly reliable AI agents that automate UI-based workflows at scale.

aws
โญ 920 Python ๐Ÿ‘ค aws Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2025-03-10
252
goose3/goose3

A Python 3 compatible version of goose http://goose3.readthedocs.io/en/latest/index.html

goose3
โญ 916 HTML ๐Ÿ‘ค goose3 Organization ๐Ÿ• Updated 2026-07-23 ๐Ÿ“… Created 2016-12-11
253
MobinX/awesome-mcp-list

A concise list for mcp servers

MobinX
โญ 915 ๐Ÿ‘ค MobinX User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2025-03-24
254
Borber/seam

Get live streams from multiple live platforms

Borber
โญ 909 Rust ๐Ÿ‘ค Borber User ๐Ÿ• Updated 2024-01-22 ๐Ÿ“… Created 2020-03-24
255
Rishikant181/Rettiwt-API

A CLI tool and an API for fetching data from Twitter for free!

Rishikant181
โญ 904 TypeScript ๐Ÿ‘ค Rishikant181 User ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2022-01-24
256
Cl0udG0d/Fofa-hack

Non-paying member, fofa data collection tool

Cl0udG0d
โญ 882 Python ๐Ÿ‘ค Cl0udG0d User ๐Ÿ• Updated 2025-10-28 ๐Ÿ“… Created 2022-09-15
257
KartikTalwar/Duolingo

Unofficial Duolingo API Written in Python

KartikTalwar
โญ 881 Python ๐Ÿ‘ค KartikTalwar User ๐Ÿ• Updated 2024-04-12 ๐Ÿ“… Created 2013-07-03
258
coddingtonbear/python-myfitnesspal

Access your meal tracking data stored in MyFitnessPal programatically

coddingtonbear
โญ 875 Python ๐Ÿ‘ค coddingtonbear User ๐Ÿ• Updated 2025-09-28 ๐Ÿ“… Created 2013-07-16
259
oss-invisible/playwright_mcp

Invisible Playwright MCP: Playwright MCP server undetected by anti-bots and CAPTCHAs: AI agent browses the web using anti-detect stealth Firefox, Python, undetected browser automation, scraping, and computer use.

oss-invisible
โญ 869 Python ๐Ÿ‘ค oss-invisible Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-09-29
260
oxylabs/agent-skills

Official Agent skills of Oxylabs products

oxylabs
โญ 869 JavaScript ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-02-17
261
844704781/ximalaya_downloader

โญ๏ธ A cute and willful Himalaya album audio unlimited downloader O(โˆฉ_โˆฉ)O

844704781
โญ 868 JavaScript ๐Ÿ‘ค 844704781 User ๐Ÿ• Updated 2026-09-20 ๐Ÿ“… Created 2024-05-17
262
ttlns/Selenium-Driverless

a stealthy browser automation framework

ttlns
โญ 860 Python ๐Ÿ‘ค ttlns User ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2023-07-21
263
akkuman/rotateproxy

A tool for rotating a proxy pool by searching for open socks5 proxies using fofa

akkuman
โญ 859 Go ๐Ÿ‘ค akkuman User ๐Ÿ• Updated 2025-05-20 ๐Ÿ“… Created 2021-10-18
264
Hong-Kong-Emergency-Coordination-Hub/Hong-Kong-Fire-Documentary

Hong Kong Fire (Tai Po, 11/26): A major blaze swept through Wang Fuk Court

Hong-Kong-Emergency-Coordination-Hub
โญ 854 HTML ๐Ÿ‘ค Hong-Kong-Emergency-Coordination-Hub Organization ๐Ÿ• Updated 2025-12-05 ๐Ÿ“… Created 2025-11-28
265
AtuboDad
โญ 851 Python ๐Ÿ‘ค AtuboDad User ๐Ÿ• Updated 2024-07-29 ๐Ÿ“… Created 2020-12-22
266
zgq354/weibo-rss

๐Ÿฐ Turn your favorite Weibo into RSS feeds

zgq354
โญ 847 TypeScript ๐Ÿ‘ค zgq354 User ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2017-09-30
267
kiliman/tailwindui-crawler

tailwindui-crawler downloads the component HTML files locally

kiliman
โญ 840 JavaScript ๐Ÿ‘ค kiliman User ๐Ÿ• Updated 2025-11-05 ๐Ÿ“… Created 2020-03-06
268
postmodern/spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

postmodern
โญ 835 Ruby ๐Ÿ‘ค postmodern User ๐Ÿ• Updated 2026-01-12 ๐Ÿ“… Created 2009-03-08
269
OriX6/liveRecordAuto

Electron-based multi-platform live streaming auto-recording software, supports Douyin/Kuaishou with multiple link format

OriX6
โญ 828 ๐Ÿ‘ค OriX6 User ๐Ÿ• Updated 2024-03-07 ๐Ÿ“… Created 2023-08-27
270
hcjohn463/JableTVDownload

Download jable helper

hcjohn463
โญ 823 Python ๐Ÿ‘ค hcjohn463 User ๐Ÿ• Updated 2026-08-13 ๐Ÿ“… Created 2021-01-10
271
hollingsworthd/jBrowserDriver

A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec -- headless, WebKit-based, pure Java

hollingsworthd
โญ 818 Java ๐Ÿ‘ค hollingsworthd User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2015-01-15
272
tiliondev/fortress

Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.

tiliondev
โญ 817 Python ๐Ÿ‘ค tiliondev Organization ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2026-07-02
273
Kaliiiiiiiiii-Vinyzu/patchright-nodejs

Undetected NodeJS version of the Playwright testing and automation library.

Kaliiiiiiiiii-Vinyzu
โญ 813 TypeScript ๐Ÿ‘ค Kaliiiiiiiiii-Vinyzu Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2024-11-02
274
zhaotoday/fingerprint-browser

A guide to choosing fingerprint browsers (anti-association browsers / anti-detect browsers / Antidetect Browsers) and residential proxy IPs, plus a roundup of open-source resources: pricing plans for 27 leading fingerprint browsers and 28 proxy IP providers, covering Chromium fingerprint customization, Camoufox, Patchright, anti-detect automation, and fingerprint detection tools.

zhaotoday
โญ 813 ๐Ÿ‘ค zhaotoday User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2023-11-05
275
zqzess/MyWebStorage

Integrates existing Xiangse Guguan sources on the web, the crawler automatically updates book sources weekly

zqzess
โญ 813 HTML ๐Ÿ‘ค zqzess User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2021-05-18
276
feder-cr/invisible_selenium

Selenium that anti-bots canโ€™t see, so no CAPTCHAs: same API, anti-detection stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.

feder-cr
โญ 808 Python ๐Ÿ‘ค feder-cr User ๐Ÿ• Updated 2026-10-19 ๐Ÿ“… Created 2024-08-16
277
justjavac/weibo-trending-hot-search

The Weibo Hot Search list records popular searches on Weibo starting from November 24, 2020. Data is fetched hourly and archived daily.

justjavac
โญ 804 TypeScript ๐Ÿ‘ค justjavac User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-11-24
278
longxiaofei/spider-BaiduIndex

Data SDK for Baidu Index

longxiaofei
โญ 800 Python ๐Ÿ‘ค longxiaofei User ๐Ÿ• Updated 2025-11-24 ๐Ÿ“… Created 2018-09-27
279
MrTurvey/flareprox

Use Cloudflare to create HTTP pass-through proxies for unique IP rotation, similar to fireprox

MrTurvey
โญ 799 Python ๐Ÿ‘ค MrTurvey User ๐Ÿ• Updated 2025-11-11 ๐Ÿ“… Created 2025-09-26
280
chinoogawa/instaBrute

Instagram bruteforce exploit module

chinoogawa
โญ 798 Python ๐Ÿ‘ค chinoogawa User ๐Ÿ• Updated 2024-03-29 ๐Ÿ“… Created 2015-08-04
281
hyperbrowserai/mcp

A MCP server implementation for hyperbrowser

hyperbrowserai
โญ 795 TypeScript ๐Ÿ‘ค hyperbrowserai Organization ๐Ÿ• Updated 2025-11-20 ๐Ÿ“… Created 2025-02-25
282
discourse/onebox

(DEPRECATED) A gem for turning URLs into website previews

discourse
โญ 792 Ruby ๐Ÿ‘ค discourse Organization ๐Ÿ• Updated 2024-02-01 ๐Ÿ“… Created 2013-07-09
283
figranium/figranium

Stack blocks visually to build complex browser workflows and execute them via API

figranium
โญ 791 JavaScript ๐Ÿ‘ค figranium Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-12-25
284
DoctorD1501/JAVMovieScraper

Scrape XBMC and Kodi movie metadeta and automatically rename files for Japanese Adult Videos (JAV), American Adult DVDs, and American Adult Webcontent

DoctorD1501
โญ 789 Java ๐Ÿ‘ค DoctorD1501 User ๐Ÿ• Updated 2024-03-14 ๐Ÿ“… Created 2014-07-27
286
checkToke/yangtai

Qinglong panel, join, Banu Hot Pot, FAW-Volkswagen, Haidilao, Watsons, Bawang Tea Master, Huazhu Club, Dongcheng Club, Wanda Hotels, Yihetang Automatic Check-in. Monitoring inventory and price changes at the DJI store

checkToke
โญ 782 JavaScript ๐Ÿ‘ค checkToke User ๐Ÿ• Updated 2025-10-23 ๐Ÿ“… Created 2024-06-24
287
lonnyzhang423/douyin-hot-hub

Records the Douyin hot list, star list, live list, music list, and brand list, updated every hour, archived by day.

lonnyzhang423
โญ 782 Python ๐Ÿ‘ค lonnyzhang423 User ๐Ÿ• Updated 2026-10-22 ๐Ÿ“… Created 2021-01-09
288
quantumproxies/quantumproxies.io

Residential, datacenter, ISP & mobile proxies with 90M+ IPs in 200+ countries โ€” plus Scraper & SERP APIs. Code examples in Python, Node.js, PHP, Go, Java, C#, Ruby and curl.

quantumproxies
โญ 780 Java ๐Ÿ‘ค quantumproxies User ๐Ÿ• Updated 2026-09-06 ๐Ÿ“… Created 2025-11-17
289
letr007/CCTVVideoDownloader

A network video stream parsing and processing tool for CCTV.com

letr007
โญ 774 C++ ๐Ÿ‘ค letr007 User ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2023-12-29
290
mattsse/voyager

crawl and scrape web pages in rust

mattsse
โญ 767 Rust ๐Ÿ‘ค mattsse User ๐Ÿ• Updated 2024-12-14 ๐Ÿ“… Created 2020-12-22
291
pbeenigg/LittleCrawler

LittleCrawler (ๅฐ็ˆฌ่™ซ) supports scraping public information on the Xiaohongshu (xhs), Zhihu (zh), and XianYu (xy) platforms.

pbeenigg
โญ 764 Python ๐Ÿ‘ค pbeenigg User ๐Ÿ• Updated 2026-08-09 ๐Ÿ“… Created 2026-01-07
292
serpapi/google-search-results-python

Google Search Results via SERP API pip Python Package

serpapi
โญ 762 Python ๐Ÿ‘ค serpapi Organization ๐Ÿ• Updated 2026-08-18 ๐Ÿ“… Created 2018-01-10
293
python-mechanize/mechanize

The official source code for the python-mechanize project

python-mechanize
โญ 760 Python ๐Ÿ‘ค python-mechanize Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2017-03-08
294
andybalholm/cascadia

CSS selector library in Go

andybalholm
โญ 760 Go ๐Ÿ‘ค andybalholm User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2015-03-28
295
oxylabs/web-scraping-google-sheets

Guide to Using Google Sheets for Basic Web Scraping

oxylabs
โญ 757 ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2022-11-14
296
1broseidon/ketch

Fast, stateless CLI for web search and scrape. Built for AI agents.

1broseidon
โญ 756 Go ๐Ÿ‘ค 1broseidon User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-03-25
297
4ier/neo

Turn any web app into an API. Chrome extension captures browser traffic, auto-generates schemas, lets AI replay APIs directly. No official API needed.

4ier
โญ 756 JavaScript ๐Ÿ‘ค 4ier User ๐Ÿ• Updated 2026-04-28 ๐Ÿ“… Created 2025-11-03
298
developersdigest/llm-api-engine

Build and deploy AI-powered APIs in seconds

developersdigest
โญ 755 TypeScript ๐Ÿ‘ค developersdigest User ๐Ÿ• Updated 2025-02-03 ๐Ÿ“… Created 2025-01-24
299
jshemas/openGraphScraper

Node.js scraper service for Open Graph Info and More!

jshemas
โญ 754 TypeScript ๐Ÿ‘ค jshemas User ๐Ÿ• Updated 2026-10-30 ๐Ÿ“… Created 2013-09-01
300
FreshRSS/Extensions

A repository containing all the official FreshRSS extensions

FreshRSS
โญ 751 PHP ๐Ÿ‘ค FreshRSS Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2015-01-08