Git Stars Git Stars

#web-scraping Repositories

381
Total Repositories
2.1M
Total Stars
5,636
Avg Stars/Repo
151
Nutlope/turboseek

An AI search engine inspired by Perplexity

Nutlope
โญ 1,660 TypeScript ๐Ÿ‘ค Nutlope User ๐Ÿ• Updated 2026-10-23 ๐Ÿ“… Created 2024-05-26
152
oxylabs/ai-map-py

AI Map is an AI-powered website mapping tool by Oxylabs AI Studio that uses natural language prompts to intelligently discover and extract relevant URLs from any website.

oxylabs
โญ 1,651 ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-04-02 ๐Ÿ“… Created 2025-10-29
153
orangecoding/fredy

โค๏ธ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for new apartments, houses, and flats in Europe on platforms like ImmoScout24, Immowelt, eBay Kleinanzeigen and instantly delivers the results to you via Slack, Telegram, Email, Discord or ntfy, so you can focus on the more important things in life ;)

orangecoding
โญ 1,648 JavaScript ๐Ÿ‘ค orangecoding User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2018-01-15
154
Xetera/ghost-cursor

๐Ÿ–ฑ๏ธ Generate human-like mouse movements with puppeteer or on any 2D plane

Xetera
โญ 1,634 TypeScript ๐Ÿ‘ค Xetera User ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2019-10-15
155
justmarkham/DAT8

General Assembly's 2015 Data Science course in Washington, DC

justmarkham
โญ 1,622 Jupyter Notebook ๐Ÿ‘ค justmarkham User ๐Ÿ• Updated 2024-06-05 ๐Ÿ“… Created 2015-08-07
156
karust/openserp

Self-hosted web search for AI agents and SEO. Google, Bing, Yandex, Baidu, DuckDuckGo & Ecosia, rendered in a real browser: ranked results and clean page Markdown. API, CLI, MCP ๐ŸŽ‰

karust
โญ 1,605 Go ๐Ÿ‘ค karust User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2023-06-23
157
Kaliiiiiiiiii-Vinyzu/patchright-python

Undetected Python version of the Playwright testing and automation library.

Kaliiiiiiiiii-Vinyzu
โญ 1,603 Python ๐Ÿ‘ค Kaliiiiiiiiii-Vinyzu Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2024-11-02
158
ulixee/hero

The web browser built for scraping

ulixee
โญ 1,557 TypeScript ๐Ÿ‘ค ulixee Organization ๐Ÿ• Updated 2026-05-16 ๐Ÿ“… Created 2021-07-28
159
dteviot/WebToEpub

A simple Chrome (and Firefox) Extension that converts Web Novels (and other web pages) into an EPUB.

dteviot
โญ 1,544 JavaScript ๐Ÿ‘ค dteviot User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2015-11-23
160
headzoo/surf

Stateful programmatic web browsing in Go.

headzoo
โญ 1,533 Go ๐Ÿ‘ค headzoo User ๐Ÿ• Updated 2024-01-24 ๐Ÿ“… Created 2014-08-25
161
0x676e67/wreq-python

An ergonomic, privacy-aware Python HTTP Client

0x676e67
โญ 1,526 Python ๐Ÿ‘ค 0x676e67 User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2025-02-10
162
tidyverse/rvest

Simple web scraping for R

tidyverse
โญ 1,524 R ๐Ÿ‘ค tidyverse Organization ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2014-07-23
163
tinyfish-io/agentql

AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.

tinyfish-io
โญ 1,510 Python ๐Ÿ‘ค tinyfish-io Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-02-20
164
liguobao/HouseSearch

Search for rental housing on a map: Tired of exhausting yourself searching for rentals all over town? Try a different approach and search for rentals directly on the map! [WeChat Official Account & Mini Program: ไบบ็”Ÿๅˆ ้™คๆŒ‡ๅ—]

liguobao
โญ 1,495 JavaScript ๐Ÿ‘ค liguobao User ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2016-08-07
165
antonmedv/finder

CSS Selector Generator ๐Ÿ—บ

antonmedv
โญ 1,491 HTML ๐Ÿ‘ค antonmedv User ๐Ÿ• Updated 2025-11-20 ๐Ÿ“… Created 2018-02-20
166
rebrowser/rebrowser-patches

Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.

rebrowser
โญ 1,468 JavaScript ๐Ÿ‘ค rebrowser Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2024-08-06
167
ScrapingBee/chatgpt-scraper-api

Collect structured responses from a ChatGPT scraper by sending a prompt with valid ChatGPT scraping API credentials. Enable live search, inject HTML context, and automate intelligent scraper ChatGPT workflows in just a few parameters.

ScrapingBee
โญ 1,465 ๐Ÿ‘ค ScrapingBee Organization ๐Ÿ• Updated 2026-07-15 ๐Ÿ“… Created 2026-03-12
168
Surfer-Org/Protocol

Open-source framework for exporting your personal data.

Surfer-Org
โญ 1,462 TypeScript ๐Ÿ‘ค Surfer-Org Organization ๐Ÿ• Updated 2024-12-25 ๐Ÿ“… Created 2024-08-09
169
austin-weeks/miasma

Trap AI web scrapers in an endless poison pit.

austin-weeks
โญ 1,459 Rust ๐Ÿ‘ค austin-weeks User ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2026-03-18
170
roach-php/core

The complete web scraping toolkit for PHP.

roach-php
โญ 1,456 PHP ๐Ÿ‘ค roach-php Organization ๐Ÿ• Updated 2025-10-27 ๐Ÿ“… Created 2021-08-17
171
wordware-ai/twitter

AI Agent for Twitter Personality Analysis

wordware-ai
โญ 1,446 TypeScript ๐Ÿ‘ค wordware-ai Organization ๐Ÿ• Updated 2026-08-02 ๐Ÿ“… Created 2024-06-03
172
mattsse/chromiumoxide

Chrome Devtools Protocol Rust API

mattsse
โญ 1,437 Rust ๐Ÿ‘ค mattsse User ๐Ÿ• Updated 2026-04-03 ๐Ÿ“… Created 2020-11-28
173
Miserlou/SoundScrape

SoundCloud (and Bandcamp and Mixcloud) downloader in Python.

Miserlou
โญ 1,436 Python ๐Ÿ‘ค Miserlou User ๐Ÿ• Updated 2024-06-23 ๐Ÿ“… Created 2013-12-29
174
foxhui/WebAI2API

WebAI2API: Web AI to OpenAI API tool based on Camoufox. Supports LMArena/Gemini etc., multi-window concurrency & account isolation. | Web AI to OpenAI API via Camoufox. Supports LMArena/Gemini and more, multi-window concurrency & account isolation.

foxhui
โญ 1,429 JavaScript ๐Ÿ‘ค foxhui User ๐Ÿ• Updated 2026-07-10 ๐Ÿ“… Created 2025-11-23
175
oxylabs/how-to-scrape-google-finance

Use Web Scraper API to extract data from Google Finance, including stock titles, pricing, and price changes in percentages.

oxylabs
โญ 1,403 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2025-09-23 ๐Ÿ“… Created 2024-09-25
176
feder-cr/invisible_puppeteer

Puppeteer for Python that anti-bot systems canโ€™t detect, so there are no CAPTCHAs: the same API, anti-detection stealth headless Firefox, an undetected fingerprint, bypassing bot detection, web scraping, and browser automation.

feder-cr
โญ 1,403 Python ๐Ÿ‘ค feder-cr User ๐Ÿ• Updated 2026-10-19 ๐Ÿ“… Created 2024-09-01
177
mckaywrigley/clarity-ai

A Perplexity clone.

mckaywrigley
โญ 1,401 TypeScript ๐Ÿ‘ค mckaywrigley User ๐Ÿ• Updated 2024-03-28 ๐Ÿ“… Created 2023-02-19
178
facundoolano/app-store-scraper

scrape data from the itunes app store

facundoolano
โญ 1,396 JavaScript ๐Ÿ‘ค facundoolano User ๐Ÿ• Updated 2025-07-27 ๐Ÿ“… Created 2016-05-31
179
regenrek/deepwiki-mcp

๐Ÿ“– MCP server for fetch deepwiki.com and get latest knowledge in Cursor and other Code Editors

regenrek
โญ 1,393 TypeScript ๐Ÿ‘ค regenrek User ๐Ÿ• Updated 2026-03-20 ๐Ÿ“… Created 2025-04-28
180
sardanioss/httpcloak

Go HTTP client with browser-identical TLS/HTTP2 fingerprinting. Bypass bot detection by perfectly mimicking Chrome, Firefox, and Safari at the cryptographic level (JA3/JA4, Akamai fingerprint, header order). Supports HTTP/1.1, HTTP/2, HTTP/3, sessions, cookies, and proxies.

sardanioss
โญ 1,387 Go ๐Ÿ‘ค sardanioss User ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2025-12-28
181
MiniGlome/Archive.org-Downloader

Python3 script to download archive.org books in PDF format

MiniGlome
โญ 1,385 Python ๐Ÿ‘ค MiniGlome User ๐Ÿ• Updated 2026-09-27 ๐Ÿ“… Created 2021-02-17
182
fugary/calibre-douban

Calibre new douban metadata source plugin. Douban no longer provides book APIs to the public, so it can only use web crawling to obtain data. This is a calibre Douban plugin based on web crawling.

fugary
โญ 1,382 Python ๐Ÿ‘ค fugary User ๐Ÿ• Updated 2025-11-08 ๐Ÿ“… Created 2021-12-12
183
firecrawl/open-scouts

๐Ÿ”ฅ AI-powered web monitoring platform. Create automated scouts that search the web and send email alerts when they find what you're looking for.

firecrawl
โญ 1,372 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2025-11-28
184
sourcegraph/webloop

WebLoop: Scriptable, headless WebKit with a Go API. Like PhantomJS, but for Go.

sourcegraph
โญ 1,366 Go ๐Ÿ‘ค sourcegraph Organization ๐Ÿ• Updated 2024-04-01 ๐Ÿ“… Created 2013-10-27
185
philippta/flyscrape

Flyscrape is a command-line web scraping tool designed for those without advanced programming skills.

philippta
โญ 1,347 Go ๐Ÿ‘ค philippta User ๐Ÿ• Updated 2025-11-24 ๐Ÿ“… Created 2023-08-28
186
Sniper970119/dianping_spider

Dianping Crawler (crawlable on the whole site, solves dynamic font encryption, not OCR). Continuous update

Sniper970119
โญ 1,329 Python ๐Ÿ‘ค Sniper970119 User ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2021-01-22
187
firecrawl/fire-enrich

๐Ÿ”ฅ AI-powered data enrichment tool that transforms emails into rich datasets with company profiles, funding data, tech stacks, and more using Firecrawl and multi-agent AI

firecrawl
โญ 1,308 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2025-06-06
188
AI4Finance-Foundation/FinNLP

Democratizing Internet-scale financial data.

AI4Finance-Foundation
โญ 1,292 Jupyter Notebook ๐Ÿ‘ค AI4Finance-Foundation Organization ๐Ÿ• Updated 2024-07-01 ๐Ÿ“… Created 2023-02-07
189
kkangert/kspider

Kspider is a crawler platform that allows you to define crawler workflows graphically, enabling the implementation of a crawler workflow without code. Kspider is not limited to web crawling, but can also be used for WEB automation testing. More features await your exploration.

kkangert
โญ 1,285 Java ๐Ÿ‘ค kkangert User ๐Ÿ• Updated 2024-08-19 ๐Ÿ“… Created 2024-07-07
190
fastcrw/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

fastcrw
โญ 1,284 Rust ๐Ÿ‘ค fastcrw Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-03-02
191
techwithtim/Price-Tracking-Web-Scraper

An automated price tracker that uses bright data, playwright, react and flask.

techwithtim
โญ 1,284 Python ๐Ÿ‘ค techwithtim User ๐Ÿ• Updated 2024-10-17 ๐Ÿ“… Created 2023-06-23
192
ovobrowsersupport/ovobrowser

Source-level Chromium fingerprinting engine, rated as ordinary Chrome by detection sites. A source-level Chromium fingerprint kernel that passes bot-detection tests. Not JS injectionโ€”C++ patches compiled into the binary.

ovobrowsersupport
โญ 1,277 ๐Ÿ‘ค ovobrowsersupport User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2026-08-23
193
raawaa/jav-scrapy

The hard laborer for batch fetching AV magnet links or covers

raawaa
โญ 1,275 JavaScript ๐Ÿ‘ค raawaa User ๐Ÿ• Updated 2026-10-17 ๐Ÿ“… Created 2015-10-31
194
TaxyAI/browser-extension

Automate your browser with GPT-4

TaxyAI
โญ 1,269 TypeScript ๐Ÿ‘ค TaxyAI Organization ๐Ÿ• Updated 2025-01-15 ๐Ÿ“… Created 2023-03-27
195
chromedp/examples

chromedp code examples.

chromedp
โญ 1,265 Go ๐Ÿ‘ค chromedp Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2017-12-26
196
Decodo/Decodo

HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

Decodo
โญ 1,264 Java ๐Ÿ‘ค Decodo Organization ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2019-01-07
197
MarioVilas/googlesearch

Google search from Python (unofficial).

MarioVilas
โญ 1,249 Python ๐Ÿ‘ค MarioVilas User ๐Ÿ• Updated 2025-11-11 ๐Ÿ“… Created 2012-10-22
198
kairi003/Get-cookies.txt-LOCALLY

Get cookies.txt, NEVER send information outside.

kairi003
โญ 1,230 JavaScript ๐Ÿ‘ค kairi003 User ๐Ÿ• Updated 2025-10-07 ๐Ÿ“… Created 2023-03-01
199
mika-cn/maoxian-web-clipper

A web extension to clip information from web page. Save it to your local machine to avoid information invalidation. Not bored registration, Not charged.

mika-cn
โญ 1,230 JavaScript ๐Ÿ‘ค mika-cn User ๐Ÿ• Updated 2026-08-23 ๐Ÿ“… Created 2018-08-09
200
eooce/Auto-login-netlib

Automatic login to netlib.re website to keep domain alive

eooce
โญ 1,220 JavaScript ๐Ÿ‘ค eooce User ๐Ÿ• Updated 2025-11-08 ๐Ÿ“… Created 2025-11-06