Git Stars Git Stars

#web-scraping Repositories

381
Total Repositories
2.1M
Total Stars
5,636
Avg Stars/Repo
301
z0m31en7/Uscrapper

Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.

z0m31en7
โญ 740 Python ๐Ÿ‘ค z0m31en7 User ๐Ÿ• Updated 2024-11-24 ๐Ÿ“… Created 2023-05-31
302
djadmin/medium-cli

Medium for Hackers - :computer: A CLI for Medium Stories.

djadmin
โญ 735 JavaScript ๐Ÿ‘ค djadmin User ๐Ÿ• Updated 2026-02-13 ๐Ÿ“… Created 2016-01-21
303
drawrowfly/amazon-product-api

Amazon Scraper. Scrape products from the amazon search result or reviews from the specific product

drawrowfly
โญ 734 JavaScript ๐Ÿ‘ค drawrowfly User ๐Ÿ• Updated 2024-07-05 ๐Ÿ“… Created 2019-10-06
304
ScrapeGraphAI/scrapecraft

๐Ÿค– AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.

ScrapeGraphAI
โญ 731 Python ๐Ÿ‘ค ScrapeGraphAI Organization ๐Ÿ• Updated 2025-12-26 ๐Ÿ“… Created 2025-07-01
305
rubycdp/vessel

Fast high-level web crawling Ruby framework

rubycdp
โญ 725 Ruby ๐Ÿ‘ค rubycdp Organization ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2019-09-17
306
MorganZhang100/zhihu-spider

A web spider for zhihu.com

MorganZhang100
โญ 725 Python ๐Ÿ‘ค MorganZhang100 User ๐Ÿ• Updated 2024-01-17 ๐Ÿ“… Created 2015-02-22
307
lzjun567/python_scripts

Some python-related demo code

lzjun567
โญ 715 HTML ๐Ÿ‘ค lzjun567 User ๐Ÿ• Updated 2024-07-21 ๐Ÿ“… Created 2017-02-07
308
lhccong/fish-island-backend

Moyu Island ๐ŸŽฃ Backend Based on Web Scraping โž• Netty โž• SpringBoot โž•Redisโž• MySQL Open Source ๐ŸŒŸ All-in-One Moyu Website

lhccong
โญ 710 Java ๐Ÿ‘ค lhccong User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-02-20
309
kovidgoyal/html5-parser

Fast C based HTML 5 parsing for python

kovidgoyal
โญ 710 C ๐Ÿ‘ค kovidgoyal User ๐Ÿ• Updated 2026-09-20 ๐Ÿ“… Created 2017-06-03
310
lumpinif/deepcrawl

100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.

lumpinif
โญ 704 TypeScript ๐Ÿ‘ค lumpinif User ๐Ÿ• Updated 2026-08-24 ๐Ÿ“… Created 2025-05-22
311
maaaaz/webscreenshot

A simple script to screenshot a list of websites

maaaaz
โญ 696 Python ๐Ÿ‘ค maaaaz User ๐Ÿ• Updated 2025-10-19 ๐Ÿ“… Created 2014-01-12
312
only-cli/oc

Turn any website into a compact CLI tailored for AI agents. Browse the web in hundreds of tokens, not tens of thousands.

only-cli
โญ 694 JavaScript ๐Ÿ‘ค only-cli User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2026-08-18
313
firecrawl/cli

Firecrawl CLI and agent skills: search, scrape, and interact with the web from your terminal or any coding agent, plus parse, crawl, map, Agent and Monitor. Firecrawl is the context API for AI agents.

firecrawl
โญ 694 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-01-07
314
karpathy/hn-time-capsule

Analyzing Hacker News discussions from a decade ago in hindsight with LLMs

karpathy
โญ 693 Python ๐Ÿ‘ค karpathy User ๐Ÿ• Updated 2025-12-10 ๐Ÿ“… Created 2025-12-10
315
xiaosimao/IP_POOL

Free IP Proxy Pool

xiaosimao
โญ 686 Python ๐Ÿ‘ค xiaosimao User ๐Ÿ• Updated 2025-09-15 ๐Ÿ“… Created 2017-09-22
316
perkeep/gphotos-cdp

This program uses the Chrome DevTools Protocol to drive a Chrome session that downloads your photos stored in Google Photos.

perkeep
โญ 683 Go ๐Ÿ‘ค perkeep Organization ๐Ÿ• Updated 2024-06-30 ๐Ÿ“… Created 2019-08-08
317
yixiu001/Free-SS-NODES

Get free ss nodes

yixiu001
โญ 681 ๐Ÿ‘ค yixiu001 User ๐Ÿ• Updated 2024-11-11 ๐Ÿ“… Created 2024-11-11
318
everyinfra/everyinfra-agent-plugins

EveryInfra MCP server & agent plugin: scraper API for TikTok, Instagram, Amazon, YouTube, Xiaohongshu & Douyin. 612 data APIs across 106 platforms, from $0.56/1K calls. For Claude Code, Cursor, Codex.

everyinfra
โญ 672 Python ๐Ÿ‘ค everyinfra Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-09-01
319
firecrawl/open-researcher

๐Ÿ”ฅ Visual AI research assistant that displays real-time thinking, provides split-view analysis, and automatic citations using Claude and Firecrawl

firecrawl
โญ 667 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2025-07-02 ๐Ÿ“… Created 2025-06-27
320
wimglenn/advent-of-code-data

Get your Advent of Code data with a single import statement

wimglenn
โญ 666 Python ๐Ÿ‘ค wimglenn User ๐Ÿ• Updated 2026-08-01 ๐Ÿ“… Created 2016-12-03
321
oxylabs/Agent-browser

Execute complex automation scripts via remote cloud sessions, featuring integrated residential proxies, automated CAPTCHA solving, and native JavaScript rendering for the toughest dynamic websites.

oxylabs
โญ 663 ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2026-04-17
322
adamschwartz/web.scraper.workers.dev

Scrape websites for text by CSS selector.

adamschwartz
โญ 658 JavaScript ๐Ÿ‘ค adamschwartz User ๐Ÿ• Updated 2024-02-14 ๐Ÿ“… Created 2019-10-10
323
asmcos/requests

A golang HTTP client library. Salute to python requests.

asmcos
โญ 656 Go ๐Ÿ‘ค asmcos User ๐Ÿ• Updated 2026-10-09 ๐Ÿ“… Created 2018-08-27
324
yanzhou/CnkiSpider

China National Knowledge Infrastructure Crawler

yanzhou
โญ 653 Python ๐Ÿ‘ค yanzhou User ๐Ÿ• Updated 2025-03-08 ๐Ÿ“… Created 2014-11-05
325
11firefly11/fir-proxy

A graphical proxy pool program with rotating proxies

11firefly11
โญ 650 Python ๐Ÿ‘ค 11firefly11 User ๐Ÿ• Updated 2025-12-04 ๐Ÿ“… Created 2025-07-22
326
KNKPA/KNKPAnime

KNKPAnime: An anime watching software that supports multi-episode sources and bullet comments.

KNKPA
โญ 648 Dart ๐Ÿ‘ค KNKPA Organization ๐Ÿ• Updated 2025-02-01 ๐Ÿ“… Created 2024-04-20
327
Threezh1/SiteCopy

sitecopy is a tool that facilitates personal website backup and network data collection

Threezh1
โญ 647 Python ๐Ÿ‘ค Threezh1 User ๐Ÿ• Updated 2024-01-21 ๐Ÿ“… Created 2019-12-12
328
ScrapingBee/google-hotels-api

Automate scraping hotel data including pricing, availability, ratings, and booking providers with a Google Hotels API integration.

ScrapingBee
โญ 645 ๐Ÿ‘ค ScrapingBee Organization ๐Ÿ• Updated 2026-04-08 ๐Ÿ“… Created 2026-04-08
โญ 640 CSS ๐Ÿ‘ค dreamsofcode-io Organization ๐Ÿ• Updated 2024-08-04 ๐Ÿ“… Created 2024-07-26
330
cfinke/EpsteOut

See which of your LinkedIn connections appear in the Epstein files.

cfinke
โญ 638 Python ๐Ÿ‘ค cfinke User ๐Ÿ• Updated 2026-10-17 ๐Ÿ“… Created 2026-02-04
331
lzjqsdd/NewsSpider

Crawl news from Toutiao, NetEase, Tencent, etc., and build a simple search engine

lzjqsdd
โญ 638 Python ๐Ÿ‘ค lzjqsdd User ๐Ÿ• Updated 2024-05-14 ๐Ÿ“… Created 2016-04-19
332
scrapfly/Antibot-Detector

Real-time detection of anti-bot systems, captchas & fingerprinting techniques. Identifies Cloudflare, Akamai, DataDome, reCAPTCHA, hCaptcha, Shape Security & more with confidence scoring and advanced capture tools.

scrapfly
โญ 636 JavaScript ๐Ÿ‘ค scrapfly Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2025-09-03
333
ohyicong/Google-Image-Scraper

A library to scrap google images

ohyicong
โญ 636 Python ๐Ÿ‘ค ohyicong User ๐Ÿ• Updated 2025-02-16 ๐Ÿ“… Created 2020-07-18
335
deedy5/primp

HTTP client that can impersonate web browsers

deedy5
โญ 634 Rust ๐Ÿ‘ค deedy5 User ๐Ÿ• Updated 2026-09-13 ๐Ÿ“… Created 2024-04-12
336
achuthasubhash/Complete-Life-Cycle-of-a-Data-Science-Project

Complete-Life-Cycle-of-a-Data-Science-Project

achuthasubhash
โญ 631 ๐Ÿ‘ค achuthasubhash User ๐Ÿ• Updated 2024-06-07 ๐Ÿ“… Created 2020-09-02
337
eliemichel/LilySurfaceScraper

Import shaders and environments in Blender from a single URL

eliemichel
โญ 625 Cython ๐Ÿ‘ค eliemichel User ๐Ÿ• Updated 2026-06-19 ๐Ÿ“… Created 2019-04-28
338
super-l/superl-url

A program that collects content from URLs in search engine results based on keywords. It can automatically gather real addresses and titles of related websites from multiple search engines, save them to files, and automatically remove duplicate URLs. Additionally, it allows custom ignoring of multiple domains.

super-l
โญ 621 Python ๐Ÿ‘ค super-l User ๐Ÿ• Updated 2025-04-12 ๐Ÿ“… Created 2017-08-03
339
Lofter1/anyflip-downloader

Download anyflip books as PDF

Lofter1
โญ 619 Go ๐Ÿ‘ค Lofter1 User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2023-02-05
340
chenluda/zhihu-download

Convert Zhihu column articles to Markdown files and save them locally

chenluda
โญ 613 Python ๐Ÿ‘ค chenluda User ๐Ÿ• Updated 2025-03-21 ๐Ÿ“… Created 2023-05-12
341
liuda30201/AI-Coding

AI Coding Benefits Guide: A curated directory of Claude Code, Codex, and Cursor relay services and community-run sites, covering free quotas, sign-up bonuses, check-in events, and review records, with a visual dashboard and Python data collection tools.

liuda30201
โญ 611 HTML ๐Ÿ‘ค liuda30201 User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-10-15
342
alchaincyf/huashu-chrome

Let any AI agent control your own Chromeโ€”with all your login sessions intact. MCP + Chrome extension, compatible with Claude Code / Codex / Cursor / Gemini CLI.

alchaincyf
โญ 609 JavaScript ๐Ÿ‘ค alchaincyf User ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2026-08-26
343
testusyd/douyin_downloader

Douyin lucky bag gacha machine, Douyin lucky bag grabbing tool, written by myself in VC++ in my spare time, with some other small functions, Douyin watermark-free video downloader, Douyin live stream recording downloader, Douyin batch unfollow tool

testusyd
โญ 605 ๐Ÿ‘ค testusyd User ๐Ÿ• Updated 2026-03-08 ๐Ÿ“… Created 2025-05-22
344
ThinkerWen/TicketMonitoring

Barley, Cat's Eye, Fenwan Island, Ticket Planet, Concert re-supply ticket monitoring

ThinkerWen
โญ 605 Python ๐Ÿ‘ค ThinkerWen User ๐Ÿ• Updated 2025-01-24 ๐Ÿ“… Created 2023-10-22
345
lunnar211/HackFacebokpass

Hacking facebook by termux.This tool is cooded by TecnicalDipesh. You can dump any facebok id and password.

lunnar211
โญ 603 Python ๐Ÿ‘ค lunnar211 User ๐Ÿ• Updated 2026-03-26 ๐Ÿ“… Created 2019-11-11
346
web-agent-master/google-search

A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.

web-agent-master
โญ 598 TypeScript ๐Ÿ‘ค web-agent-master Organization ๐Ÿ• Updated 2025-04-06 ๐Ÿ“… Created 2025-03-04
347
jiuzhi-1/meituan-shenquan

Meituan Daily God Coupons Regional Activity Automation Script

jiuzhi-1
โญ 595 Python ๐Ÿ‘ค jiuzhi-1 User ๐Ÿ• Updated 2024-05-08 ๐Ÿ“… Created 2021-10-15
348
ytdl-org/ytdl-nightly

Nightly builds for youtube-dl.

ytdl-org
โญ 593 Python ๐Ÿ‘ค ytdl-org Organization ๐Ÿ• Updated 2026-11-06 ๐Ÿ“… Created 2023-07-02
349
shaikhsajid1111/social-media-profile-scrapers

Fetch user's data across social media

shaikhsajid1111
โญ 588 Python ๐Ÿ‘ค shaikhsajid1111 User ๐Ÿ• Updated 2026-09-15 ๐Ÿ“… Created 2020-04-26
350
spekulatius/PHPScraper

A universal web-util for PHP.

spekulatius
โญ 588 PHP ๐Ÿ‘ค spekulatius User ๐Ÿ• Updated 2026-09-08 ๐Ÿ“… Created 2020-04-21