Git Stars Git Stars

#web-scraper Repositories

Discover 36 active open source GitHub repositories tagged with #web-scraper, accumulating 565,282 total stars (averaging 15,702 stars per project). Leading this topic is firecrawl/firecrawl with 205,758 stars.

36
Total Repositories
565,282
Total Stars
15,702
Avg Stars/Repo
1
firecrawl/firecrawl

Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

firecrawl
โญ 205,758 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-04-15
2
Panniantong/Agent-Reach

Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu โ€” one CLI, zero API fees.

Panniantong
โญ 113,683 Python ๐Ÿ‘ค Panniantong User ๐Ÿ• Updated 2026-11-08 ๐Ÿ“… Created 2026-02-24
3
D4Vinci/Scrapling

๐Ÿ•ท๏ธ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shyโ€”join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

D4Vinci
โญ 104,760 Python ๐Ÿ‘ค D4Vinci User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2024-10-13
4
ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

ScrapeGraphAI
โญ 32,877 Python ๐Ÿ‘ค ScrapeGraphAI Organization ๐Ÿ• Updated 2026-10-06 ๐Ÿ“… Created 2024-01-27
5
lexmount/moli

Best headless browser for AI agents. Lightweight, Fast, Highly Compatible. Built in Rust

lexmount
โญ 24,190 Rust ๐Ÿ‘ค lexmount Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-08-10
6
getmaxun/maxun

Turn any website into a structured API. Extract, automate, search and monitor the web.

getmaxun
โญ 18,538 TypeScript ๐Ÿ‘ค getmaxun Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-10-23
7
gosom/google-maps-scraper

scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews, email and more for each place

gosom
โญ 8,291 Go ๐Ÿ‘ค gosom User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-04-22
8
BruceDone/awesome-crawler

A collection of awesome web crawler,spider in different languages

BruceDone
โญ 7,051 ๐Ÿ‘ค BruceDone User ๐Ÿ• Updated 2024-06-16 ๐Ÿ“… Created 2016-10-10
9
jaypyles/Scraperr

Self-hosted webscraper.

jaypyles
โญ 4,876 TypeScript ๐Ÿ‘ค jaypyles User ๐Ÿ• Updated 2025-10-12 ๐Ÿ“… Created 2024-07-06
10
AhmadHassan-BTed/JinaClip

Professional-grade, local-first browser extension that transforms any webpage into high-fidelity, LLM-friendly Markdown entirely within the browser's runtime. Zero-cloud, privacy-centric, and designed specifically for AI workflows.

AhmadHassan-BTed
โญ 4,160 JavaScript ๐Ÿ‘ค AhmadHassan-BTed User ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2026-05-16
11
oxylabs/google-ai-mode-scraper

Scrape Google AI Mode responses without blocks on a large scale.

oxylabs
โญ 3,619 Java ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2025-09-08
12
arpit-omprakash/100ProjectsOfCode

A list of practical knowledge-building projects.

arpit-omprakash
โญ 3,548 ๐Ÿ‘ค arpit-omprakash User ๐Ÿ• Updated 2024-04-12 ๐Ÿ“… Created 2020-10-05
13
itsOwen/CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

itsOwen
โญ 3,302 Python ๐Ÿ‘ค itsOwen User ๐Ÿ• Updated 2026-10-12 ๐Ÿ“… Created 2024-08-17
14
php-curl-class/php-curl-class

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

php-curl-class
โญ 3,293 PHP ๐Ÿ‘ค php-curl-class Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2013-08-16
15
oxylabs/how-to-scrape-amazon-product-data

The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.

oxylabs
โญ 3,248 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-09-03
16
spider-rs/spider

Foundational low-latency web data collection in Rust

spider-rs
โญ 2,856 Rust ๐Ÿ‘ค spider-rs Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-01-07
17
lncrawl/lightnovel-crawler

Generate and download e-books from online sources.

lncrawl
โญ 2,717 Python ๐Ÿ‘ค lncrawl Organization ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2018-01-17
18
anaskhan96/soup

Web Scraper in Go, similar to BeautifulSoup

anaskhan96
โญ 2,302 Go ๐Ÿ‘ค anaskhan96 User ๐Ÿ• Updated 2026-10-24 ๐Ÿ“… Created 2017-02-23
19
oxylabs/how-to-scrape-google-scholar

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

oxylabs
โญ 2,095 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-03-07
20
oxylabs/how-to-scrape-amazon-prices

A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

oxylabs
โญ 2,083 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-03-21
21
gildas-lormeau/single-file-cli

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)

gildas-lormeau
โญ 1,702 JavaScript ๐Ÿ‘ค gildas-lormeau User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2022-05-31
22
fastcrw/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

fastcrw
โญ 1,284 Rust ๐Ÿ‘ค fastcrw Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-03-02
23
dondai44423/donsetch

Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.

dondai44423
โญ 1,138 Rust ๐Ÿ‘ค dondai44423 User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-08-07
24
juancarlospaco/faster-than-requests

Faster requests on Python 3

juancarlospaco
โญ 1,131 Nim ๐Ÿ‘ค juancarlospaco User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2018-11-13
25
0x676e67/wreq

An ergonomic, privacy-aware Rust HTTP Client

0x676e67
โญ 1,118 Rust ๐Ÿ‘ค 0x676e67 User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-09-19
26
oxylabs/ai-scraper-py

AI Scraper is a powerful scraping tool and scrape agent built to automate data extraction with unmatched precision. Ideal for scalable AI scraping tasks across diverse web sources, this tool simplifies complex scraping operations into efficient, intelligent workflows.

oxylabs
โญ 1,089 ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-04-02 ๐Ÿ“… Created 2025-09-25
27
postmodern/spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

postmodern
โญ 835 Ruby ๐Ÿ‘ค postmodern User ๐Ÿ• Updated 2026-01-12 ๐Ÿ“… Created 2009-03-08
28
cassidoo/scrapers

A list of scrapers from around the web.

cassidoo
โญ 723 ๐Ÿ‘ค cassidoo User ๐Ÿ• Updated 2025-02-07 ๐Ÿ“… Created 2016-04-18
29
lumpinif/deepcrawl

100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.

lumpinif
โญ 704 TypeScript ๐Ÿ‘ค lumpinif User ๐Ÿ• Updated 2026-08-24 ๐Ÿ“… Created 2025-05-22
30
shaikhsajid1111/social-media-profile-scrapers

Fetch user's data across social media

shaikhsajid1111
โญ 588 Python ๐Ÿ‘ค shaikhsajid1111 User ๐Ÿ• Updated 2026-09-15 ๐Ÿ“… Created 2020-04-26
31
spekulatius/PHPScraper

A universal web-util for PHP.

spekulatius
โญ 588 PHP ๐Ÿ‘ค spekulatius User ๐Ÿ• Updated 2026-09-08 ๐Ÿ“… Created 2020-04-21
32
jaebradley/basketball_reference_web_scraper

NBA Stats API via Basketball Reference

jaebradley
โญ 563 HTML ๐Ÿ‘ค jaebradley User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2015-10-23
33
oxylabs/oxylabs-ai-studio-openclaw

Official Oxylabs AI-Studio Openclaw plugin

oxylabs
โญ 549 TypeScript ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-09-16 ๐Ÿ“… Created 2026-02-26
34
oxylabs/quick-start-guide

Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.

oxylabs
โญ 512 ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-06-19 ๐Ÿ“… Created 2022-03-14
35
AlexMathew/scrapple

A framework for creating semi-automatic web content extractors

AlexMathew
โญ 506 Python ๐Ÿ‘ค AlexMathew User ๐Ÿ• Updated 2026-01-16 ๐Ÿ“… Created 2014-12-13
36
oxylabs/web-scraper-api

All-in-one web scraping API for real-time, large-scale data extraction โ€“ proxies, CAPTCHA management, JS rendering, and parsing in a single request returning HTML or structured JSON.

oxylabs
โญ 503 ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-10-01 ๐Ÿ“… Created 2026-06-05

Frequently Asked Questions

1

What is the most popular #web-scraper repository on GitHub?

The top ranked open source #web-scraper project on GitHub is firecrawl/firecrawl with 205,758 stars. Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

2

Which programming languages are most used in #web-scraper repositories?

Among the top ranked #web-scraper projects on GitHub, the most common programming languages are Python, TypeScript, Rust, Go.

3

How is the #web-scraper repository ranking calculated?

We track 36 active open source repositories tagged with #web-scraper (500+ stars), totaling 565,282 stars with an average of 15,702 stars per repository. Rankings update daily from the GitHub API.