Git Stars Git Stars

#scraper Repositories

Discover 128 active open source GitHub repositories tagged with #scraper, accumulating 792,441 total stars (averaging 6,190 stars per project). Leading this topic is firecrawl/firecrawl with 205,758 stars.

128
Total Repositories
792,441
Total Stars
6,190
Avg Stars/Repo
1
firecrawl/firecrawl

Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

firecrawl
โญ 205,758 TypeScript ๐Ÿ‘ค firecrawl Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2024-04-15
2
unclecode/crawl4ai

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

unclecode
โญ 87,627 Python ๐Ÿ‘ค unclecode User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-05-09
3
huginn/huginn

Create agents that monitor and act on your behalf. Your agents are standing by!

huginn
โญ 50,601 Ruby ๐Ÿ‘ค huginn Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2013-03-10
4
NaiboWang/EasySpider

A visual no-code web crawler/spider: A browser automation, data collection, and web scraping software that allows for the design and execution of crawling tasks graphically without writing code. Alias: ServiceWrapper, an intelligent service encapsulation system for web applications.

NaiboWang
โญ 44,801 JavaScript ๐Ÿ‘ค NaiboWang User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2020-07-18
5
iawia002/lux

๐Ÿ‘พ Fast and simple video download library and CLI tool written in Go

iawia002
โญ 31,854 Go ๐Ÿ‘ค iawia002 User ๐Ÿ• Updated 2026-03-29 ๐Ÿ“… Created 2018-02-24
6
cheeriojs/cheerio

The fast, flexible, and elegant library for parsing and manipulating HTML and XML.

cheeriojs
โญ 30,552 TypeScript ๐Ÿ‘ค cheeriojs Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2011-10-09
7
apify/crawlee

Crawleeโ€”A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

apify
โญ 26,510 TypeScript ๐Ÿ‘ค apify Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2016-08-26
8
gocolly/colly

Elegant Scraper and Crawler Framework for Golang

gocolly
โญ 25,608 Go ๐Ÿ‘ค gocolly Organization ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2017-09-29
9
Evil0ctal/Douyin_TikTok_Download_API

๐Ÿš€ Self-hosted data collection and watermark-free video download API for Douyin, TikTok, and Bilibili. Supports AI agent integration via MCP and offline speech-to-text. One-click Docker deployment. | Self-hosted TikTok, Douyin & Bilibili scraper and watermark-free video downloader โ€” asynchronous REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one `docker compose up`.

Evil0ctal
โญ 21,507 Python ๐Ÿ‘ค Evil0ctal User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2021-11-07
10
getmaxun/maxun

Turn any website into a structured API. Extract, automate, search and monitor the web.

getmaxun
โญ 18,538 TypeScript ๐Ÿ‘ค getmaxun Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-10-23
11
codelucas/newspaper

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

codelucas
โญ 15,192 Python ๐Ÿ‘ค codelucas User ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2013-11-25
12
NopeCHALLC/nopecha-extension

Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.

NopeCHALLC
โญ 11,140 ๐Ÿ‘ค NopeCHALLC User ๐Ÿ• Updated 2026-10-02 ๐Ÿ“… Created 2023-03-23
13
guyueyingmu/avbook

AV Movie Management System, avmoo, javbus, javlibrary Crawler, Online AV Film Library, AV Magnet Link Database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database

guyueyingmu
โญ 9,894 PHP ๐Ÿ‘ค guyueyingmu User ๐Ÿ• Updated 2024-06-01 ๐Ÿ“… Created 2017-02-19
14
apify/crawlee-python

Crawleeโ€”A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.

apify
โญ 9,679 Python ๐Ÿ‘ค apify Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-01-10
15
alirezamika/autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

alirezamika
โญ 8,102 Python ๐Ÿ‘ค alirezamika User ๐Ÿ• Updated 2026-07-29 ๐Ÿ“… Created 2020-08-31
16
go-rod/rod

A Chrome DevTools Protocol driver for web automation and scraping.

go-rod
โญ 7,160 Go ๐Ÿ‘ค go-rod Organization ๐Ÿ• Updated 2026-08-11 ๐Ÿ“… Created 2020-01-21
17
BruceDone/awesome-crawler

A collection of awesome web crawler,spider in different languages

BruceDone
โญ 7,051 ๐Ÿ‘ค BruceDone User ๐Ÿ• Updated 2024-06-16 ๐Ÿ“… Created 2016-10-10
18
mishushakov/llm-scraper

Turn any webpage into structured data using LLMs

mishushakov
โญ 6,956 TypeScript ๐Ÿ‘ค mishushakov User ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2024-04-20
19
AhmadIbrahiim/Website-downloader

๐Ÿ’ก Download the complete source code of any website (including all assets). [ Javascripts, Stylesheets, Images ] using Node.js

AhmadIbrahiim
โญ 6,206 HTML ๐Ÿ‘ค AhmadIbrahiim User ๐Ÿ• Updated 2026-08-12 ๐Ÿ“… Created 2019-10-26
20
niespodd/browser-fingerprinting

Analysis of Bot Protection systems with available countermeasures ๐Ÿšฟ. How to defeat anti-bot system ๐Ÿ‘ป and get around browser fingerprinting scripts ๐Ÿ•ต๏ธโ€โ™‚๏ธ when scraping the web?

niespodd
โญ 5,171 JavaScript ๐Ÿ‘ค niespodd User ๐Ÿ• Updated 2026-07-27 ๐Ÿ“… Created 2021-01-23
21
d60/twikit

Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot

d60
โญ 4,785 Python ๐Ÿ‘ค d60 User ๐Ÿ• Updated 2026-03-10 ๐Ÿ“… Created 2024-01-20
22
fent/node-ytdl-core

YouTube video downloader in javascript.

fent
โญ 4,732 JavaScript ๐Ÿ‘ค fent User ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2014-06-29
23
joeyism/linkedin_scraper

A library that scrapes Linkedin for user data

joeyism
โญ 4,699 Python ๐Ÿ‘ค joeyism User ๐Ÿ• Updated 2026-04-10 ๐Ÿ“… Created 2017-10-27
24
bjesus/pipet

Swiss-army tool for scraping and extracting data from online assets, made for hackers

bjesus
โญ 4,610 Go ๐Ÿ‘ค bjesus User ๐Ÿ• Updated 2024-10-12 ๐Ÿ“… Created 2024-08-31
25
myreader-io/myGPTReader

A community-driven way to read and chat with AI bots - powered by chatGPT.

myreader-io
โญ 4,416 Python ๐Ÿ‘ค myreader-io Organization ๐Ÿ• Updated 2026-02-05 ๐Ÿ“… Created 2023-03-08
26
UltimaHoarder/UltimaScraper

A data scraper currently focused on adult content (OnlyFans, etc.) โ€” updated regularly

UltimaHoarder
โญ 4,302 TypeScript ๐Ÿ‘ค UltimaHoarder User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-06-24
27
IonicaBizau/scrape-it

๐Ÿ”ฎ A Node.js scraper for humans.

IonicaBizau
โญ 4,069 JavaScript ๐Ÿ‘ค IonicaBizau User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2016-04-28
28
edoardottt/cariddi

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

edoardottt
โญ 3,817 Go ๐Ÿ‘ค edoardottt User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2021-04-27
29
Anakin-Inc/anakin

Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up

Anakin-Inc
โญ 3,764 Go ๐Ÿ‘ค Anakin-Inc Organization ๐Ÿ• Updated 2026-08-18 ๐Ÿ“… Created 2026-03-17
30
sqzw-x/mdcx

Movie metadata scraper

sqzw-x
โญ 3,727 Python ๐Ÿ‘ค sqzw-x User ๐Ÿ• Updated 2026-08-23 ๐Ÿ“… Created 2023-11-06
31
JavScraper/Emby.Plugins.JavScraper

An Emby/Jellyfin Japanese movie scraper plugin that fetches movie information from certain websites.

JavScraper
โญ 3,709 C# ๐Ÿ‘ค JavScraper User ๐Ÿ• Updated 2024-07-08 ๐Ÿ“… Created 2020-01-20
32
itsOwen/CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

itsOwen
โญ 3,302 Python ๐Ÿ‘ค itsOwen User ๐Ÿ• Updated 2026-10-12 ๐Ÿ“… Created 2024-08-17
33
aapatre/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE

Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!

aapatre
โญ 3,289 Python ๐Ÿ‘ค aapatre User ๐Ÿ• Updated 2026-04-28 ๐Ÿ“… Created 2020-09-12
34
facundoolano/google-play-scraper

Node.js scraper to get data from Google Play

facundoolano
โญ 3,005 JavaScript ๐Ÿ‘ค facundoolano User ๐Ÿ• Updated 2026-10-17 ๐Ÿ“… Created 2015-04-07
35
vladkens/twscrape

Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.

vladkens
โญ 2,980 Python ๐Ÿ‘ค vladkens User ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2023-05-05
36
oxylabs/how-to-scrape-google-trends

Learn step-by-step how to scrape Google Trends data and make a result comparison using Python and Oxylabs SERP API. Extract keywords, their popularity, breakdown by region, related queries, and more.

oxylabs
โญ 2,907 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-08-06
37
geziyor/geziyor

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

geziyor
โญ 2,781 Go ๐Ÿ‘ค geziyor Organization ๐Ÿ• Updated 2026-07-02 ๐Ÿ“… Created 2019-06-06
38
microlinkhq/metascraper

Scrape metadata from any URL using Open Graph, JSON-LD, HTML meta tags, and smart fallbacks.

microlinkhq
โญ 2,754 HTML ๐Ÿ‘ค microlinkhq Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2016-05-25
39
jae-jae/QueryList

:spider: The progressive PHP crawler framework! ไผ˜้›…็š„ๆธ่ฟ›ๅผPHP้‡‡้›†ๆก†ๆžถใ€‚

jae-jae
โญ 2,689 PHP ๐Ÿ‘ค jae-jae User ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2015-12-20
40
Serene-Arc/bulk-downloader-for-reddit

Downloads and archives content from reddit

Serene-Arc
โญ 2,631 Python ๐Ÿ‘ค Serene-Arc User ๐Ÿ• Updated 2026-04-12 ๐Ÿ“… Created 2018-06-14
41
zerodytrash/TikTok-Live-Connector

TikTok LIVE API for Node.js: An unofficial library to receive live stream events (comments, gifts, etc.) in realtime from TikTok LIVE.

zerodytrash
โญ 2,332 TypeScript ๐Ÿ‘ค zerodytrash User ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2022-02-04
42
watercrawl/WaterCrawl

Transform Web Content into LLM-Ready Data

watercrawl
โญ 2,244 TypeScript ๐Ÿ‘ค watercrawl Organization ๐Ÿ• Updated 2026-08-17 ๐Ÿ“… Created 2024-12-15
43
cyberagiinc/DevDocs

Completely free, private, UI based Tech Documentation MCP server. Designed for coders and software developers in mind. Easily integrate into Cursor, Windsurf, Cline, Roo Code, Claude Desktop App

cyberagiinc
โญ 2,106 TypeScript ๐Ÿ‘ค cyberagiinc Organization ๐Ÿ• Updated 2026-02-04 ๐Ÿ“… Created 2025-01-15
44
PaulMcInnis/JobFunnel

Scrape job websites into a single spreadsheet with no duplicates.

PaulMcInnis
โญ 2,103 Python ๐Ÿ‘ค PaulMcInnis User ๐Ÿ• Updated 2025-12-04 ๐Ÿ“… Created 2017-08-25
45
TeamNewPipe/NewPipeExtractor

NewPipe's core library for extracting data from streaming sites

TeamNewPipe
โญ 2,070 Java ๐Ÿ‘ค TeamNewPipe Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2017-03-01
46
KatrielMoses/MailAccess

Free email OSINT tool, 5000+ platforms, identity clustering, breach detection. No API keys required. pip install mailaccess

KatrielMoses
โญ 1,824 Python ๐Ÿ‘ค KatrielMoses User ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2026-05-18
47
eatmoreduck/boss-zhipin-scraper

BOSS Zhipin scraper / BOSS Zhipin job data crawler, based on Chrome CDP protocol to reuse real login sessions, bypass font anti-scraping, and output plaintext salary JSON/CSV + salary-skill analysis.

eatmoreduck
โญ 1,781 Python ๐Ÿ‘ค eatmoreduck User ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2026-06-11
48
website-scraper/node-website-scraper

Download website to local directory (including all css, images, js, etc.)

website-scraper
โญ 1,773 JavaScript ๐Ÿ‘ค website-scraper Organization ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2014-09-04
49
oxylabs/how-to-scrape-google-jobs

Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabsโ€™ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs

oxylabs
โญ 1,761 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-02-08
50
oxylabs/google-play-scraper

Google Play Scraper for collecting public details about applications, games, movies & TV shows, books, and more.

oxylabs
โญ 1,746 Python ๐Ÿ‘ค oxylabs Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2023-09-01

Frequently Asked Questions

1

What is the most popular #scraper repository on GitHub?

The top ranked open source #scraper project on GitHub is firecrawl/firecrawl with 205,758 stars. Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

2

Which programming languages are most used in #scraper repositories?

Among the top ranked #scraper projects on GitHub, the most common programming languages are Python, TypeScript, Go, JavaScript.

3

How is the #scraper repository ranking calculated?

We track 128 active open source repositories tagged with #scraper (500+ stars), totaling 792,441 stars with an average of 6,190 stars per repository. Rankings update daily from the GitHub API.