#scraper Repositories
Discover 128 active open source GitHub repositories tagged with #scraper, accumulating 792,441 total stars (averaging 6,190 stars per project). Leading this topic is firecrawl/firecrawl with 205,758 stars.
128 repositories total
Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
A visual no-code web crawler/spider: A browser automation, data collection, and web scraping software that allows for the design and execution of crawling tasks graphically without writing code. Alias: ServiceWrapper, an intelligent service encapsulation system for web applications.
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
CrawleeโA web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
๐ Self-hosted data collection and watermark-free video download API for Douyin, TikTok, and Bilibili. Supports AI agent integration via MCP and offline speech-to-text. One-click Docker deployment. | Self-hosted TikTok, Douyin & Bilibili scraper and watermark-free video downloader โ asynchronous REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one `docker compose up`.
Turn any website into a structured API. Extract, automate, search and monitor the web.
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.
AV Movie Management System, avmoo, javbus, javlibrary Crawler, Online AV Film Library, AV Magnet Link Database, Japanese Adult Video Library, Adult Video Magnet Links - Japanese Adult Video Database
CrawleeโA web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
๐ก Download the complete source code of any website (including all assets). [ Javascripts, Stylesheets, Images ] using Node.js
Analysis of Bot Protection systems with available countermeasures ๐ฟ. How to defeat anti-bot system ๐ป and get around browser fingerprinting scripts ๐ต๏ธโโ๏ธ when scraping the web?
Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot
Swiss-army tool for scraping and extracting data from online assets, made for hackers
A community-driven way to read and chat with AI bots - powered by chatGPT.
A data scraper currently focused on adult content (OnlyFans, etc.) โ updated regularly
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
An Emby/Jellyfin Japanese movie scraper plugin that fetches movie information from certain websites.
Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!
Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.
Learn step-by-step how to scrape Google Trends data and make a result comparison using Python and Oxylabs SERP API. Extract keywords, their popularity, breakdown by region, related queries, and more.
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
Scrape metadata from any URL using Open Graph, JSON-LD, HTML meta tags, and smart fallbacks.
:spider: The progressive PHP crawler framework! ไผ้ ็ๆธ่ฟๅผPHP้้ๆกๆถใ
TikTok LIVE API for Node.js: An unofficial library to receive live stream events (comments, gifts, etc.) in realtime from TikTok LIVE.
Completely free, private, UI based Tech Documentation MCP server. Designed for coders and software developers in mind. Easily integrate into Cursor, Windsurf, Cline, Roo Code, Claude Desktop App
Free email OSINT tool, 5000+ platforms, identity clustering, breach detection. No API keys required. pip install mailaccess
BOSS Zhipin scraper / BOSS Zhipin job data crawler, based on Chrome CDP protocol to reuse real login sessions, bypass font anti-scraping, and output plaintext salary JSON/CSV + salary-skill analysis.
Download website to local directory (including all css, images, js, etc.)
Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabsโ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs
Google Play Scraper for collecting public details about applications, games, movies & TV shows, books, and more.
Frequently Asked Questions
What is the most popular #scraper repository on GitHub?
The top ranked open source #scraper project on GitHub is firecrawl/firecrawl with 205,758 stars. Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
Which programming languages are most used in #scraper repositories?
Among the top ranked #scraper projects on GitHub, the most common programming languages are Python, TypeScript, Go, JavaScript.
How is the #scraper repository ranking calculated?
We track 128 active open source repositories tagged with #scraper (500+ stars), totaling 792,441 stars with an average of 6,190 stars per repository. Rankings update daily from the GitHub API.