#web-scraper Repositories
Discover 36 active open source GitHub repositories tagged with #web-scraper, accumulating 565,282 total stars (averaging 15,702 stars per project). Leading this topic is firecrawl/firecrawl with 205,758 stars.
36 repositories total
Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu โ one CLI, zero API fees.
๐ท๏ธ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shyโjoin here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Best headless browser for AI agents. Lightweight, Fast, Highly Compatible. Built in Rust
Turn any website into a structured API. Extract, automate, search and monitor the web.
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews, email and more for each place
Professional-grade, local-first browser extension that transforms any webpage into high-fidelity, LLM-friendly Markdown entirely within the browser's runtime. Zero-cloud, privacy-centric, and designed specifically for AI workflows.
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.
A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.
AI Scraper is a powerful scraping tool and scrape agent built to automate data extraction with unmatched precision. Ideal for scalable AI scraping tasks across diverse web sources, this tool simplifies complex scraping operations into efficient, intelligent workflows.
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.
Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.
All-in-one web scraping API for real-time, large-scale data extraction โ proxies, CAPTCHA management, JS rendering, and parsing in a single request returning HTML or structured JSON.
Frequently Asked Questions
What is the most popular #web-scraper repository on GitHub?
The top ranked open source #web-scraper project on GitHub is firecrawl/firecrawl with 205,758 stars. Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
Which programming languages are most used in #web-scraper repositories?
Among the top ranked #web-scraper projects on GitHub, the most common programming languages are Python, TypeScript, Rust, Go.
How is the #web-scraper repository ranking calculated?
We track 36 active open source repositories tagged with #web-scraper (500+ stars), totaling 565,282 stars with an average of 15,702 stars per repository. Rankings update daily from the GitHub API.