Git Stars Git Stars

#webcrawler Repositories

Discover 6 active open source GitHub repositories tagged with #webcrawler, accumulating 20,820 total stars (averaging 3,470 stars per project). Leading this topic is crawlab-team/crawlab with 12,301 stars.

6
Total Repositories
20,820
Total Stars
3,470
Avg Stars/Repo
Programming Languages: Python Go PHP
1
crawlab-team/crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks.

crawlab-team
โญ 12,301 Go ๐Ÿ‘ค crawlab-team Organization ๐Ÿ• Updated 2026-02-10 ๐Ÿ“… Created 2019-02-10
2
kingname/GeneralNewsExtractor

Universal Extractor for News Webpage Body Beta Version.

kingname
โญ 3,807 Python ๐Ÿ‘ค kingname User ๐Ÿ• Updated 2026-04-21 ๐Ÿ“… Created 2019-09-08
3
zorlan/skycaiji

Blue Sky Collector is an open-source and free crawler system that only requires clicking and editing rules to collect data. It can run locally, on a virtual host, or on a cloud server, and can crawl almost all types of web pages. It seamlessly integrates with various CMS website building programs, publishes data in real-time without login, and is fully automated without human intervention! It is a completely cross-platform cloud crawler system in web big data collection software.

zorlan
โญ 2,098 PHP ๐Ÿ‘ค zorlan User ๐Ÿ• Updated 2026-10-07 ๐Ÿ“… Created 2018-02-26
4
spidersuite/SpiderSuite

SpiderSuite (web security crawler) releases, wiki and roadmap

spidersuite
โญ 990 ๐Ÿ‘ค spidersuite Organization ๐Ÿ• Updated 2026-08-13 ๐Ÿ“… Created 2023-04-05
5
scrapinghub/scrapyrt

HTTP API for Scrapy spiders

scrapinghub
โญ 884 Python ๐Ÿ‘ค scrapinghub Organization ๐Ÿ• Updated 2026-10-13 ๐Ÿ“… Created 2015-01-06
6
z0m31en7/Uscrapper

Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.

z0m31en7
โญ 740 Python ๐Ÿ‘ค z0m31en7 User ๐Ÿ• Updated 2024-11-24 ๐Ÿ“… Created 2023-05-31

Frequently Asked Questions

1

What is the most popular #webcrawler repository on GitHub?

The top ranked open source #webcrawler project on GitHub is crawlab-team/crawlab with 12,301 stars. Distributed web crawler admin platform for spiders management regardless of languages and frameworks.

2

Which programming languages are most used in #webcrawler repositories?

Among the top ranked #webcrawler projects on GitHub, the most common programming languages are Python, Go, PHP.

3

How is the #webcrawler repository ranking calculated?

We track 6 active open source repositories tagged with #webcrawler (500+ stars), totaling 20,820 stars with an average of 3,470 stars per repository. Rankings update daily from the GitHub API.