Git Stars Git Stars

#webcrawling Repositories

Discover 2 active open source GitHub repositories tagged with #webcrawling, accumulating 4,236 total stars (averaging 2,118 stars per project). Leading this topic is internetarchive/heritrix3 with 3,354 stars.

2
Total Repositories
4,236
Total Stars
2,118
Avg Stars/Repo
Programming Languages: Java Python

2 repositories total

1
internetarchive/heritrix3

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

internetarchive
⭐ 3,354 Java 👤 internetarchive Organization 🕐 Updated 2026-11-20 📅 Created 2011-10-21
2
scrapinghub/scrapyrt

HTTP API for Scrapy spiders

scrapinghub
⭐ 884 Python 👤 scrapinghub Organization 🕐 Updated 2026-10-13 📅 Created 2015-01-06

Frequently Asked Questions

1

What is the most popular #webcrawling repository on GitHub?

The top ranked open source #webcrawling project on GitHub is internetarchive/heritrix3 with 3,354 stars. Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

2

Which programming languages are most used in #webcrawling repositories?

Among the top ranked #webcrawling projects on GitHub, the most common programming languages are Java, Python.

3

How is the #webcrawling repository ranking calculated?

We track 2 active open source repositories tagged with #webcrawling (500+ stars), totaling 4,236 stars with an average of 2,118 stars per repository. Rankings update daily from the GitHub API.