Git Stars Git Stars

#hadoop Repositories

Discover 43 active open source GitHub repositories tagged with #hadoop, accumulating 238,277 total stars (averaging 5,541 stars per project). Leading this topic is donnemartin/data-science-ipython-notebooks with 28,776 stars.

43
Total Repositories
238,277
Total Stars
5,541
Avg Stars/Repo
1
donnemartin/data-science-ipython-notebooks

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

donnemartin
โญ 28,776 Python ๐Ÿ‘ค donnemartin User ๐Ÿ• Updated 2024-03-20 ๐Ÿ“… Created 2015-01-23
2
spotify/luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

spotify
โญ 18,793 Python ๐Ÿ‘ค spotify Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2012-09-20
3
APIJSON/APIJSON

๐Ÿ† Real-Time no-code, powerful and secure ORM ๐Ÿš€ providing APIs and Docs without coding by Backend, and Frontend(Client) can customize response JSONs ๐Ÿ† Real-time zero-code fully functional highly secure ORM library ๐Ÿš€ Backend APIs and documentation are zero-code, while Frontend(client) can customize returned JSON data and structure.

APIJSON
โญ 18,395 Java ๐Ÿ‘ค APIJSON Organization ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2016-11-21
4
heibaiying/BigData-Notes

Big Data Introduction Guide :star:

heibaiying
โญ 16,818 Java ๐Ÿ‘ค heibaiying User ๐Ÿ• Updated 2024-01-05 ๐Ÿ“… Created 2019-03-10
5
prestodb/presto

The official home of the Presto distributed SQL query engine for big data

prestodb
โญ 16,777 Java ๐Ÿ‘ค prestodb Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2012-08-09
6
apache/hadoop

Apache Hadoop

apache
โญ 15,719 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2014-08-28
7
deeplearning4j/deeplearning4j

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...

deeplearning4j
โญ 14,283 Java ๐Ÿ‘ค deeplearning4j Organization ๐Ÿ• Updated 2026-10-05 ๐Ÿ“… Created 2013-11-27
8
trinodb/trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

trinodb
โญ 13,532 Java ๐Ÿ‘ค trinodb Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-01-19
9
HariSekhon/DevOps-Bash-tools

1200+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..

HariSekhon
โญ 8,439 Shell ๐Ÿ‘ค HariSekhon User ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2016-01-17
10
linkedin/school-of-sre

At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.

linkedin
โญ 8,154 HTML ๐Ÿ‘ค linkedin Organization ๐Ÿ• Updated 2025-12-25 ๐Ÿ“… Created 2020-10-26
11
h2oai/h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

h2oai
โญ 7,527 Jupyter Notebook ๐Ÿ‘ค h2oai Organization ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2014-03-03
12
Alluxio/alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud

Alluxio
โญ 7,264 Java ๐Ÿ‘ค Alluxio Organization ๐Ÿ• Updated 2026-09-01 ๐Ÿ“… Created 2012-12-21
13
apache/hive

Apache Hive

apache
โญ 6,038 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2009-05-21
14
apache/calcite

Apache Calcite

apache
โญ 5,216 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2014-06-25
15
apache/ignite

Apache Ignite

apache
โญ 5,090 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2015-02-19
16
apache/nutch

Apache Nutch is an extensible and scalable web crawler

apache
โญ 3,311 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2009-05-21
17
WeBankFinTech/DataSphereStudio

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

WeBankFinTech
โญ 3,275 Java ๐Ÿ‘ค WeBankFinTech Organization ๐Ÿ• Updated 2025-11-04 ๐Ÿ“… Created 2019-11-24
18
MoRan1607/BigDataGuide

Big Data Learning, Learn Big Data from Scratch, Includes Learning Videos and Interview Materials for Each Stage of Big Data Learning

MoRan1607
โญ 3,259 ๐Ÿ‘ค MoRan1607 User ๐Ÿ• Updated 2026-01-20 ๐Ÿ“… Created 2019-11-30
19
RayXBuilds/Movie_Recommend

Historical learning project: Spark movie recommendations, web scraping, web app, and backend. Ongoing development of the new version: cinematch-recommender.

RayXBuilds
โญ 3,007 Java ๐Ÿ‘ค RayXBuilds User ๐Ÿ• Updated 2026-10-21 ๐Ÿ“… Created 2018-04-18
20
geekyouth/SZT-bigdata

Shenzhen Metro Big Data Passenger Flow Analysis System๐Ÿš‡๐Ÿš„๐ŸŒŸ

geekyouth
โญ 2,469 Scala ๐Ÿ‘ค geekyouth User ๐Ÿ• Updated 2026-05-12 ๐Ÿ“… Created 2020-04-13
21
apache/kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

apache
โญ 2,379 Scala ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2017-12-18
22
big-data-europe/docker-hadoop

Apache Hadoop Docker Image

big-data-europe
โญ 2,312 Shell ๐Ÿ‘ค big-data-europe Organization ๐Ÿ• Updated 2024-02-01 ๐Ÿ“… Created 2016-03-09
23
h2oai/h2o-2

Please visit https://github.com/h2oai/h2o-3 for latest H2O

h2oai
โญ 2,307 Java ๐Ÿ‘ค h2oai Organization ๐Ÿ• Updated 2024-10-24 ๐Ÿ“… Created 2013-02-06
24
cdarlint/winutils

winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows

cdarlint
โญ 2,205 Shell ๐Ÿ‘ค cdarlint User ๐Ÿ• Updated 2024-05-16 ๐Ÿ“… Created 2019-01-04
25
apache/drill

Apache Drill is a distributed MPP query layer for self describing data

apache
โญ 2,026 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2012-09-05
26
elastic/elasticsearch-hadoop

:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop

elastic
โญ 1,966 Java ๐Ÿ‘ค elastic Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2013-03-11
27
collabH/bigdata-growth

The Big Data knowledge repository involves data warehouse modeling, real-time computing, Big Data, data middle platform, system design, Java, algorithms, etc.

collabH
โญ 1,838 Python ๐Ÿ‘ค collabH User ๐Ÿ• Updated 2026-04-18 ๐Ÿ“… Created 2020-06-10
28
OBenner/data-engineering-interview-questions

More than 2000+ Data engineer interview questions.

OBenner
โญ 1,815 Python ๐Ÿ‘ค OBenner User ๐Ÿ• Updated 2026-01-13 ๐Ÿ“… Created 2021-08-08
29
Qihoo360/hbox

AI on Hadoop

Qihoo360
โญ 1,729 Java ๐Ÿ‘ค Qihoo360 Organization ๐Ÿ• Updated 2025-07-01 ๐Ÿ“… Created 2017-12-01
30
wgzhao/Addax

Actively maintained successor to Alibaba DataX โ€” a fast, versatile, open-source ETL tool for 30+ RDBMS and NoSQL data sources.

wgzhao
โญ 1,455 Java ๐Ÿ‘ค wgzhao User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-07-17
31
apache/carbondata

High performance data store solution

apache
โญ 1,452 Scala ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-10-02 ๐Ÿ“… Created 2016-06-28
32
HariSekhon/Dockerfiles

50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak

HariSekhon
โญ 1,382 Shell ๐Ÿ‘ค HariSekhon User ๐Ÿ• Updated 2026-02-03 ๐Ÿ“… Created 2016-01-17
33
apache/ozone

Scalable, reliable, distributed storage system optimized for data analytics and object store workloads.

apache
โญ 1,335 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-10-02
34
DTStack/Taier

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display

DTStack
โญ 1,285 Java ๐Ÿ‘ค DTStack Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2021-03-02
35
HariSekhon/Nagios-Plugins

450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

HariSekhon
โญ 1,155 Python ๐Ÿ‘ค HariSekhon User ๐Ÿ• Updated 2026-02-03 ๐Ÿ“… Created 2012-12-30
36
realguoshuai/hadoop_study

Regularly update documentation for commonly used big data components in the Hadoop ecosystem. Priorities in order: Flink, Solr, Sparksql, ES, Scala, Kafka, Hbase/phoenix, Redis, Kerberos (Project includes Hadoop mind map, Evernote notes, simple Scala version demo, common utility classes, desensitized train code, continuous updates!!!)

realguoshuai
โญ 916 Java ๐Ÿ‘ค realguoshuai User ๐Ÿ• Updated 2026-03-09 ๐Ÿ“… Created 2017-12-15
37
HariSekhon/DevOps-Python-tools

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

HariSekhon
โญ 824 Python ๐Ÿ‘ค HariSekhon User ๐Ÿ• Updated 2026-02-03 ๐Ÿ“… Created 2015-10-27
38
HuQi2018/BiSheServer

This system is my graduation design project, titled "Design and Implementation of a Movie Recommendation System Based on User Profiles". It primarily uses Django as the base framework, adopting the MTV pattern, with MongoDB, MySQL, and Redis as databases. The movie data scraped from the Douban platform serves as the primary data source. User tags are mainly developed based on basic user information and behavioral information such as usage operation records. It is a recommendation system that utilizes Hadoop and Spark big data components for analysis and processing. The management system uses Django's built-in management system, enhanced with SimpleUI.

HuQi2018
โญ 743 Python ๐Ÿ‘ค HuQi2018 User ๐Ÿ• Updated 2026-06-02 ๐Ÿ“… Created 2021-03-28
39
WeBankFinTech/WeDataSphere

WeDataSphere is a financial grade, one-stop big data platform suite.

WeBankFinTech
โญ 676 ๐Ÿ‘ค WeBankFinTech Organization ๐Ÿ• Updated 2024-03-14 ๐Ÿ“… Created 2019-07-23
40
AbsaOSS/spline

Data Lineage Tracking And Visualization Solution

AbsaOSS
โญ 669 Scala ๐Ÿ‘ค AbsaOSS Organization ๐Ÿ• Updated 2026-11-04 ๐Ÿ“… Created 2017-05-30
41
linkedin/venice

Venice, Derived Data Platform for Planet-Scale Workloads.

linkedin
โญ 622 Java ๐Ÿ‘ค linkedin Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-03-18
42
ittqqzz/ECommerceRecommendSystem

E-commerce Big Data Real-time Product Recommendation System, Vue + TypeScript + ElementUI + Spring + Spark

ittqqzz
โญ 528 Java ๐Ÿ‘ค ittqqzz User ๐Ÿ• Updated 2026-02-13 ๐Ÿ“… Created 2019-10-28
43
apache/tez

Apache Tez

apache
โญ 523 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2013-04-08

Frequently Asked Questions

1

What is the most popular #hadoop repository on GitHub?

The top ranked open source #hadoop project on GitHub is donnemartin/data-science-ipython-notebooks with 28,776 stars. Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

2

Which programming languages are most used in #hadoop repositories?

Among the top ranked #hadoop projects on GitHub, the most common programming languages are Java, Python, Shell, Scala.

3

How is the #hadoop repository ranking calculated?

We track 43 active open source repositories tagged with #hadoop (500+ stars), totaling 238,277 stars with an average of 5,541 stars per repository. Rankings update daily from the GitHub API.