Git Stars Git Stars

#big-data Repositories

133
Total Repositories
683,600
Total Stars
5,139
Avg Stars/Repo
101
natayadev/dataengineering-roadmap

Another repository with basic concepts, technical challenges, and resources on data engineering in Spanish ๐Ÿง™โœจ

natayadev
โญ 919 ๐Ÿ‘ค natayadev User ๐Ÿ• Updated 2026-02-28 ๐Ÿ“… Created 2023-11-24
102
joshday/OnlineStats.jl

โšก Single-pass algorithms for statistics

joshday
โญ 900 Julia ๐Ÿ‘ค joshday User ๐Ÿ• Updated 2026-09-21 ๐Ÿ“… Created 2015-02-04
โญ 872 Jupyter Notebook ๐Ÿ‘ค miguelgfierro User ๐Ÿ• Updated 2026-02-09 ๐Ÿ“… Created 2016-10-02
104
gvinciguerra/PGM-index

๐Ÿ…State-of-the-art learned data structure that enables fast lookup, predecessor, range searches and updates in arrays of billions of items using orders of magnitude less space than traditional indexes

gvinciguerra
โญ 859 C++ ๐Ÿ‘ค gvinciguerra User ๐Ÿ• Updated 2024-11-28 ๐Ÿ“… Created 2019-10-18
105
apache/samza

Mirror of Apache Samza

apache
โญ 844 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-09-01 ๐Ÿ“… Created 2015-03-14
106
nipy/nipype

Workflows and interfaces for neuroimaging packages

nipy
โญ 837 Python ๐Ÿ‘ค nipy Organization ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2010-07-22
107
cernopendata/opendata.cern.ch

Source code for the CERN Open Data portal

cernopendata
โญ 828 Python ๐Ÿ‘ค cernopendata Organization ๐Ÿ• Updated 2026-10-05 ๐Ÿ“… Created 2014-06-23
108
awslabs/python-deequ

Python API for Deequ

awslabs
โญ 826 Jupyter Notebook ๐Ÿ‘ค awslabs Organization ๐Ÿ• Updated 2026-10-24 ๐Ÿ“… Created 2020-11-09
109
fry404006308/fry_course_materials

Fan Renyi recorded course materials will successively launch various completely free courses on front-end, back-end, big data, artificial intelligence, etc. Course website: https://fanrenyi.com ; Bilibili course address: https://space.bilibili.com/45664489 ;

fry404006308
โญ 773 Jupyter Notebook ๐Ÿ‘ค fry404006308 User ๐Ÿ• Updated 2024-08-19 ๐Ÿ“… Created 2019-12-21
110
apache/orc

Apache ORC - the smallest, fastest columnar storage for Hadoop workloads

apache
โญ 773 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2015-05-06
111
derar-alhussein/Databricks-Certified-Data-Engineer-Associate

The resources of the preparation course for Databricks Data Engineer Associate certification exam

derar-alhussein
โญ 762 Python ๐Ÿ‘ค derar-alhussein User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2022-11-06
112
TouK/nussknacker

Low-code tool for automating actions on real time data | Stream processing for the users.

TouK
โญ 746 Scala ๐Ÿ‘ค TouK Organization ๐Ÿ• Updated 2026-09-30 ๐Ÿ“… Created 2017-06-29
โญ 742 C++ ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2025-08-15 ๐Ÿ“… Created 2021-07-26
114
caioricciuti/ch-ui

Use CH-UI to work with your data from self-hosted ClickHouse through a user-friendly interface. CH-UI is a modern, feature-rich user interface for ClickHouse databases. It provides an intuitive platform for querying ClickHouse databases, executing queries, and visualizing metrics about your instance.

caioricciuti
โญ 727 Go ๐Ÿ‘ค caioricciuti User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-05-16
115
MrXujiang/v6.dooring.public

Visual dashboard solution, providing a visual editing engine to help individuals or businesses easily customize their own visual dashboard applications.

MrXujiang
โญ 715 TypeScript ๐Ÿ‘ค MrXujiang User ๐Ÿ• Updated 2025-07-06 ๐Ÿ“… Created 2021-03-08
116
elastic/eland

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

elastic
โญ 693 Python ๐Ÿ‘ค elastic Organization ๐Ÿ• Updated 2026-06-16 ๐Ÿ“… Created 2019-06-11
117
apache/bigtop

Bigtop is an Apache Foundation project for Infrastructure Engineers and Data Scientists looking for comprehensive packaging, testing, and configuration of the leading open source big data components.

apache
โญ 680 Groovy ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-09-28 ๐Ÿ“… Created 2011-08-04
118
uxlfoundation/oneDAL

oneAPI Data Analytics Library (oneDAL)

uxlfoundation
โญ 654 C++ ๐Ÿ‘ค uxlfoundation Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2016-03-28
119
unum-cloud/UStore

Embedded transactional database for blobs, documents & graphs, with kernel-bypass NVMe through io_uring & SPDK, GPU compaction, and one C ABI over its own, RocksDB & LevelDB engines ๐Ÿ—„๏ธ

unum-cloud
โญ 649 C++ ๐Ÿ‘ค unum-cloud Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2022-06-12
120
shaiwz/data-platform-open

๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ Visual drag-and-drop big data integration platform, big data platform, and big data solution, featuring data flow, data sources, data alignment, query templates, intelligent Q&A, AI integration, and comprehensive monitoring. Synchronize and clean data in a no-code, flowchart-like manner.

shaiwz
โญ 648 Java ๐Ÿ‘ค shaiwz Organization ๐Ÿ• Updated 2026-10-30 ๐Ÿ“… Created 2025-01-19
121
iejMac/video2dataset

Easily create large video dataset from video urls

iejMac
โญ 644 Python ๐Ÿ‘ค iejMac User ๐Ÿ• Updated 2024-07-30 ๐Ÿ“… Created 2022-10-15
122
Mario343214420/Webgl-Data-Demo

A data visualization project based on Vue, three.js, and echarts, including features such as interactive 3D model import and 3D model annotation.

Mario343214420
โญ 635 Vue ๐Ÿ‘ค Mario343214420 User ๐Ÿ• Updated 2024-12-09 ๐Ÿ“… Created 2020-05-20
123
nghuyong/weibo-cov

Weibo-COV: A Large-Scale COVID-19 Social Media Dataset from Weibo

nghuyong
โญ 626 ๐Ÿ‘ค nghuyong User ๐Ÿ• Updated 2025-08-23 ๐Ÿ“… Created 2020-02-20
124
synmetrix/synmetrix

Synmetrix โ€“ production-ready open source semantic layer on Cube

synmetrix
โญ 616 JavaScript ๐Ÿ‘ค synmetrix Organization ๐Ÿ• Updated 2025-02-07 ๐Ÿ“… Created 2021-06-18
125
yahoo/redislite

Redis in a python module.

yahoo
โญ 598 Python ๐Ÿ‘ค yahoo Organization ๐Ÿ• Updated 2024-08-16 ๐Ÿ“… Created 2015-03-03
126
YotpoLtd/metorikku

A simplified, lightweight ETL framework based on Apache Spark

YotpoLtd
โญ 589 Scala ๐Ÿ‘ค YotpoLtd Organization ๐Ÿ• Updated 2026-09-01 ๐Ÿ“… Created 2017-10-10
127
spark-examples/spark-scala-examples

This project provides Apache Spark SQL, RDD, DataFrame and Dataset examples in Scala language

spark-examples
โญ 567 Scala ๐Ÿ‘ค spark-examples Organization ๐Ÿ• Updated 2024-03-20 ๐Ÿ“… Created 2019-11-26
128
Interana/eventsim

Event data simulator. Generates a stream of pseudo-random events from a set of users, designed to simulate web traffic.

Interana
โญ 553 Scala ๐Ÿ‘ค Interana Organization ๐Ÿ• Updated 2026-01-27 ๐Ÿ“… Created 2014-09-05
129
nicgirault/circosJS

d3 library to build circular graphs

nicgirault
โญ 534 JavaScript ๐Ÿ‘ค nicgirault User ๐Ÿ• Updated 2026-01-23 ๐Ÿ“… Created 2014-11-28
130
Lonero-Team/Decentralized-Internet

A SDK/library for decentralized web and distributing computing projects

Lonero-Team
โญ 525 JavaScript ๐Ÿ‘ค Lonero-Team Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-10-31
131
apache/tez

Apache Tez

apache
โญ 523 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2013-04-08
132
LearningJournal/Spark-Programming-In-Python

Apache Spark 3 - Spark Programming in Python for Beginners

LearningJournal
โญ 510 Python ๐Ÿ‘ค LearningJournal User ๐Ÿ• Updated 2024-07-25 ๐Ÿ“… Created 2020-05-31
133
apache/helix

Mirror of Apache Helix

apache
โญ 505 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2014-01-13