Git Stars Git Stars

#data-engineering Repositories

Discover 152 active open source GitHub repositories tagged with #data-engineering, accumulating 727,632 total stars (averaging 4,787 stars per project). Leading this topic is apache/superset with 75,603 stars.

152
Total Repositories
727,632
Total Stars
4,787
Avg Stars/Repo
1
apache/superset

Apache Superset is a Data Visualization and Data Exploration Platform

apache
โญ 75,603 Python ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2015-07-21
2
GokuMohandas/Made-With-ML

Learn how to develop, deploy and iterate on production-grade ML applications.

GokuMohandas
โญ 50,112 Jupyter Notebook ๐Ÿ‘ค GokuMohandas User ๐Ÿ• Updated 2026-03-04 ๐Ÿ“… Created 2018-11-05
3
apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

apache
โญ 47,583 Python ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2015-04-13
4
DataTalksClub/data-engineering-zoomcamp

Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here ๐Ÿ‘‡๐Ÿผ

DataTalksClub
โญ 46,997 Jupyter Notebook ๐Ÿ‘ค DataTalksClub Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2021-10-21
5
kestra-io/kestra

Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

kestra-io
โญ 30,303 Java ๐Ÿ‘ค kestra-io Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-08-24
6
eugeneyan/applied-ml

๐Ÿ“š Papers & tech blogs by companies sharing their work on data science & machine learning in production.

eugeneyan
โญ 28,577 ๐Ÿ‘ค eugeneyan User ๐Ÿ• Updated 2024-07-18 ๐Ÿ“… Created 2020-07-04
7
PrefectHQ/prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

PrefectHQ
โญ 24,203 Python ๐Ÿ‘ค PrefectHQ Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-06-29
8
airbytehq/airbyte

Open-source data movement for ELT pipelines and AI agents โ€” from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

airbytehq
โญ 22,426 Python ๐Ÿ‘ค airbytehq Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-07-27
9
Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.

Avaiga
โญ 19,458 Python ๐Ÿ‘ค Avaiga Organization ๐Ÿ• Updated 2026-08-10 ๐Ÿ“… Created 2022-02-18
10
argoproj/argo-workflows

Workflow Engine for Kubernetes

argoproj
โญ 17,108 Go ๐Ÿ‘ค argoproj Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2017-08-21
11
dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

dagster-io
โญ 16,453 Python ๐Ÿ‘ค dagster-io Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-04-30
12
andkret/Cookbook

The Data Engineering Cookbook

andkret
โญ 15,594 Python ๐Ÿ‘ค andkret User ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2019-03-10
13
semantica-agi/semantica

Graph-Native Semantic Layer for Trustworthy Enterprise AI

semantica-agi
โญ 14,701 Python ๐Ÿ‘ค semantica-agi Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-06-25
14
fivetran/great_expectations

Always know what to expect from your data.

fivetran
โญ 11,941 Python ๐Ÿ‘ค fivetran Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2017-09-11
15
cocoindex-io/cocoindex

Incremental engine for long horizon agents ๐ŸŒŸ Star if you like it!

cocoindex-io
โญ 11,923 Rust ๐Ÿ‘ค cocoindex-io Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2025-03-03
16
xonsh/xonsh

๐Ÿš Python-powered shell. Full-featured, cross-platform and AI-friendly.

xonsh
โญ 9,704 Python ๐Ÿ‘ค xonsh Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2015-01-21
17
risingwavelabs/risingwave

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

risingwavelabs
โญ 9,425 Rust ๐Ÿ‘ค risingwavelabs Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-01-28
18
mage-ai/mage-ai

๐Ÿง™ Build, run, and manage data pipelines for integrating and transforming data.

mage-ai
โญ 8,850 Python ๐Ÿ‘ค mage-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-05-16
19
redpanda-data/connect

Fancy stream processing made operationally mundane

redpanda-data
โญ 8,804 Go ๐Ÿ‘ค redpanda-data Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2016-03-22
20
growthbook/growthbook

Open Source Feature Flags, Experimentation, and Product Analytics

growthbook
โญ 8,639 TypeScript ๐Ÿ‘ค growthbook Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-05-07
21
Zipstack/unstract

LLM-Driven Extraction of Unstructured Data โ€” Built for API Deployments & ETL Pipeline Workflows

Zipstack
โญ 7,462 Python ๐Ÿ‘ค Zipstack Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2024-02-21
22
feast-dev/feast

The Open Source Feature Store for AI/ML

feast-dev
โญ 7,362 Python ๐Ÿ‘ค feast-dev Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-12-10
23
evidence-dev/evidence

Business intelligence as code: build fast, interactive data visualizations in SQL and markdown

evidence-dev
โญ 7,140 TypeScript ๐Ÿ‘ค evidence-dev Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2021-05-27
24
cloudquery/cloudquery

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

cloudquery
โญ 6,561 Go ๐Ÿ‘ค cloudquery Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-11-18
25
dlt-hub/dlt

data load tool (dlt) is an open source Python library that makes data loading easy ๐Ÿ› ๏ธ

dlt-hub
โญ 6,115 Python ๐Ÿ‘ค dlt-hub Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-01-26
26
WeiYe-Jing/datax-web

DataX integrated visualization page: Select a data source to generate data synchronization tasks with one click. Supports data sources such as RDBMS, Hive, HBase, ClickHouse, MongoDB, etc. Batch creation of RDBMS data synchronization tasks, integration with open-source scheduling systems, support for distributed processing, incremental data synchronization, real-time viewing of running logs, monitoring executor resources, KILL running processes, data source information encryption, and more.

WeiYe-Jing
โญ 5,958 Java ๐Ÿ‘ค WeiYe-Jing User ๐Ÿ• Updated 2024-06-02 ๐Ÿ“… Created 2019-11-19
27
Eventual-Inc/Daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Eventual-Inc
โญ 5,858 Rust ๐Ÿ‘ค Eventual-Inc Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2022-04-25
28
treeverse/lakeFS

lakeFS - Data version control for your data lake | Git for data

treeverse
โญ 5,590 Go ๐Ÿ‘ค treeverse Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2019-09-12
29
chiphuyen/dmls-book

Summaries and resources for Designing Machine Learning Systems book (Chip Huyen, O'Reilly 2022)

chiphuyen
โญ 5,481 ๐Ÿ‘ค chiphuyen User ๐Ÿ• Updated 2026-06-09 ๐Ÿ“… Created 2022-05-27
30
ruc-datalab/DeepAnalyze

DeepAnalyze is the first agentic LLM for autonomous data science. ๐ŸŽˆ Your AI data analyst automatically analyzes massive amounts of data and generates professional analysis reports with a single click!

ruc-datalab
โญ 4,833 Python ๐Ÿ‘ค ruc-datalab Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2025-10-11
31
rudderlabs/rudder-server

Privacy and Security focused Segment-alternative, in Golang and React

rudderlabs
โญ 4,520 Go ๐Ÿ‘ค rudderlabs Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-07-19
32
Moataz-Elmesmary/Data-Science-Roadmap

Data Science Roadmap from A to Z

Moataz-Elmesmary
โญ 4,429 ๐Ÿ‘ค Moataz-Elmesmary User ๐Ÿ• Updated 2025-12-06 ๐Ÿ“… Created 2022-04-17
33
whoiskatrin/sql-translator

SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.

whoiskatrin
โญ 4,315 TypeScript ๐Ÿ‘ค whoiskatrin User ๐Ÿ• Updated 2025-07-06 ๐Ÿ“… Created 2023-03-11
34
aws/aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

aws
โญ 4,124 Python ๐Ÿ‘ค aws Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-02-26
35
adilkhash/Data-Engineering-HowTo

A list of useful resources to learn Data Engineering from scratch

adilkhash
โญ 4,049 ๐Ÿ‘ค adilkhash User ๐Ÿ• Updated 2026-10-30 ๐Ÿ“… Created 2019-03-28
36
knownsec/aipyapp

AI-Powered Python & Python-Powered AI (Python-Use)

knownsec
โญ 4,026 HTML ๐Ÿ‘ค knownsec Organization ๐Ÿ• Updated 2026-02-15 ๐Ÿ“… Created 2025-04-06
37
quadratichq/quadratic

Spreadsheet with AI, Code, Connections

quadratichq
โญ 3,958 Rust ๐Ÿ‘ค quadratichq Organization ๐Ÿ• Updated 2026-03-07 ๐Ÿ“… Created 2021-11-11
38
camelot-dev/camelot

A Python library to extract tabular data from PDFs

camelot-dev
โญ 3,858 Python ๐Ÿ‘ค camelot-dev Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-07-01
39
Netflix/maestro

Maestro: Netflixโ€™s Workflow Orchestrator

Netflix
โญ 3,852 Java ๐Ÿ‘ค Netflix Organization ๐Ÿ• Updated 2026-11-06 ๐Ÿ“… Created 2024-04-17
40
dathere/qsv

Blazing-fast, neuro-symbolic data-wrangling toolkit

dathere
โญ 3,836 Rust ๐Ÿ‘ค dathere Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-12-11
41
moabukar/tech-vault

A list of many interview questions & real-world challenges in Tech! (https://tech-vault-web.vercel.app/)

moabukar
โญ 3,599 HCL ๐Ÿ‘ค moabukar User ๐Ÿ• Updated 2026-03-01 ๐Ÿ“… Created 2023-03-27
42
lakehq/sail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

lakehq
โญ 3,504 Rust ๐Ÿ‘ค lakehq Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-12-21
43
superstreamlabs/memphis

Memphis.dev is a highly scalable and effortless data streaming platform

superstreamlabs
โญ 3,442 Go ๐Ÿ‘ค superstreamlabs Organization ๐Ÿ• Updated 2026-03-02 ๐Ÿ“… Created 2022-02-01
44
GokuMohandas/mlops-course

Learn how to design, develop, deploy and iterate on production-grade ML applications.

GokuMohandas
โญ 3,272 Jupyter Notebook ๐Ÿ‘ค GokuMohandas User ๐Ÿ• Updated 2024-08-16 ๐Ÿ“… Created 2020-11-12
45
apache/devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

apache
โญ 3,204 Go ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2021-07-08
46
itsdecodingai/second-brain-ai-assistant-course

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.

itsdecodingai
โญ 3,138 Jupyter Notebook ๐Ÿ‘ค itsdecodingai Organization ๐Ÿ• Updated 2026-04-06 ๐Ÿ“… Created 2024-12-30
47
Zleap-AI/SAG

A new SOTA for RAG โ€” an original retrieval architecture and an open-source knowledge base for humans and agents.

Zleap-AI
โญ 3,004 Python ๐Ÿ‘ค Zleap-AI Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2025-11-07
48
gunnarmorling/awesome-opensource-data-engineering

An Awesome List of Open-Source Data Engineering Projects

gunnarmorling
โญ 2,946 ๐Ÿ‘ค gunnarmorling User ๐Ÿ• Updated 2024-10-04 ๐Ÿ“… Created 2020-02-11
49
danielbeach/data-engineering-practice

Data Engineering Practice Problems

danielbeach
โญ 2,706 Python ๐Ÿ‘ค danielbeach User ๐Ÿ• Updated 2025-01-08 ๐Ÿ“… Created 2022-02-17
50
meltano/meltano

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

meltano
โญ 2,683 Python ๐Ÿ‘ค meltano Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-06-21

Frequently Asked Questions

1

What is the most popular #data-engineering repository on GitHub?

The top ranked open source #data-engineering project on GitHub is apache/superset with 75,603 stars. Apache Superset is a Data Visualization and Data Exploration Platform

2

Which programming languages are most used in #data-engineering repositories?

Among the top ranked #data-engineering projects on GitHub, the most common programming languages are Python, Go, Rust, Jupyter Notebook.

3

How is the #data-engineering repository ranking calculated?

We track 152 active open source repositories tagged with #data-engineering (500+ stars), totaling 727,632 stars with an average of 4,787 stars per repository. Rankings update daily from the GitHub API.