Git Stars Git Stars

#etl Repositories

105
Total Repositories
408,616
Total Stars
3,891
Avg Stars/Repo
51
datavane/tis

Support agile Ontology DataOps Based on Flink, DataX and Flink-CDC with Web-UI

datavane
โญ 1,448 Java ๐Ÿ‘ค datavane Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-01-23
52
toluaina/pgsync

Postgres, MySQL, or MariaDB to Elasticsearch/OpenSearch sync

toluaina
โญ 1,433 Python ๐Ÿ‘ค toluaina User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-08-17
53
xitongsys/parquet-go

pure golang library for reading/writing parquet file

xitongsys
โญ 1,432 Go ๐Ÿ‘ค xitongsys User ๐Ÿ• Updated 2026-05-08 ๐Ÿ“… Created 2017-09-02
54
amphi-ai/amphi-etl

visual data prep powered by python

amphi-ai
โญ 1,410 TypeScript ๐Ÿ‘ค amphi-ai Organization ๐Ÿ• Updated 2026-08-24 ๐Ÿ“… Created 2024-03-20
55
spark-examples/pyspark-examples

Pyspark RDD, DataFrame and Dataset Examples in Python language

spark-examples
โญ 1,371 Python ๐Ÿ‘ค spark-examples Organization ๐Ÿ• Updated 2025-12-07 ๐Ÿ“… Created 2019-11-26
56
gtoonstra/etl-with-airflow

ETL best practices with airflow, with examples

gtoonstra
โญ 1,353 Shell ๐Ÿ‘ค gtoonstra User ๐Ÿ• Updated 2024-09-25 ๐Ÿ“… Created 2016-10-16
57
yhat/pandasql

sqldf for pandas

yhat
โญ 1,351 Python ๐Ÿ‘ค yhat Organization ๐Ÿ• Updated 2024-07-24 ๐Ÿ“… Created 2013-02-18
58
subsquid/squid-sdk

TypeScript ETL toolkit for indexing Ethereum, Solana, and Substrate data, sourced from SQD Network.

subsquid
โญ 1,347 TypeScript ๐Ÿ‘ค subsquid Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2021-12-10
59
singer-io/getting-started

This repository is a getting started guide to Singer.

singer-io
โญ 1,342 Makefile ๐Ÿ‘ค singer-io Organization ๐Ÿ• Updated 2025-08-08 ๐Ÿ“… Created 2016-10-31
60
rwynn/monstache

a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.

rwynn
โญ 1,331 Go ๐Ÿ‘ค rwynn User ๐Ÿ• Updated 2025-08-22 ๐Ÿ“… Created 2016-05-14
61
2ndQuadrant/pglogical

Logical Replication extension for PostgreSQL 17, 16, 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.

2ndQuadrant
โญ 1,243 C ๐Ÿ‘ค 2ndQuadrant Organization ๐Ÿ• Updated 2026-07-28 ๐Ÿ“… Created 2016-05-10
62
react-csv/react-csv

React components to build CSV files on the fly basing on Array/literal object of data

react-csv
โญ 1,182 JavaScript ๐Ÿ‘ค react-csv Organization ๐Ÿ• Updated 2024-04-03 ๐Ÿ“… Created 2016-11-25
63
DataWithBaraa/sql-data-warehouse-project

Build a data warehouse in SQL Server from two source systems, a CRM and an ERP, through bronze, silver and gold. The files and scripts of the free full project.

DataWithBaraa
โญ 1,089 TSQL ๐Ÿ‘ค DataWithBaraa User ๐Ÿ• Updated 2026-11-06 ๐Ÿ“… Created 2024-12-30
64
jf-tech/omniparser

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

jf-tech
โญ 1,085 Go ๐Ÿ‘ค jf-tech User ๐Ÿ• Updated 2026-10-11 ๐Ÿ“… Created 2020-08-16
65
dataform-co/dataform

Dataform is a framework for managing SQL based data operations in BigQuery

dataform-co
โญ 998 TypeScript ๐Ÿ‘ค dataform-co Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-09-03
66
estuary/flow

๐ŸŒŠ Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. ๐ŸŒŠ

estuary
โญ 995 Rust ๐Ÿ‘ค estuary Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-02-13
67
uniVocity/univocity-parsers

uniVocity-parsers is a suite of extremely fast and reliable parsers for Java. It provides a consistent interface for handling different file formats, and a solid framework for the development of new parsers.

uniVocity
โญ 931 Java ๐Ÿ‘ค uniVocity Organization ๐Ÿ• Updated 2024-08-17 ๐Ÿ“… Created 2014-07-12
68
slingdata-io/sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.

slingdata-io
โญ 926 Go ๐Ÿ‘ค slingdata-io Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-10-15
69
flow-php/flow

The most advanced data processing framework allowing to build scalable data processing pipelines and move data between various data sources and destinations.

flow-php
โญ 875 PHP ๐Ÿ‘ค flow-php Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2021-05-23
70
NeumTry/NeumAI

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

NeumTry
โญ 864 Python ๐Ÿ‘ค NeumTry Organization ๐Ÿ• Updated 2024-01-15 ๐Ÿ“… Created 2023-09-14
71
Cinchoo/ChoETL

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Cinchoo
โญ 860 C# ๐Ÿ‘ค Cinchoo User ๐Ÿ• Updated 2026-10-19 ๐Ÿ“… Created 2016-10-19
72
aelassas/wexflow

Workflow Automation Engine

aelassas
โญ 845 C# ๐Ÿ‘ค aelassas User ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2022-10-29
73
awslabs/python-deequ

Python API for Deequ

awslabs
โญ 826 Jupyter Notebook ๐Ÿ‘ค awslabs Organization ๐Ÿ• Updated 2026-10-24 ๐Ÿ“… Created 2020-11-09
74
pgflo/pg_flo

Stream, transform, and route PostgreSQL data in real-time.

pgflo
โญ 825 Go ๐Ÿ‘ค pgflo Organization ๐Ÿ• Updated 2025-12-29 ๐Ÿ“… Created 2024-09-02
75
paillave/Etl.Net

Mass processing data with a complete ETL for .net developers

paillave
โญ 812 C# ๐Ÿ‘ค paillave User ๐Ÿ• Updated 2026-10-09 ๐Ÿ“… Created 2018-06-23
76
BasPH/data-pipelines-with-apache-airflow

Code for Data Pipelines with Apache Airflow

BasPH
โญ 810 Python ๐Ÿ‘ค BasPH User ๐Ÿ• Updated 2024-08-15 ๐Ÿ“… Created 2019-06-10
77
kzaremski/apple-notes-exporter

MacOS app written in Swift that bulk exports Apple Notes (including iCloud Notes) to a multitude of formats preserving note folder structure.

kzaremski
โญ 780 Swift ๐Ÿ‘ค kzaremski User ๐Ÿ• Updated 2026-09-19 ๐Ÿ“… Created 2023-02-15
78
raystack/optimus

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

raystack
โญ 757 Go ๐Ÿ‘ค raystack Organization ๐Ÿ• Updated 2024-06-08 ๐Ÿ“… Created 2021-03-22
79
microsoft/etl2pcapng

Utility that converts an .etl file containing a Windows network packet capture into .pcapng format.

microsoft
โญ 718 C ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2025-06-05 ๐Ÿ“… Created 2019-09-16
80
koopjs/koop

Transform, query, and download geospatial data on the web.

koopjs
โญ 715 JavaScript ๐Ÿ‘ค koopjs Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2013-07-20
81
neuml/paperetl

๐Ÿ“„ โš™๏ธ ETL processes for medical and scientific papers

neuml
โญ 701 Python ๐Ÿ‘ค neuml Organization ๐Ÿ• Updated 2025-12-07 ๐Ÿ“… Created 2020-07-10
82
awslabs/aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.

awslabs
โญ 701 Python ๐Ÿ‘ค awslabs Organization ๐Ÿ• Updated 2026-10-30 ๐Ÿ“… Created 2017-08-11
83
elastic/eland

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

elastic
โญ 693 Python ๐Ÿ‘ค elastic Organization ๐Ÿ• Updated 2026-06-16 ๐Ÿ“… Created 2019-06-11
84
tableau/webdataconnector

Bring the data you care about into Tableau

tableau
โญ 688 HTML ๐Ÿ‘ค tableau Organization ๐Ÿ• Updated 2026-10-19 ๐Ÿ“… Created 2015-10-12
85
dbt-labs/dbt-codegen

Macros that generate dbt code

dbt-labs
โญ 677 Makefile ๐Ÿ‘ค dbt-labs Organization ๐Ÿ• Updated 2026-10-01 ๐Ÿ“… Created 2019-06-10
86
datacleaner/DataCleaner

The premier open source Data Quality solution

datacleaner
โญ 646 Java ๐Ÿ‘ค datacleaner Organization ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2014-02-20
87
catalyst-cooperative/pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.

catalyst-cooperative
โญ 621 Python ๐Ÿ‘ค catalyst-cooperative Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2017-02-01
88
josephmachado/beginner_de_project

Beginner data engineering project - batch edition

josephmachado
โญ 620 HTML ๐Ÿ‘ค josephmachado User ๐Ÿ• Updated 2026-04-13 ๐Ÿ“… Created 2020-05-23
89
ConduitIO/conduit

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

ConduitIO
โญ 617 Go ๐Ÿ‘ค ConduitIO Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-01-11
90
apecloud/ape-dts

ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.

apecloud
โญ 616 Rust ๐Ÿ‘ค apecloud Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2023-01-12
91
insitro/redun

Yet another redundant workflow engine

insitro
โญ 610 Python ๐Ÿ‘ค insitro Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2021-11-04
92
aryn-ai/sycamore

Sycamore is an LLM-powered search and analytics platform for unstructured data.

aryn-ai
โญ 605 Python ๐Ÿ‘ค aryn-ai Organization ๐Ÿ• Updated 2026-08-01 ๐Ÿ“… Created 2023-07-07
93
Datavault-UK/automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)

Datavault-UK
โญ 599 ๐Ÿ‘ค Datavault-UK Organization ๐Ÿ• Updated 2026-02-05 ๐Ÿ“… Created 2019-09-27
94
YotpoLtd/metorikku

A simplified, lightweight ETL framework based on Apache Spark

YotpoLtd
โญ 589 Scala ๐Ÿ‘ค YotpoLtd Organization ๐Ÿ• Updated 2026-09-01 ๐Ÿ“… Created 2017-10-10
95
singer-io/singer-python

Writes the Singer format from Python

singer-io
โญ 585 Python ๐Ÿ‘ค singer-io Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2016-10-28
96
taz4et/bitcoin-etl

A Python toolkit for extracting and transforming Bitcoin blockchain data into structured formats.

taz4et
โญ 580 Python ๐Ÿ‘ค taz4et User ๐Ÿ• Updated 2025-11-24 ๐Ÿ“… Created 2025-11-14
97
stn1slv/awesome-integration

A curated list of awesome system integration software and resources.

stn1slv
โญ 572 Markdown ๐Ÿ‘ค stn1slv User ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2021-07-23
98
JoeyBling/webkettle

A distributed comprehensive scheduling, management, and ETL development user professional edition B/S architecture tool developed based on the web version of Kettle

JoeyBling
โญ 572 JavaScript ๐Ÿ‘ค JoeyBling User ๐Ÿ• Updated 2024-06-25 ๐Ÿ“… Created 2018-05-23
99
pracdata/awesome-open-source-data-engineering

A curated list of open source tools used in analytics platforms and data engineering ecosystem

pracdata
โญ 564 ๐Ÿ‘ค pracdata Organization ๐Ÿ• Updated 2025-03-12 ๐Ÿ“… Created 2024-02-22
100
vlm-run/vlmrun-hub

A hub for various industry-specific schemas to be used with VLMs.

vlm-run
โญ 555 Python ๐Ÿ‘ค vlm-run Organization ๐Ÿ• Updated 2025-12-15 ๐Ÿ“… Created 2024-11-11