#data-integration Repositories
Discover 57 active open source GitHub repositories tagged with #data-integration, accumulating 259,956 total stars (averaging 4,560 stars per project). Leading this topic is apache/airflow with 47,583 stars.
57 repositories total
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Open-source data movement for ELT pipelines and AI agents โ from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
An orchestration platform for the development, production, and observation of data assets.
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
DataX integrated visualization page: Select a data source to generate data synchronization tasks with one click. Supports data sources such as RDBMS, Hive, HBase, ClickHouse, MongoDB, etc. Batch creation of RDBMS data synchronization tasks, integration with open-source scheduling systems, support for distributed processing, incremental data synchronization, real-time viewing of running logs, monitoring executor resources, KILL running processes, data source information encryption, and more.
๐ฆ event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc.
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
Connect your company's data, let an agent draft an ontology over it, and load it into HelixDB as one graph you can explore and ask questions of. Hosted at foundry.helix-db.com, or run it on your own machine.
:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop
Open-source ETL/ELT that you deploy on your own servers or in the cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 446 components, dbt, CDC, data quality, reverse ETL, lineage, and MCP for AI agents. No vendor cloud, no per-row billing.
This repository stores the N8N workflow configurations shared on my YouTube channel. Users can directly download the JSON files and import them into N8N for use.
Actively maintained successor to Alibaba DataX โ a fast, versatile, open-source ETL tool for 30+ RDBMS and NoSQL data sources.
Dropbase helps developers build and prototype web apps faster with AI. Dropbase is local-first and self hosted.
Insightful Tutorials and Papers about Knowledge Graphs
๐ Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. ๐
qData is a lightweight, feature-complete, AI-native open-source data platform. It covers data integration, development, modeling, governance, assets and API services, with self-hosted deployment, flexible scaling and commercial use.
SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).
Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.
Continuously updated paper list on advancements in Data Agents. Companion repo to our paper "A Survey of Data Agents: Emerging Paradigm or Overstated Hype?"
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own data sources and code.
Python library to train neural networks with a strong focus on hydrological applications.
This tutorial explains step by step how to connect your data to the Charting Library
Frequently Asked Questions
What is the most popular #data-integration repository on GitHub?
The top ranked open source #data-integration project on GitHub is apache/airflow with 47,583 stars. Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Which programming languages are most used in #data-integration repositories?
Among the top ranked #data-integration projects on GitHub, the most common programming languages are Java, Python, Go, Rust.
How is the #data-integration repository ranking calculated?
We track 57 active open source repositories tagged with #data-integration (500+ stars), totaling 259,956 stars with an average of 4,560 stars per repository. Rankings update daily from the GitHub API.