#etl Repositories
105 repositories total
TypeScript ETL toolkit for indexing Ethereum, Solana, and Substrate data, sourced from SQD Network.
a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.
Logical Replication extension for PostgreSQL 17, 16, 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.
React components to build CSV files on the fly basing on Array/literal object of data
Build a data warehouse in SQL Server from two source systems, a CRM and an ERP, through bronze, silver and gold. The files and scripts of the free full project.
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
๐ Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. ๐
uniVocity-parsers is a suite of extremely fast and reliable parsers for Java. It provides a consistent interface for handling different file formats, and a solid framework for the development of new parsers.
Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
The most advanced data processing framework allowing to build scalable data processing pipelines and move data between various data sources and destinations.
Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.
ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
MacOS app written in Swift that bulk exports Apple Notes (including iCloud Notes) to a multitude of formats preserving note folder structure.
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Utility that converts an .etl file containing a Windows network packet capture into .pcapng format.
AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.
A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
A Python toolkit for extracting and transforming Bitcoin blockchain data into structured formats.
A distributed comprehensive scheduling, management, and ETL development user professional edition B/S architecture tool developed based on the web version of Kettle
A curated list of open source tools used in analytics platforms and data engineering ecosystem