#data-ingestion Repositories
Discover 33 active open source GitHub repositories tagged with #data-ingestion, accumulating 66,183 total stars (averaging 2,005 stars per project). Leading this topic is apache/seatunnel with 9,821 stars.
33 repositories total
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
LLM Zoomcamp - a free online course about real-life applications of LLMs. In 10 weeks you will learn how to build an AI system that answers questions about your knowledge base. Register here ๐๐ผ
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
Concurrent and multi-stage data ingestion and data processing with Elixir
OLake Go: open-source replication from databases, Kafka, and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
Projects on shell scripting. Goal is to simulate real time projects as much as posible.
A Collaborative Recipe Keeper, Meal Planner, and Shopping List Organizer in PWA form.
uniVocity-parsers is a suite of extremely fast and reliable parsers for Java. It provides a consistent interface for handling different file formats, and a solid framework for the development of new parsers.
Copy to/from Parquet in S3, Azure Blob Storage, Google Cloud Storage, http(s) stores, local files or standard inout stream from within PostgreSQL
A pure python based utility to extract text and images from docx files.
Kerf (Kerf1) is a columnar tick database and time-series language for Linux/OSX/BSD/iOS/Android. It is written in C and natively speaks JSON and SQL. Kerf can be used for trading platforms, feedhandlers, low-latency networking, high-volume analysis of realtime and historical data, logfile processing, and more.
The Supabase of AI era. A modular, open-source backend for building AI-native software โ designed for knowledge, not static data.
reth-indexer reads directly from the reth db and indexes the data into traditional and alternative databases / datastores (postgres, GCP bigquery, etc) all decoded with a simple config file and no extra setup alongside exposing a API ready to query the data.
Frequently Asked Questions
What is the most popular #data-ingestion repository on GitHub?
The top ranked open source #data-ingestion project on GitHub is apache/seatunnel with 9,821 stars. SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
Which programming languages are most used in #data-ingestion repositories?
Among the top ranked #data-ingestion projects on GitHub, the most common programming languages are Python, Java, Go, TypeScript.
How is the #data-ingestion repository ranking calculated?
We track 33 active open source repositories tagged with #data-ingestion (500+ stars), totaling 66,183 stars with an average of 2,005 stars per repository. Rankings update daily from the GitHub API.