#pipeline Repositories
Discover 82 active open source GitHub repositories tagged with #pipeline, accumulating 355,781 total stars (averaging 4,338 stars per project). Leading this topic is kestra-io/kestra with 30,303 stars.
82 repositories total
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
A reactive notebook for Python โ run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.
Open-source data movement for ELT pipelines and AI agents โ from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
PRQL is a modern language for transforming data โ a simple, powerful, pipelined SQL replacement
httpx is a fast and multi-purpose HTTP toolkit that allows running multiple probes using the retryablehttp library.
Install Jenkins, configure Docker as slave, set up cicd, deploy applications to k8s using Argo CD in GitOps way.
๐ง Build, run, and manage data pipelines for integrating and transforming data.
The most comprehensive Game Development Map. Helps game developers save time on known issues, freeing up more energy for more creative work.
cube studio Open-Source Cloud-Native All-in-One Machine Learning/Deep Learning/Large Model AI Platform, MLOps algorithm link full process, computing power rental platform, notebook online development, drag-and-drop task flow pipeline orchestration, multi-machine multi-GPU distributed training, hyperparameter search, inference service VGPU virtualization, edge computing, annotation platform automated annotation, deepseek and other large models sft fine-tuning/reward model/reinforcement learning training, vllm/ollama/mindie large model multi-machine inference, private knowledge base, AI model market, supports domestic CPU/GPU/NPU Ascend ecosystem, supports RDMA, supports distributed training such as pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano
Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests using Cloud Native pipelines from Tekton
A guideline for building practical production-level deep learning systems to be deployed in real world applications.
Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
Blender addons that improve the game development workflow between Blender and Unreal.
QuantMind is an open source agent-native knowledge extraction and retrieval framework for quantitative finance.
SaaS Boilerplate - Open Source and free SaaS stack that lets you build SaaS products faster in React, Django and AWS. Focus on essential business logic instead of coding repeatable features!
CubeStudio is an open-source cloud-native one-stop machine learning/deep learning/large model AI platform/MaaS/MLOps/artificial intelligence platform/training and inference platform, covering the full algorithm pipeline, multi-tenancy, computing power leasing platform, token relay, drag-and-drop task flow pipeline orchestration, multi-machine multi-card distributed training, hyperparameter search, inference services, VGPU virtualization, cloud-edge-device collaboration, edge computing, automated annotation platform, SFT fine-tuning/reward model/reinforcement learning training for large models like DeepSeek, multi-machine inference for large models with vLLM/Ollama/MindIE, private knowledge base LLMOps intelligent agents, AI model marketplace, support for domestic heterogeneous computing power scheduling (Ascend, Cambricon, Hygon, Moore Threads, Muxi, etc.), support for IB/RoCE/RDMA, and Xinchuang support.
HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance
Python library for creating data pipelines with chain functional programming
Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
A common used C++ & Python DAG framework. A cross-platform, zero third-party dependency, flow-graph based parallel computing framework, included in awesome-cpp. Welcome star & fork & exchange
The fastest unified streaming SQL engine in a single C++ binary. โก Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
RNA sequencing analysis pipeline using STAR, RSEM, HISAT2 or Salmon with gene/isoform counts and extensive quality control.
Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust ๐ฆ
A micro-library of stream components for building custom JSON and JSONC processing pipelines with a minimal memory footprint โ parse, filter, and transform JSON far larger than available memory with a SAX-inspired token API, on Node.js or Web Streams.
Frequently Asked Questions
What is the most popular #pipeline repository on GitHub?
The top ranked open source #pipeline project on GitHub is kestra-io/kestra with 30,303 stars. Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
Which programming languages are most used in #pipeline repositories?
Among the top ranked #pipeline projects on GitHub, the most common programming languages are Python, Go, TypeScript, Rust.
How is the #pipeline repository ranking calculated?
We track 82 active open source repositories tagged with #pipeline (500+ stars), totaling 355,781 stars with an average of 4,338 stars per repository. Rankings update daily from the GitHub API.