Git Stars Git Stars

#pipeline Repositories

Discover 82 active open source GitHub repositories tagged with #pipeline, accumulating 355,781 total stars (averaging 4,338 stars per project). Leading this topic is kestra-io/kestra with 30,303 stars.

82
Total Repositories
355,781
Total Stars
4,338
Avg Stars/Repo
1
kestra-io/kestra

Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

kestra-io
โญ 30,303 Java ๐Ÿ‘ค kestra-io Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-08-24
2
argoproj/argo-cd

Declarative Continuous Deployment for Kubernetes

argoproj
โญ 24,670 Go ๐Ÿ‘ค argoproj Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-02-09
3
PrefectHQ/prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

PrefectHQ
โญ 24,203 Python ๐Ÿ‘ค PrefectHQ Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-06-29
4
marimo-team/marimo

A reactive notebook for Python โ€” run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

marimo-team
โญ 23,666 Python ๐Ÿ‘ค marimo-team Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-08-14
5
airbytehq/airbyte

Open-source data movement for ELT pipelines and AI agents โ€” from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

airbytehq
โญ 22,426 Python ๐Ÿ‘ค airbytehq Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-07-27
6
jina-ai/serve

โ˜๏ธ Build multimodal AI applications with cloud-native stack

jina-ai
โญ 21,864 Python ๐Ÿ‘ค jina-ai Organization ๐Ÿ• Updated 2025-03-24 ๐Ÿ“… Created 2020-02-13
7
Avaiga/taipy

Turns Data and AI algorithms into production-ready web applications in no time.

Avaiga
โญ 19,458 Python ๐Ÿ‘ค Avaiga Organization ๐Ÿ• Updated 2026-08-10 ๐Ÿ“… Created 2022-02-18
8
fivetran/great_expectations

Always know what to expect from your data.

fivetran
โญ 11,941 Python ๐Ÿ‘ค fivetran Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2017-09-11
9
PRQL/prql

PRQL is a modern language for transforming data โ€” a simple, powerful, pipelined SQL replacement

PRQL
โญ 10,948 Rust ๐Ÿ‘ค PRQL Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-01-18
10
projectdiscovery/httpx

httpx is a fast and multi-purpose HTTP toolkit that allows running multiple probes using the retryablehttp library.

projectdiscovery
โญ 10,640 Go ๐Ÿ‘ค projectdiscovery Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-05-28
11
iam-veeramalla/Jenkins-Zero-To-Hero

Install Jenkins, configure Docker as slave, set up cicd, deploy applications to k8s using Argo CD in GitOps way.

iam-veeramalla
โญ 9,752 Python ๐Ÿ‘ค iam-veeramalla User ๐Ÿ• Updated 2025-03-12 ๐Ÿ“… Created 2023-02-01
12
tektoncd/pipeline

A cloud-native Pipeline resource.

tektoncd
โญ 9,110 Go ๐Ÿ‘ค tektoncd Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-08-29
13
mage-ai/mage-ai

๐Ÿง™ Build, run, and manage data pipelines for integrating and transforming data.

mage-ai
โญ 8,850 Python ๐Ÿ‘ค mage-ai Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-05-16
14
brunch/brunch

๐Ÿด Web applications made easy. Since 2011.

brunch
โญ 6,753 JavaScript ๐Ÿ‘ค brunch Organization ๐Ÿ• Updated 2026-04-19 ๐Ÿ“… Created 2011-01-21
15
gonglei007/GameDevMind

The most comprehensive Game Development Map. Helps game developers save time on known issues, freeing up more energy for more creative work.

gonglei007
โญ 6,700 Python ๐Ÿ‘ค gonglei007 User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2022-06-22
16
nteract/papermill

๐Ÿ“š Parameterize, execute, and analyze notebooks

nteract
โญ 6,499 Python ๐Ÿ‘ค nteract Organization ๐Ÿ• Updated 2026-10-05 ๐Ÿ“… Created 2017-07-06
17
tencentmusic/cube-studio

cube studio Open-Source Cloud-Native All-in-One Machine Learning/Deep Learning/Large Model AI Platform, MLOps algorithm link full process, computing power rental platform, notebook online development, drag-and-drop task flow pipeline orchestration, multi-machine multi-GPU distributed training, hyperparameter search, inference service VGPU virtualization, edge computing, annotation platform automated annotation, deepseek and other large models sft fine-tuning/reward model/reinforcement learning training, vllm/ollama/mindie large model multi-machine inference, private knowledge base, AI model market, supports domestic CPU/GPU/NPU Ascend ecosystem, supports RDMA, supports distributed training such as pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano

tencentmusic
โญ 5,085 ๐Ÿ‘ค tencentmusic Organization ๐Ÿ• Updated 2026-10-09 ๐Ÿ“… Created 2021-08-17
18
jenkins-x/jx

Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests using Cloud Native pipelines from Tekton

jenkins-x
โญ 4,692 Go ๐Ÿ‘ค jenkins-x Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2018-01-05
19
alirezadir/Production-Level-Deep-Learning

A guideline for building practical production-level deep learning systems to be deployed in real world applications.

alirezadir
โญ 4,641 ๐Ÿ‘ค alirezadir User ๐Ÿ• Updated 2025-06-13 ๐Ÿ“… Created 2019-11-19
20
kubeflow/pipelines

Machine Learning Pipelines for Kubeflow

kubeflow
โญ 4,269 Python ๐Ÿ‘ค kubeflow Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-05-12
21
firecow/gitlab-ci-local

Tired of pushing to test your .gitlab-ci.yml?

firecow
โญ 4,165 TypeScript ๐Ÿ‘ค firecow User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2020-01-04
22
nextflow-io/nextflow

A workflow language for data-driven computational pipelines

nextflow-io
โญ 3,530 Groovy ๐Ÿ‘ค nextflow-io Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2013-03-27
23
towhee-io/towhee

Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.

towhee-io
โญ 3,444 Python ๐Ÿ‘ค towhee-io Organization ๐Ÿ• Updated 2024-10-18 ๐Ÿ“… Created 2021-07-13
24
EpicGames/BlenderTools

Blender addons that improve the game development workflow between Blender and Unreal.

EpicGames
โญ 3,418 Python ๐Ÿ‘ค EpicGames Organization ๐Ÿ• Updated 2026-09-09 ๐Ÿ“… Created 2020-06-24
25
LLMQuant/quant-mind

QuantMind is an open source agent-native knowledge extraction and retrieval framework for quantitative finance.

LLMQuant
โญ 3,137 Python ๐Ÿ‘ค LLMQuant Organization ๐Ÿ• Updated 2026-08-15 ๐Ÿ“… Created 2025-03-14
26
apptension/saas-boilerplate

SaaS Boilerplate - Open Source and free SaaS stack that lets you build SaaS products faster in React, Django and AWS. Focus on essential business logic instead of coding repeatable features!

apptension
โญ 3,031 TypeScript ๐Ÿ‘ค apptension Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2020-04-14
27
numaproj/numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs

numaproj
โญ 2,837 Rust ๐Ÿ‘ค numaproj Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2022-05-20
28
data-infra/cube-studio

CubeStudio is an open-source cloud-native one-stop machine learning/deep learning/large model AI platform/MaaS/MLOps/artificial intelligence platform/training and inference platform, covering the full algorithm pipeline, multi-tenancy, computing power leasing platform, token relay, drag-and-drop task flow pipeline orchestration, multi-machine multi-card distributed training, hyperparameter search, inference services, VGPU virtualization, cloud-edge-device collaboration, edge computing, automated annotation platform, SFT fine-tuning/reward model/reinforcement learning training for large models like DeepSeek, multi-machine inference for large models with vLLM/Ollama/MindIE, private knowledge base LLMOps intelligent agents, AI model marketplace, support for domestic heterogeneous computing power scheduling (Ascend, Cambricon, Hygon, Moore Threads, Muxi, etc.), support for IB/RoCE/RDMA, and Xinchuang support.

data-infra
โญ 2,615 Python ๐Ÿ‘ค data-infra User ๐Ÿ• Updated 2026-10-12 ๐Ÿ“… Created 2024-07-02
29
alibaba/pipcook

Machine learning platform for Web developers

alibaba
โญ 2,589 TypeScript ๐Ÿ‘ค alibaba Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-12-05
30
InternLM/HuixiangDou

HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance

InternLM
โญ 2,500 Python ๐Ÿ‘ค InternLM Organization ๐Ÿ• Updated 2025-11-24 ๐Ÿ“… Created 2023-12-28
31
EntilZha/PyFunctional

Python library for creating data pipelines with chain functional programming

EntilZha
โญ 2,488 Python ๐Ÿ‘ค EntilZha User ๐Ÿ• Updated 2025-03-13 ๐Ÿ“… Created 2015-02-05
32
instill-ai/instill-core

Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

instill-ai
โญ 2,322 Python ๐Ÿ‘ค instill-ai Organization ๐Ÿ• Updated 2026-06-01 ๐Ÿ“… Created 2022-01-13
33
ChunelFeng/CGraph

A common used C++ & Python DAG framework. A cross-platform, zero third-party dependency, flow-graph based parallel computing framework, included in awesome-cpp. Welcome star & fork & exchange

ChunelFeng
โญ 2,319 C++ ๐Ÿ‘ค ChunelFeng User ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2021-04-26
34
timeplus-io/proton

The fastest unified streaming SQL engine in a single C++ binary. โšก Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

timeplus-io
โญ 2,286 C++ ๐Ÿ‘ค timeplus-io Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2023-08-14
35
reugn/go-streams

A lightweight stream processing library for Go

reugn
โญ 2,178 Go ๐Ÿ‘ค reugn User ๐Ÿ• Updated 2026-01-14 ๐Ÿ“… Created 2019-04-30
36
google/xls

XLS: Accelerated HW Synthesis

google
โญ 1,991 C++ ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2020-05-07
37
JuliaAI/MLJ.jl

A Julia machine learning framework

JuliaAI
โญ 1,947 Julia ๐Ÿ‘ค JuliaAI Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2018-08-01
38
galaxyproject/galaxy

Data intensive science for everyone.

galaxyproject
โญ 1,900 Python ๐Ÿ‘ค galaxyproject Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2015-02-23
39
destel/rill

Go toolkit for clean, composable, channel-based concurrency.

destel
โญ 1,898 Go ๐Ÿ‘ค destel User ๐Ÿ• Updated 2026-09-29 ๐Ÿ“… Created 2024-02-02
40
BloodAxe/pytorch-toolbelt

PyTorch extensions for fast R&D prototyping and Kaggle farming

BloodAxe
โญ 1,575 Python ๐Ÿ‘ค BloodAxe User ๐Ÿ• Updated 2025-10-09 ๐Ÿ“… Created 2019-03-15
41
apache/hop

Hop Orchestration Platform

apache
โญ 1,544 Java ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2019-09-24
42
nf-core/rnaseq

RNA sequencing analysis pipeline using STAR, RSEM, HISAT2 or Salmon with gene/isoform counts and extensive quality control.

nf-core
โญ 1,410 Nextflow ๐Ÿ‘ค nf-core Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2018-03-29
43
weifuwan/seatunnel-web

Making Apache SeaTunnel easier to use.

weifuwan
โญ 1,406 TypeScript ๐Ÿ‘ค weifuwan User ๐Ÿ• Updated 2026-07-16 ๐Ÿ“… Created 2026-02-10
44
tenox7/ttyplot

a realtime plotting utility for terminal/console with data input from stdin

tenox7
โญ 1,390 C ๐Ÿ‘ค tenox7 User ๐Ÿ• Updated 2026-10-13 ๐Ÿ“… Created 2018-10-11
45
ropensci/drake

An R-focused pipeline toolkit for reproducibility and high-performance computing

ropensci
โญ 1,341 R ๐Ÿ‘ค ropensci Organization ๐Ÿ• Updated 2024-12-04 ๐Ÿ“… Created 2017-02-20
46
StarlightSearch/EmbedAnything

Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust ๐Ÿฆ€

StarlightSearch
โญ 1,334 Rust ๐Ÿ‘ค StarlightSearch Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-03-31
47
TencentBlueKing/bk-sops

Blue Whale Intelligent Cloud Standard Operations (SOPS)

TencentBlueKing
โญ 1,288 Python ๐Ÿ‘ค TencentBlueKing Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-10-15
48
uhop/stream-json

A micro-library of stream components for building custom JSON and JSONC processing pipelines with a minimal memory footprint โ€” parse, filter, and transform JSON far larger than available memory with a SAX-inspired token API, on Node.js or Web Streams.

uhop
โญ 1,209 JavaScript ๐Ÿ‘ค uhop User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2013-08-11
49
shenwei356/rush

A cross-platform command-line tool for executing jobs in parallel

shenwei356
โญ 1,137 Go ๐Ÿ‘ค shenwei356 User ๐Ÿ• Updated 2026-11-14 ๐Ÿ“… Created 2017-01-03
50
codeskyblue/go-sh

like python-sh, for easy call shell with golang.

codeskyblue
โญ 1,133 Go ๐Ÿ‘ค codeskyblue User ๐Ÿ• Updated 2025-05-09 ๐Ÿ“… Created 2014-02-07

Frequently Asked Questions

1

What is the most popular #pipeline repository on GitHub?

The top ranked open source #pipeline project on GitHub is kestra-io/kestra with 30,303 stars. Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

2

Which programming languages are most used in #pipeline repositories?

Among the top ranked #pipeline projects on GitHub, the most common programming languages are Python, Go, TypeScript, Rust.

3

How is the #pipeline repository ranking calculated?

We track 82 active open source repositories tagged with #pipeline (500+ stars), totaling 355,781 stars with an average of 4,338 stars per repository. Rankings update daily from the GitHub API.