Git Stars Git Stars

#distributed-computing Repositories

99
Total Repositories
277,188
Total Stars
2,799
Avg Stars/Repo
51
dotnetcore/SmartSql

SmartSql = MyBatis in C# + .NET Core+ Cache(Memory | Redis) + R/W Splitting + PropertyChangedTrack + Dynamic Repository + InvokeSync + Diagnostics

dotnetcore
โญ 1,134 C# ๐Ÿ‘ค dotnetcore Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2017-04-01
52
taikoxyz/simple-taiko-node

Start your Taiko node with a single command. ๐ŸŒ

taikoxyz
โญ 1,106 Shell ๐Ÿ‘ค taikoxyz Organization ๐Ÿ• Updated 2026-11-06 ๐Ÿ“… Created 2022-12-05
53
JohnSnowLabs/spark-nlp-workshop

Public runnable examples of using John Snow Labs' NLP for Apache Spark.

JohnSnowLabs
โญ 1,089 Jupyter Notebook ๐Ÿ‘ค JohnSnowLabs Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2018-08-20
54
joshuaboniface/rffmpeg

rffmpeg: remote SSH FFmpeg wrapper tool

joshuaboniface
โญ 1,087 Python ๐Ÿ‘ค joshuaboniface User ๐Ÿ• Updated 2025-11-03 ๐Ÿ“… Created 2019-10-19
55
mahmoudparsian/data-algorithms-book

MapReduce, Spark, Java, and Scala for Data Algorithms Book

mahmoudparsian
โญ 1,083 Java ๐Ÿ‘ค mahmoudparsian User ๐Ÿ• Updated 2024-10-14 ๐Ÿ“… Created 2014-08-06
56
NVIDIA-NeMo/Skills

A project to improve skills of large language models

NVIDIA-NeMo
โญ 1,065 Python ๐Ÿ‘ค NVIDIA-NeMo Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2024-02-11
57
pytorch/tensordict

A dict[str, torch.Tensor] that behaves like a single tensor, and TensorClass: typed, batched dataclasses that compile.

pytorch
โญ 1,046 Python ๐Ÿ‘ค pytorch Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2022-10-19
58
bcbio/bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis

bcbio
โญ 1,023 Python ๐Ÿ‘ค bcbio Organization ๐Ÿ• Updated 2024-08-24 ๐Ÿ“… Created 2013-02-06
59
futureverse/future

:rocket: R package: future: Unified Parallel and Distributed Processing in R for Everyone

futureverse
โญ 1,014 R ๐Ÿ‘ค futureverse Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2015-06-08
60
tensorflow/transform

Input pipeline framework

tensorflow
โญ 992 Python ๐Ÿ‘ค tensorflow Organization ๐Ÿ• Updated 2026-08-14 ๐Ÿ“… Created 2017-02-10
61
lizardfs/lizardfs

LizardFS is an Open Source Distributed File System licensed under GPLv3.

lizardfs
โญ 987 C++ ๐Ÿ‘ค lizardfs Organization ๐Ÿ• Updated 2024-08-11 ๐Ÿ“… Created 2013-05-01
62
nv-legate/cupynumeric

End of life; no longer maintained. Final public release: v26.06.01.

nv-legate
โญ 982 Python ๐Ÿ‘ค nv-legate Organization ๐Ÿ• Updated 2026-10-01 ๐Ÿ“… Created 2021-04-11
63
dask/dask-ml

Scalable Machine Learning with Dask

dask
โญ 950 Python ๐Ÿ‘ค dask Organization ๐Ÿ• Updated 2025-09-27 ๐Ÿ“… Created 2017-06-15
64
couler-proj/couler

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

couler-proj
โญ 944 Python ๐Ÿ‘ค couler-proj Organization ๐Ÿ• Updated 2024-10-08 ๐Ÿ“… Created 2020-08-18
65
lichess-org/fishnet

Distributed Stockfish analysis for lichess.org

lichess-org
โญ 944 Rust ๐Ÿ‘ค lichess-org Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2016-03-10
66
criteo/autofaiss

Automatically create Faiss knn indices with the most optimal similarity search parameters.

criteo
โญ 906 Python ๐Ÿ‘ค criteo Organization ๐Ÿ• Updated 2025-11-04 ๐Ÿ“… Created 2021-04-28
67
aphyr/tesser

Clojure reducers, but for parallel execution: locally and on distributed systems.

aphyr
โญ 893 Clojure ๐Ÿ‘ค aphyr User ๐Ÿ• Updated 2026-07-17 ๐Ÿ“… Created 2014-11-11
68
BaguaSys/bagua

Bagua Speeds up PyTorch

BaguaSys
โญ 881 Python ๐Ÿ‘ค BaguaSys Organization ๐Ÿ• Updated 2024-08-01 ๐Ÿ“… Created 2021-05-25
69
pywren/pywren

Teraflops and microservices

pywren
โญ 854 Python ๐Ÿ‘ค pywren Organization ๐Ÿ• Updated 2024-02-10 ๐Ÿ“… Created 2016-10-03
70
gojek/awesome-distributed-systems

Awesome list of distributed systems resources

gojek
โญ 836 ๐Ÿ‘ค gojek Organization ๐Ÿ• Updated 2024-07-10 ๐Ÿ“… Created 2018-08-16
71
fyoorer/ShadowClone

Unleash the power of cloud

fyoorer
โญ 822 Python ๐Ÿ‘ค fyoorer User ๐Ÿ• Updated 2026-09-30 ๐Ÿ“… Created 2022-01-21
72
centerforaisafety/HarmBench

HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

centerforaisafety
โญ 819 Jupyter Notebook ๐Ÿ‘ค centerforaisafety Organization ๐Ÿ• Updated 2024-08-16 ๐Ÿ“… Created 2024-02-02
73
taskflow/awesome-parallel-computing

A curated list of awesome parallel computing resources

taskflow
โญ 810 ๐Ÿ‘ค taskflow Organization ๐Ÿ• Updated 2026-11-11 ๐Ÿ“… Created 2019-09-19
74
zju3dv/DetectorFreeSfM

Code for "Detector-Free Structure from Motion", CVPR 2024

zju3dv
โญ 765 Python ๐Ÿ‘ค zju3dv Organization ๐Ÿ• Updated 2026-09-15 ๐Ÿ“… Created 2023-06-27
75
databricks/tensorframes

[DEPRECATED] Tensorflow wrapper for DataFrames on Apache Spark

databricks
โญ 745 Scala ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2024-07-30 ๐Ÿ“… Created 2016-03-04
76
muaimingjun/bigData

Big Data Competition Project Library

muaimingjun
โญ 734 Python ๐Ÿ‘ค muaimingjun User ๐Ÿ• Updated 2025-05-07 ๐Ÿ“… Created 2019-10-14
77
JuliaParallel/Dagger.jl

A framework for out-of-core and parallel execution

JuliaParallel
โญ 734 Julia ๐Ÿ‘ค JuliaParallel Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2015-12-03
78
lablup/backend.ai

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

lablup
โญ 681 Python ๐Ÿ‘ค lablup Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2016-10-06
79
google/yggdrasil-decision-forests

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

google
โญ 680 C++ ๐Ÿ‘ค google Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2021-04-22
80
DrHazemAli/enterprise-system-design

A source-grounded course and architectural reference for engineers designing systems that must survive real traffic, partial failure, security review, and changing requirements, spanning enterprise system design, distributed systems, AI systems, cybersecurity, reliability, cloud, HPC, edge, and mission-critical infrastructure.

DrHazemAli
โญ 663 ๐Ÿ‘ค DrHazemAli User ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2026-08-12
81
google-research/weatherbench2

A benchmark for the next generation of data-driven global weather models.

google-research
โญ 650 Python ๐Ÿ‘ค google-research Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2023-02-24
82
WwZzz/easyFL

An experimental platform for federated learning.

WwZzz
โญ 635 Jupyter Notebook ๐Ÿ‘ค WwZzz User ๐Ÿ• Updated 2026-10-20 ๐Ÿ“… Created 2021-05-10
83
Lightning-AI/litData

Speed up model training by fixing data loading.

Lightning-AI
โญ 617 Python ๐Ÿ‘ค Lightning-AI Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2024-02-15
84
scottdraves/electricsheep

infinite evolving crowdsourced artwork

scottdraves
โญ 617 C ๐Ÿ‘ค scottdraves User ๐Ÿ• Updated 2026-04-17 ๐Ÿ“… Created 2015-03-28
โญ 613 HTML ๐Ÿ‘ค gridcoin-community Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2014-08-25
86
apache/datafusion-python

Apache DataFusion Python Bindings

apache
โญ 612 Python ๐Ÿ‘ค apache Organization ๐Ÿ• Updated 2026-11-15 ๐Ÿ“… Created 2022-07-20
87
jaylfc/taOS

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).

jaylfc
โญ 609 Python ๐Ÿ‘ค jaylfc User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-04-05
88
AIxBlock-2023/awesome-ai-dev-platform-opensource

An On-Chain Open-Source Platform for Rapid AI Model Productization Using Decentralized Resources with Flexibility and Scalability

AIxBlock-2023
โญ 605 TypeScript ๐Ÿ‘ค AIxBlock-2023 Organization ๐Ÿ• Updated 2025-11-29 ๐Ÿ“… Created 2024-12-30
โญ 604 Jupyter Notebook ๐Ÿ‘ค generative-ai-on-aws Organization ๐Ÿ• Updated 2024-10-09 ๐Ÿ“… Created 2023-08-24
90
jmerelnyc/Talos

GPU worker client for the Talos network. Pairs with your Talos account, serves open-model inference jobs over a WebSocket, and reports uptime for payouts.

jmerelnyc
โญ 600 Python ๐Ÿ‘ค jmerelnyc User ๐Ÿ• Updated 2026-07-08 ๐Ÿ“… Created 2026-07-02
91
geopandas/dask-geopandas

Parallel GeoPandas with Dask

geopandas
โญ 593 Python ๐Ÿ‘ค geopandas Organization ๐Ÿ• Updated 2026-07-21 ๐Ÿ“… Created 2020-02-13
92
YotpoLtd/metorikku

A simplified, lightweight ETL framework based on Apache Spark

YotpoLtd
โญ 589 Scala ๐Ÿ‘ค YotpoLtd Organization ๐Ÿ• Updated 2026-09-01 ๐Ÿ“… Created 2017-10-10
93
It4innovations/hyperqueue

User-friendly Scheduler for sub-node tasks for HPC systems

It4innovations
โญ 576 Rust ๐Ÿ‘ค It4innovations Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2021-03-18
94
Harries/springboot-demo

a simple springboot demo with some components for example: redis,solr,rockmq and so on.

Harries
โญ 575 Java ๐Ÿ‘ค Harries User ๐Ÿ• Updated 2026-05-10 ๐Ÿ“… Created 2023-12-01
95
hazelcast/hazelcast-code-samples

Hazelcast Code Samples

hazelcast
โญ 558 Java ๐Ÿ‘ค hazelcast Organization ๐Ÿ• Updated 2026-11-02 ๐Ÿ“… Created 2013-07-27
96
kubeflow/mpi-operator

Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)

kubeflow
โญ 543 Go ๐Ÿ‘ค kubeflow Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-05-26
97
cas-bigdatalab/piflow

ฯ€flow is a big data flow engine with spark support

cas-bigdatalab
โญ 541 Python ๐Ÿ‘ค cas-bigdatalab Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-05-03
98
LearningJournal/Spark-Programming-In-Python

Apache Spark 3 - Spark Programming in Python for Beginners

LearningJournal
โญ 510 Python ๐Ÿ‘ค LearningJournal User ๐Ÿ• Updated 2024-07-25 ๐Ÿ“… Created 2020-05-31
99
NVIDIA/NeMo-Framework-Launcher

Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.

NVIDIA
โญ 508 Python ๐Ÿ‘ค NVIDIA Organization ๐Ÿ• Updated 2025-04-18 ๐Ÿ“… Created 2023-01-05