Git Stars Git Stars

#apache-spark Repositories

Discover 44 active open source GitHub repositories tagged with #apache-spark, accumulating 97,788 total stars (averaging 2,222 stars per project). Leading this topic is mlflow/mlflow with 28,920 stars.

44
Total Repositories
97,788
Total Stars
2,222
Avg Stars/Repo
1
mlflow/mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow
โญ 28,920 Python ๐Ÿ‘ค mlflow Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2018-06-05
2
treeverse/lakeFS

lakeFS - Data version control for your data lake | Git for data

treeverse
โญ 5,590 Go ๐Ÿ‘ค treeverse Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2019-09-12
3
JerryLead/SparkInternals

Notes talking about the design and implementation of Apache Spark

JerryLead
โญ 5,353 ๐Ÿ‘ค JerryLead User ๐Ÿ• Updated 2024-04-02 ๐Ÿ“… Created 2014-08-08
4
microsoft/SynapseML

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

microsoft
โญ 5,257 Scala ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2017-06-05
5
databricks/learning-spark

Example code from Learning Spark book

databricks
โญ 3,895 Java ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2026-06-30 ๐Ÿ“… Created 2014-06-16
6
lakehq/sail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

lakehq
โญ 3,504 Rust ๐Ÿ‘ค lakehq Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2023-12-21
7
kubeflow/spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

kubeflow
โญ 3,162 Python ๐Ÿ‘ค kubeflow Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-01-03
8
intel/BigDL

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

intel
โญ 2,699 Jupyter Notebook ๐Ÿ‘ค intel Organization ๐Ÿ• Updated 2026-06-12 ๐Ÿ“… Created 2017-05-05
9
dotnet/spark

.NET for Apacheยฎ Sparkโ„ข makes Apache Sparkโ„ข easily accessible to .NET developers.

dotnet
โญ 2,096 C# ๐Ÿ‘ค dotnet Organization ๐Ÿ• Updated 2026-11-05 ๐Ÿ“… Created 2019-04-22
10
big-data-europe/docker-spark

Apache Spark docker image

big-data-europe
โญ 2,045 Shell ๐Ÿ‘ค big-data-europe Organization ๐Ÿ• Updated 2026-04-20 ๐Ÿ“… Created 2015-10-08
11
feathr-ai/feathr

Feathr โ€“ A scalable, unified data and AI engineering platform for enterprise

feathr-ai
โญ 1,916 Scala ๐Ÿ‘ค feathr-ai Organization ๐Ÿ• Updated 2024-04-04 ๐Ÿ“… Created 2022-02-18
12
awesome-spark/awesome-spark

A curated list of awesome Apache Spark packages and resources.

awesome-spark
โญ 1,903 Shell ๐Ÿ‘ค awesome-spark Organization ๐Ÿ• Updated 2026-02-27 ๐Ÿ“… Created 2016-02-01
13
holdenk/spark-testing-base

Base classes to use when writing tests with Spark

holdenk
โญ 1,555 Scala ๐Ÿ‘ค holdenk User ๐Ÿ• Updated 2026-10-31 ๐Ÿ“… Created 2015-01-30
14
japila-books/apache-spark-internals

The Internals of Apache Spark

japila-books
โญ 1,553 ๐Ÿ‘ค japila-books Organization ๐Ÿ• Updated 2026-10-10 ๐Ÿ“… Created 2015-08-31
15
aws-samples/aws-glue-samples

AWS Glue code samples

aws-samples
โญ 1,538 Python ๐Ÿ‘ค aws-samples Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2017-05-21
16
sryza/aas

Code to accompany Advanced Analytics with Spark from O'Reilly Media

sryza
โญ 1,531 Scala ๐Ÿ‘ค sryza User ๐Ÿ• Updated 2024-09-25 ๐Ÿ“… Created 2014-11-08
17
locationtech/geomesa

GeoMesa is a suite of tools for working with big geo-spatial data in a distributed fashion.

locationtech
โญ 1,499 Scala ๐Ÿ‘ค locationtech Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2013-09-30
โญ 1,435 ๐Ÿ‘ค anshlambagit User ๐Ÿ• Updated 2026-01-31 ๐Ÿ“… Created 2025-10-25
19
databricks/LearningSparkV2

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

databricks
โญ 1,396 Scala ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2025-01-28 ๐Ÿ“… Created 2019-02-10
20
spark-examples/pyspark-examples

Pyspark RDD, DataFrame and Dataset Examples in Python language

spark-examples
โญ 1,371 Python ๐Ÿ‘ค spark-examples Organization ๐Ÿ• Updated 2025-12-07 ๐Ÿ“… Created 2019-11-26
21
graphframes/graphframes

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

graphframes
โญ 1,213 Scala ๐Ÿ‘ค graphframes Organization ๐Ÿ• Updated 2026-11-12 ๐Ÿ“… Created 2016-01-20
22
youngwookim/awesome-hadoop

A curated list of amazingly awesome Hadoop and Hadoop ecosystem resources

youngwookim
โญ 1,113 ๐Ÿ‘ค youngwookim User ๐Ÿ• Updated 2024-05-07 ๐Ÿ“… Created 2014-07-08
23
JohnSnowLabs/spark-nlp-workshop

Public runnable examples of using John Snow Labs' NLP for Apache Spark.

JohnSnowLabs
โญ 1,089 Jupyter Notebook ๐Ÿ‘ค JohnSnowLabs Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2018-08-20
24
mahmoudparsian/data-algorithms-book

MapReduce, Spark, Java, and Scala for Data Algorithms Book

mahmoudparsian
โญ 1,083 Java ๐Ÿ‘ค mahmoudparsian User ๐Ÿ• Updated 2024-10-14 ๐Ÿ“… Created 2014-08-06
โญ 1,064 ๐Ÿ‘ค CodementorIO Organization ๐Ÿ• Updated 2024-01-31 ๐Ÿ“… Created 2015-05-26
26
sparklyr/sparklyr

R interface for Apache Spark

sparklyr
โญ 970 R ๐Ÿ‘ค sparklyr Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2016-05-20
27
dbpedia/extraction-framework

The software used to extract structured data from Wikipedia

dbpedia
โญ 950 Scala ๐Ÿ‘ค dbpedia Organization ๐Ÿ• Updated 2026-11-13 ๐Ÿ“… Created 2013-02-07
28
microsoft/Mobius

C# and F# language binding and extensions to Apache Spark

microsoft
โญ 948 C# ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2025-12-11 ๐Ÿ“… Created 2015-10-27
29
aloneguid/parquet-dotnet

Fully managed Apache Parquet implementation

aloneguid
โญ 912 C# ๐Ÿ‘ค aloneguid User ๐Ÿ• Updated 2026-10-26 ๐Ÿ“… Created 2019-12-12
30
pyspark-ai/pyspark-ai

English SDK for Apache Spark

pyspark-ai
โญ 879 Python ๐Ÿ‘ค pyspark-ai Organization ๐Ÿ• Updated 2026-10-27 ๐Ÿ“… Created 2023-04-27
31
LucaCanali/sparkMeasure

This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simplifies collecting, aggregating, and exporting Spark task/stage metrics, and is designed for practical use by developers and data engineers in interactive analysis, testing, and production monitoring workflows.

LucaCanali
โญ 831 Scala ๐Ÿ‘ค LucaCanali User ๐Ÿ• Updated 2026-10-16 ๐Ÿ“… Created 2017-03-16
32
awslabs/python-deequ

Python API for Deequ

awslabs
โญ 826 Jupyter Notebook ๐Ÿ‘ค awslabs Organization ๐Ÿ• Updated 2026-10-24 ๐Ÿ“… Created 2020-11-09
33
databricks/tensorframes

[DEPRECATED] Tensorflow wrapper for DataFrames on Apache Spark

databricks
โญ 745 Scala ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2024-07-30 ๐Ÿ“… Created 2016-03-04
34
databricks/devrel

This repository contains the notebooks and presentations we use for our Databricks Tech Talks "

databricks
โญ 734 HTML ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2025-01-06 ๐Ÿ“… Created 2020-04-03
35
muaimingjun/bigData

Big Data Competition Project Library

muaimingjun
โญ 734 Python ๐Ÿ‘ค muaimingjun User ๐Ÿ• Updated 2025-05-07 ๐Ÿ“… Created 2019-10-14
36
awslabs/aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.

awslabs
โญ 701 Python ๐Ÿ‘ค awslabs Organization ๐Ÿ• Updated 2026-10-30 ๐Ÿ“… Created 2017-08-11
37
mrpowers-io/quinn

pyspark methods to enhance developer productivity ๐Ÿ“ฃ ๐Ÿ‘ฏ ๐ŸŽ‰

mrpowers-io
โญ 691 Python ๐Ÿ‘ค mrpowers-io Organization ๐Ÿ• Updated 2026-10-25 ๐Ÿ“… Created 2017-09-15
38
databricks/reference-apps

Spark reference applications

databricks
โญ 652 Scala ๐Ÿ‘ค databricks Organization ๐Ÿ• Updated 2024-10-03 ๐Ÿ“… Created 2014-08-19
39
nchammas/flintrock

A command-line tool for launching Apache Spark clusters.

nchammas
โญ 651 Python ๐Ÿ‘ค nchammas User ๐Ÿ• Updated 2024-12-13 ๐Ÿ“… Created 2015-06-04
40
infoslack/awesome-kafka

A list about Apache Kafka

infoslack
โญ 595 ๐Ÿ‘ค infoslack User ๐Ÿ• Updated 2026-05-05 ๐Ÿ“… Created 2016-04-29
41
openscoring/openscoring

REST web service for the true real-time scoring (<1 ms) of Scikit-Learn, R and Apache Spark models

openscoring
โญ 587 Java ๐Ÿ‘ค openscoring Organization ๐Ÿ• Updated 2026-02-02 ๐Ÿ“… Created 2013-04-02
42
spark-examples/spark-scala-examples

This project provides Apache Spark SQL, RDD, DataFrame and Dataset examples in Scala language

spark-examples
โญ 567 Scala ๐Ÿ‘ค spark-examples Organization ๐Ÿ• Updated 2024-03-20 ๐Ÿ“… Created 2019-11-26
43
cas-bigdatalab/piflow

ฯ€flow is a big data flow engine with spark support

cas-bigdatalab
โญ 541 Python ๐Ÿ‘ค cas-bigdatalab Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2018-05-03
44
LearningJournal/Spark-Programming-In-Python

Apache Spark 3 - Spark Programming in Python for Beginners

LearningJournal
โญ 510 Python ๐Ÿ‘ค LearningJournal User ๐Ÿ• Updated 2024-07-25 ๐Ÿ“… Created 2020-05-31

Frequently Asked Questions

1

What is the most popular #apache-spark repository on GitHub?

The top ranked open source #apache-spark project on GitHub is mlflow/mlflow with 28,920 stars. The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

2

Which programming languages are most used in #apache-spark repositories?

Among the top ranked #apache-spark projects on GitHub, the most common programming languages are Scala, Python, Java, Jupyter Notebook.

3

How is the #apache-spark repository ranking calculated?

We track 44 active open source repositories tagged with #apache-spark (500+ stars), totaling 97,788 stars with an average of 2,222 stars per repository. Rankings update daily from the GitHub API.