Git Stars Git Stars

#inference Repositories

144
Total Repositories
870,238
Total Stars
6,043
Avg Stars/Repo
101
OpenIntroStat/ims

๐Ÿ“š Introduction to Modern Statistics - A college-level open-source textbook with a modern approach highlighting multivariable relationships and simulation-based inference.

OpenIntroStat
โญ 957 JavaScript ๐Ÿ‘ค OpenIntroStat Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2015-07-13
102
alvarobartt/hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

alvarobartt
โญ 956 Python ๐Ÿ‘ค alvarobartt User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2025-12-22
103
uber/neuropod

A uniform interface to run deep learning models from multiple frameworks

uber
โญ 941 C++ ๐Ÿ‘ค uber Organization ๐Ÿ• Updated 2024-01-03 ๐Ÿ“… Created 2019-01-23
104
TrenTorch/TrenTorch

Don't memorize ML. Understand it from first principles, including inference, CUDA, deep learning, data science, and reinforcement learning.

TrenTorch
โญ 867 Python ๐Ÿ‘ค TrenTorch Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-08-21
105
PABannier/bark.cpp

Suno AI's Bark model in C/C++ for fast text-to-speech generation

PABannier
โญ 850 C++ ๐Ÿ‘ค PABannier User ๐Ÿ• Updated 2024-11-16 ๐Ÿ“… Created 2023-07-01
106
triton-inference-server/pytriton

PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.

triton-inference-server
โญ 847 Python ๐Ÿ‘ค triton-inference-server Organization ๐Ÿ• Updated 2025-08-13 ๐Ÿ“… Created 2022-11-08
107
cyrusbehr/tensorrt-cpp-api

TensorRT C++ API Tutorial

cyrusbehr
โญ 813 C++ ๐Ÿ‘ค cyrusbehr User ๐Ÿ• Updated 2026-05-30 ๐Ÿ“… Created 2021-10-25
108
fudan-generative-vision/Remory

Remory: Learning Residual Memory for Context Compaction

fudan-generative-vision
โญ 809 Python ๐Ÿ‘ค fudan-generative-vision Organization ๐Ÿ• Updated 2026-11-09 ๐Ÿ“… Created 2026-10-07
109
serizba/cppflow

Run TensorFlow models in C++ without installation and without Bazel

serizba
โญ 809 C++ ๐Ÿ‘ค serizba User ๐Ÿ• Updated 2024-08-16 ๐Ÿ“… Created 2019-05-16
110
SharpAI/SwiftLM

โšก Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

SharpAI
โญ 802 Swift ๐Ÿ‘ค SharpAI Organization ๐Ÿ• Updated 2026-11-19 ๐Ÿ“… Created 2026-03-21
111
pipeless-ai/pipeless

An open-source computer vision framework to build and deploy apps in minutes

pipeless-ai
โญ 794 Rust ๐Ÿ‘ค pipeless-ai Organization ๐Ÿ• Updated 2024-05-08 ๐Ÿ“… Created 2023-07-21
112
emlearn/emlearn

Machine Learning inference engine for Microcontrollers and Embedded devices

emlearn
โญ 776 Python ๐Ÿ‘ค emlearn Organization ๐Ÿ• Updated 2026-11-10 ๐Ÿ“… Created 2018-03-16
113
rnchg/APT

AI Productivity Tool - Free and open source, improve user productivity, and protect privacy and data security. Including but not limited to: built-in local exclusive ChatGPT, DeepSeek, Phi, Qwen and other models, one-click batch intelligent processing of pictures, videos, audio, etc.

rnchg
โญ 768 C# ๐Ÿ‘ค rnchg User ๐Ÿ• Updated 2025-12-13 ๐Ÿ“… Created 2024-06-19
114
h2non/filetype.py

Small, dependency-free, fast Python package to infer binary file types checking the magic numbers signature

h2non
โญ 766 Python ๐Ÿ‘ค h2non User ๐Ÿ• Updated 2025-05-02 ๐Ÿ“… Created 2016-06-18
115
stamparm/identYwaf

Blind WAF identification tool

stamparm
โญ 758 Python ๐Ÿ‘ค stamparm User ๐Ÿ• Updated 2026-09-10 ๐Ÿ“… Created 2019-01-10
116
aws/studio-lab-examples

Example notebooks for working with SageMaker Studio Lab. Sign up for an account at the link below!

aws
โญ 753 Jupyter Notebook ๐Ÿ‘ค aws Organization ๐Ÿ• Updated 2024-08-24 ๐Ÿ“… Created 2021-11-23
117
pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine โ€” no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

pegainfer-project
โญ 748 Rust ๐Ÿ‘ค pegainfer-project Organization ๐Ÿ• Updated 2026-11-21 ๐Ÿ“… Created 2026-02-17
118
MegEngine/InferLLM

a lightweight LLM model inference framework

MegEngine
โญ 746 C++ ๐Ÿ‘ค MegEngine Organization ๐Ÿ• Updated 2024-04-07 ๐Ÿ“… Created 2023-05-04
119
hidet-org/hidet

An open-source efficient deep learning framework/compiler, written in python.

hidet-org
โญ 742 Python ๐Ÿ‘ค hidet-org Organization ๐Ÿ• Updated 2025-09-04 ๐Ÿ“… Created 2022-01-04
120
microsoft/vidur

Accurate, large-scale, and extensible simulator for LLM inference Systems

microsoft
โญ 733 Python ๐Ÿ‘ค microsoft Organization ๐Ÿ• Updated 2026-08-24 ๐Ÿ“… Created 2023-11-02
121
zhiqwang/yolort

yolort is a runtime stack for yolov5 on specialized accelerators such as tensorrt, libtorch, onnxruntime, tvm and ncnn.

zhiqwang
โญ 730 Python ๐Ÿ‘ค zhiqwang User ๐Ÿ• Updated 2026-04-10 ๐Ÿ“… Created 2020-08-14
122
intel/ai-reference-models

Intelยฎ AI Reference Models: contains Intel optimizations for running deep learning workloads on Intelยฎ Xeonยฎ Scalable processors and Intelยฎ Data Center GPUs

intel
โญ 728 Python ๐Ÿ‘ค intel Organization ๐Ÿ• Updated 2026-02-11 ๐Ÿ“… Created 2018-10-23
123
privacytrustlab/ml_privacy_meter

Privacy Meter: An open-source library to audit data privacy in statistical and machine learning algorithms.

privacytrustlab
โญ 715 Jupyter Notebook ๐Ÿ‘ค privacytrustlab Organization ๐Ÿ• Updated 2025-04-26 ๐Ÿ“… Created 2020-04-08
124
DerwenAI/kglab

Graph Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries โ€“ atop Pandas, NetworkX, RAPIDS, RDFlib, pySHACL, PyVis, morph-kgc, pslpython, pyarrow, etc.

DerwenAI
โญ 696 Jupyter Notebook ๐Ÿ‘ค DerwenAI Organization ๐Ÿ• Updated 2026-01-25 ๐Ÿ“… Created 2020-10-25
125
kossisoroyce/timber

Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference code. One command to load, one command to serve. 336x faster than Python inference.

kossisoroyce
โญ 692 Python ๐Ÿ‘ค kossisoroyce User ๐Ÿ• Updated 2026-04-16 ๐Ÿ“… Created 2026-02-27
126
zhongkaifu/TensorSharp

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, an iPhone app, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows, macOS, iOS, and Linux with full GPU capability.

zhongkaifu
โญ 681 C# ๐Ÿ‘ค zhongkaifu User ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-04-03
127
AntonMu/TrainYourOwnYOLO

Train a state-of-the-art yolov3 object detector from scratch!

AntonMu
โญ 673 Jupyter Notebook ๐Ÿ‘ค AntonMu User ๐Ÿ• Updated 2024-08-02 ๐Ÿ“… Created 2019-07-17
128
Nehanth/pooled

Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.

Nehanth
โญ 658 JavaScript ๐Ÿ‘ค Nehanth User ๐Ÿ• Updated 2026-11-03 ๐Ÿ“… Created 2026-09-01
129
xboot/libonnx

A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.

xboot
โญ 657 C ๐Ÿ‘ค xboot Organization ๐Ÿ• Updated 2026-07-07 ๐Ÿ“… Created 2020-10-14
130
open-edge-platform/dlstreamer

Deep Learning Streamer (DL Streamer) Pipeline Framework is an open-source streaming media analytics framework, based on GStreamer* multimedia framework, for creating complex media analytics pipelines for the Cloud or at the Edge.

open-edge-platform
โญ 648 C++ ๐Ÿ‘ค open-edge-platform Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2019-03-13
131
WaveSpeedAI/waverless

High-performance serverless GPU task orchestration โ€” the scheduling layer behind wavespeed.ai

WaveSpeedAI
โญ 635 Go ๐Ÿ‘ค WaveSpeedAI Organization ๐Ÿ• Updated 2026-08-21 ๐Ÿ“… Created 2025-10-16
132
therealoliver/Deepdive-llama3-from-scratch

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

therealoliver
โญ 629 Jupyter Notebook ๐Ÿ‘ค therealoliver User ๐Ÿ• Updated 2025-02-24 ๐Ÿ“… Created 2025-02-19
133
Rudrabha/LipGAN

This repository contains the codes for LipGAN. LipGAN was published as a part of the paper titled "Towards Automatic Face-to-Face Translation".

Rudrabha
โญ 617 Python ๐Ÿ‘ค Rudrabha User ๐Ÿ• Updated 2025-06-22 ๐Ÿ“… Created 2020-01-02
134
grimoire/mmdetection-to-tensorrt

convert mmdetection model to tensorrt, support fp16, int8, batch input, dynamic shape etc.

grimoire
โญ 598 Python ๐Ÿ‘ค grimoire User ๐Ÿ• Updated 2024-02-18 ๐Ÿ“… Created 2020-07-12
135
agentic-in/inferoa

Inference-native Tokenmaxxing Agent Harness for Loop Engineering

agentic-in
โญ 570 TypeScript ๐Ÿ‘ค agentic-in Organization ๐Ÿ• Updated 2026-06-18 ๐Ÿ“… Created 2026-06-08
136
spark-arena/sparkrun

sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD

spark-arena
โญ 567 Python ๐Ÿ‘ค spark-arena Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2026-02-16
137
Koldim2001/YOLO-Patch-Based-Inference

Python library for YOLO small object detection and instance segmentation

Koldim2001
โญ 562 Python ๐Ÿ‘ค Koldim2001 User ๐Ÿ• Updated 2026-04-22 ๐Ÿ“… Created 2024-03-04
138
dllm-labs/mojo-dllm

Local inference for diffusion language models (LLaDA, Dream), written in Mojo

dllm-labs
โญ 546 Python ๐Ÿ‘ค dllm-labs Organization ๐Ÿ• Updated 2026-11-20 ๐Ÿ“… Created 2026-10-04
139
kaito-project/aikit

๐Ÿ—๏ธ Fine-tune, build, and deploy open-source LLMs easily!

kaito-project
โญ 545 Go ๐Ÿ‘ค kaito-project Organization ๐Ÿ• Updated 2026-11-16 ๐Ÿ“… Created 2023-09-20
140
kubedl-io/kubedl

Run your deep learning workloads on Kubernetes more easily and efficiently.

kubedl-io
โญ 533 Go ๐Ÿ‘ค kubedl-io Organization ๐Ÿ• Updated 2024-03-04 ๐Ÿ“… Created 2019-12-10
141
triton-inference-server/model_analyzer

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

triton-inference-server
โญ 531 Python ๐Ÿ‘ค triton-inference-server Organization ๐Ÿ• Updated 2026-11-17 ๐Ÿ“… Created 2020-09-01
142
NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation

Deep learned, NVIDIA-accelerated 3D object pose estimation

NVIDIA-ISAAC-ROS
โญ 520 C++ ๐Ÿ‘ค NVIDIA-ISAAC-ROS Organization ๐Ÿ• Updated 2026-11-18 ๐Ÿ“… Created 2021-11-04
143
mithril-security/blindai

Confidential AI deployment with secure enclaves :lock:

mithril-security
โญ 512 Rust ๐Ÿ‘ค mithril-security Organization ๐Ÿ• Updated 2024-03-19 ๐Ÿ“… Created 2022-02-06
144
vectorch-ai/ScaleLLM

A high-performance inference system for large language models, designed for production environments.

vectorch-ai
โญ 502 C++ ๐Ÿ‘ค vectorch-ai Organization ๐Ÿ• Updated 2025-12-19 ๐Ÿ“… Created 2023-07-24