#interpretability Repositories
Discover 39 active open source GitHub repositories tagged with #interpretability, accumulating 134,415 total stars (averaging 3,446 stars per project). Leading this topic is shap/shap with 25,883 stars.
39 repositories total
A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.
A curated list of awesome responsible machine learning resources.
๐ Shapash: User-friendly Explainability and Interpretability to Develop Reliable and Transparent Machine Learning Models
Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM, Finer-CAM, LeGrad, RefineCAM)
Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results
A JAX research toolkit for building, editing, and visualizing neural networks.
Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.
Interpretable ML package ๐ for concise, transparent, and accurate predictive modeling (sklearn-compatible).
The nnsight package enables interpreting and manipulating the internals of deep learned models.
Stanford NLP Python library for understanding and improving PyTorch models via interventions
A collection of anomaly detection methods (iid/point-based, graph and time series) including active learning for anomaly detection/discovery, bayesian rule-mining, description for diversity/explanation/interpretability. Analysis of incorporating label feedback with ensemble and tree-based detectors. Includes adversarial attacks with Graph Convolutional Network.
A computer, compiled into the weights of a language model. An exact CPU inside Qwen3.5-4B, with Gated DeltaNet state as RAM.
Recent high-citation/top conference papers on neural network model interpretability in deep learning (with code)
Inspect, visualize, and intervene in any operation in any neural network. Capture every activation and gradient, the full computational graph, and 800+ metadata fields without changing model code. Supports PyTorch, JAX, TensorFlow, MLX, PaddlePaddle, and tinygrad. Validated on 6,400+ models. Built for researchers and their AI agents.
A collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models in Keras.
[JMLR 2023] Quantus is an eXplainable AI toolkit for responsible evaluation of neural network explanations
A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.
Examples of techniques for training interpretable ML models, explaining ML models, and debugging ML models for accuracy, discrimination, and security.
๐ญ Aspect-Based-Sentiment-Analysis: Transformer & Explainable ML (TensorFlow)
Frequently Asked Questions
What is the most popular #interpretability repository on GitHub?
The top ranked open source #interpretability project on GitHub is shap/shap with 25,883 stars. A game theoretic approach to explain the output of any machine learning model
Which programming languages are most used in #interpretability repositories?
Among the top ranked #interpretability projects on GitHub, the most common programming languages are Python, Jupyter Notebook, C++, TypeScript.
How is the #interpretability repository ranking calculated?
We track 39 active open source repositories tagged with #interpretability (500+ stars), totaling 134,415 stars with an average of 3,446 stars per repository. Rankings update daily from the GitHub API.