#data Repositories
Discover 232 active open source GitHub repositories tagged with #data, accumulating 1,106,761 total stars (averaging 4,770 stars per project). Leading this topic is D4Vinci/Scrapling with 104,760 stars.
232 repositories total
๐ท๏ธ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shyโjoin here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
The 30 Days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days. This challenge may take more than 100 days. Follow your own pace. These videos may help too: https://www.youtube.com/channel/UC7PNRuno1rzYPb1xLa4yktw
๐ค Powerful asynchronous state management, server-state utilities and data fetching for the web. TS/JS, React Query, Solid Query, Svelte Query and Vue Query.
The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
This is a repo with links to everything you'd ever want to learn about data engineering
๐ SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.
AKShare is an elegant and simple financial data interface library for Python, built for human beings! Open source financial data interface library
Open-source data movement for ELT pipelines and AI agents โ from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
A curated list of awesome big data frameworks, ressources and other awesomeness.
PRQL is a modern language for transforming data โ a simple, powerful, pipelined SQL replacement
:card_index: A simple fake data generator for C#, F#, and VB.NET. Based on and ported from the famed faker.js.
A web interface to create custom vector-based visualizations on top of RAWGraphs core
๐ง Build, run, and manage data pipelines for integrating and transforming data.
[SIGMOD'27] Easy Data Preparation with the Latest LLM-Based Operators and Pipelines.
Data processing for and with foundation models! ๐ ๐ ๐ฝ โก๏ธ โก๏ธ๐ธ ๐น ๐ท
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
The home of the CUE language! Validate and define text-based and dynamic configuration
data load tool (dlt) is an open source Python library that makes data loading easy ๐ ๏ธ
A next-generation curated knowledge sharing platform for data scientists and other technical professions.
Edit Banana: A framework for converting statistical formats into editable.
๐ Glide Data Grid is a no compromise, outrageously fast react data grid with rich rendering, first class accessibility, and full TypeScript support.
Superduper: End-to-end framework for building custom AI applications and agents.
CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.
Mimesis is a Python library for generating fake but realistic data in multiple languages and locales.
DeepAnalyze is the first agentic LLM for autonomous data science. ๐ Your AI data analyst automatically analyzes massive amounts of data and generates professional analysis reports with a single click!
Smarter YAML front matter parser, used by metalsmith, Gatsby, Netlify, Assemble, mapbox-gl, phenomic, vuejs vitepress, TinaCMS, Shopify Polaris, Ant Design, Astro, hashicorp, garden, slidev, saber, sourcegraph, and many others. Simple to use, and battle tested. Parses YAML by default but can also parse JSON Front Matter, Coffee Front Matter, TOML Front Matter, and has support for custom parsers. Please follow gray-matter's author: https://github.com/jonschlinkert
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
Preswald is a WASM packager for Python-based interactive data apps: bundle full complex data workflows, particularly visualizations, into single files, runnable completely in-browser, using Pyodide, DuckDB, Pandas, and Plotly, Matplotlib, etc. Build dashboards, reports, and notebooks that run offline, load fast, and share like a document.
A framework for collecting and analyzing prediction market data, including the largest publicly available dataset of Polymarket and Kalshi market and trade data.
Frequently Asked Questions
What is the most popular #data repository on GitHub?
The top ranked open source #data project on GitHub is D4Vinci/Scrapling with 104,760 stars. ๐ท๏ธ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shyโjoin here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Which programming languages are most used in #data repositories?
Among the top ranked #data projects on GitHub, the most common programming languages are Python, TypeScript, JavaScript, Rust.
How is the #data repository ranking calculated?
We track 232 active open source repositories tagged with #data (500+ stars), totaling 1,106,761 stars with an average of 4,770 stars per repository. Rankings update daily from the GitHub API.