#data Repositories
232 repositories total
[NeurIPS 2026] DataFlex: A Unified Benchmark and Evaluation Platform for Data-Centric Training of Large Language Models
The Data Transfer Project makes it easy for platforms to build interoperable user data portability features. We are establishing a common framework, including data models and protocols, to enable direct transfer of data both into and out of participating online service providers.
AnyCrawl ๐: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
A curated list of awesome JSON datasets that don't require authentication.
A simple, declarative, and composable way to fetch data for React components
Extract data from a wide range of Internet sources into a pandas DataFrame.
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
QuantMind is an open source agent-native knowledge extraction and retrieval framework for quantitative finance.
Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.
Deepnote is a drop-in replacement for Jupyter with an AI-first design, sleek UI, new blocks, and native data integrations. Use Python, R, and SQL locally in your favorite IDE, then scale to Deepnote cloud for real-time collaboration, Deepnote agent, and deployable data apps. https://deepnote.com/
A well tested and comprehensive Golang statistics library package with no dependencies.
A list of publicly available datasets with real-time data maintained by the team at bytewax.io
PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.
๐ 7,400,000+ Unsplash images made available for research and machine learning
Book_6_"The Way of Data" | Iris Book: From Addition, Subtraction, Multiplication, and Division to Machine Learning; we welcome criticism and corrections! Students who provide many corrections will receive a free book as a thank you!
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.
Jupyter extensions that help you write code faster: Context aware AI Chat, Autocomplete, and Spreadsheet
Malloy is a modern open source language for describing data relationships and transformations.
Python library for creating data pipelines with chain functional programming
ISO 3166-1 country lists merged with their UN Geoscheme regional codes in ready-to-use JSON, XML, CSV data sets
LLM based data scientist, AI native data application. AI-driven infinite thinking redefines BI.
๐ ๐จ๐ณ China Statutory Holidays Data Automatically Fetched Daily from State Council Announcements
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
Python's nested data operator (and CLI), for all your declarative restructuring needs. Got data? Glom it! โ๏ธ
Open Public Domain Exercise Dataset in JSON format, over 800 exercises with a browsable public searchable frontend
๐ A comprehensive list of open-source datasets for voice and sound computing (95+ datasets).
The best place to learn data engineering. Built and maintained by the data engineering community.
๐พ nao is the #1 open-source agentic BI. (1) Create context with the nao-core CLI, (2) deploy the nao chat interface for everyone.
Zui is a powerful desktop application for exploring and working with data. The official front-end to the Zed lake.
The AI Datastore for Schemas, BLOBs, and Predictions. Use with your apps or integrate built-in Human Supervision, Data Workflow, and UI Catalog to get the most value out of your AI Data.