#spark Repositories
126 repositories total
Know your data better๏ผDatavines is Next-gen Data Observability Platform, support metadata manage and data quality.
This system is my graduation design project, titled "Design and Implementation of a Movie Recommendation System Based on User Profiles". It primarily uses Django as the base framework, adopting the MTV pattern, with MongoDB, MySQL, and Redis as databases. The movie data scraped from the Douban platform serves as the primary data source. User tags are mainly developed based on basic user information and behavioral information such as usage operation records. It is a recommendation system that utilizes Hadoop and Spark big data components for analysis and processing. The management system uses Django's built-in management system, enhanced with SimpleUI.
Pandas, Polars, Spark, and Snowpark DataFrame comparison for humans and more!
A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.
Cross-platform real-time collaboration client optimized for business and organizations.
A secondary development-ready Chat Bot single-turn dialogue Web MVP prototype template, built with mainstream technologies including Vue 3, Vite 8, TypeScript, Naive UI, Pinia (v3), and UnoCSS. ๐งค Features simple integration of LLM APIs, single-turn AI Q&A mode with independent responses (no context required), SSE typewriter-effect streaming output, and integrated markdown-it with Mermaid/KaTex/LaTex formula highlighting. Supports iFlytek Spark, Zhipu AI, SiliconFlow, and DeepSeek V4/V3/R1 deep reasoning model previews, compatible with <think> tags, and includes distilled skills. ๐ผ Easy to customize and rapidly build Chat-based LLM products (example screenshots included).
A simple spark standalone cluster for your testing environment purposes
Big Data Collection, Extraction Platform, zdh_web is the visual management platform for zdh series services, including modules such as data collection, scheduling, permissions, approval workflow, private domain marketing, etc.
Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.
E-commerce Big Data Real-time Product Recommendation System, Vue + TypeScript + ElementUI + Spring + Spark
A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with love by Data Scientists & Security Analysts from the Cyber Security Industry. [BETA]
Learn Apache Spark in Scala, Python (PySpark) and R (SparkR) by building your own cluster with a JupyterLab interface on Docker. :zap:
Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) can be used to generate large simulated/synthetic datasets for testing, POCs, and other uses in Databricks environments, including Delta Live Tables pipelines.