#hadoop Repositories
Discover 43 active open source GitHub repositories tagged with #hadoop, accumulating 238,277 total stars (averaging 5,541 stars per project). Leading this topic is donnemartin/data-science-ipython-notebooks with 28,776 stars.
43 repositories total
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
๐ Real-Time no-code, powerful and secure ORM ๐ providing APIs and Docs without coding by Backend, and Frontend(Client) can customize response JSONs ๐ Real-time zero-code fully functional highly secure ORM library ๐ Backend APIs and documentation are zero-code, while Frontend(client) can customize returned JSON data and structure.
Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
1200+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..
At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
Big Data Learning, Learn Big Data from Scratch, Includes Learning Videos and Interview Materials for Each Stage of Big Data Learning
Historical learning project: Spark movie recommendations, web scraping, web app, and backend. Ongoing development of the new version: cinematch-recommender.
Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop
The Big Data knowledge repository involves data warehouse modeling, real-time computing, Big Data, data middle platform, system design, Java, algorithms, etc.
Actively maintained successor to Alibaba DataX โ a fast, versatile, open-source ETL tool for 30+ RDBMS and NoSQL data sources.
50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak
Scalable, reliable, distributed storage system optimized for data analytics and object store workloads.
Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display
450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...
Regularly update documentation for commonly used big data components in the Hadoop ecosystem. Priorities in order: Flink, Solr, Sparksql, ES, Scala, Kafka, Hbase/phoenix, Redis, Kerberos (Project includes Hadoop mind map, Evernote notes, simple Scala version demo, common utility classes, desensitized train code, continuous updates!!!)
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
This system is my graduation design project, titled "Design and Implementation of a Movie Recommendation System Based on User Profiles". It primarily uses Django as the base framework, adopting the MTV pattern, with MongoDB, MySQL, and Redis as databases. The movie data scraped from the Douban platform serves as the primary data source. User tags are mainly developed based on basic user information and behavioral information such as usage operation records. It is a recommendation system that utilizes Hadoop and Spark big data components for analysis and processing. The management system uses Django's built-in management system, enhanced with SimpleUI.
E-commerce Big Data Real-time Product Recommendation System, Vue + TypeScript + ElementUI + Spring + Spark
Frequently Asked Questions
What is the most popular #hadoop repository on GitHub?
The top ranked open source #hadoop project on GitHub is donnemartin/data-science-ipython-notebooks with 28,776 stars. Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
Which programming languages are most used in #hadoop repositories?
Among the top ranked #hadoop projects on GitHub, the most common programming languages are Java, Python, Shell, Scala.
How is the #hadoop repository ranking calculated?
We track 43 active open source repositories tagged with #hadoop (500+ stars), totaling 238,277 stars with an average of 5,541 stars per repository. Rankings update daily from the GitHub API.