Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Hadoop Open Source Projects

Open source projects categorized as Python Hadoop
Spark ⭐ 36,821
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 903total releases 46most recent commit 6 hours ago
Data Science Ipython Notebooks ⭐ 25,242
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
most recent commit 3 months ago
Luigi ⭐ 16,816
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
dependent packages 71total releases 79most recent commit 2 days agopypi luigi} Downloads
Deeplearning4j ⭐ 13,157
Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.
dependent packages 110total releases 53most recent commit 15 hours ago
It_book ⭐ 8,543
本项目收藏这些年来看过或者听过的一些不错的常用的上千本书籍，没准你想找的书就在这里呢，包含了互联网行
most recent commit 2 years ago
School Of Sre ⭐ 7,352
At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.
most recent commit 8 days ago
H2o 3 ⭐ 6,489
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
dependent packages 32total releases 241most recent commit 5 hours ago
Bigdl ⭐ 4,396
Accelerating LLM with low-bit (INT3 / INT4 / NF4 / INT5 / INT8) optimizations using bigdl-llm
dependent packages 10total releases 16most recent commit 6 hours ago
Tensorflowonspark ⭐ 3,851
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
total releases 32most recent commit 3 months ago
Ibis ⭐ 3,148
The flexibility of Python with the scale and performance of modern SQL.
dependent packages 24total releases 48most recent commit 6 hours agopypi ibis-framework} Downloads
Mrjob ⭐ 2,584
Run MapReduce jobs on Hadoop or Amazon Web Services
dependent packages 2total releases 62most recent commit a year agopypi mrjob} Downloads
Ambari ⭐ 1,991
Apache Ambari simplifies provisioning, managing, and monitoring of Apache Hadoop clusters.
total releases 2most recent commit 5 days ago
Studybooks ⭐ 999
我的学习资料，包括书籍、网址等
most recent commit 9 years ago
Coding Now ⭐ 925
学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、
most recent commit a year ago
Snakebite ⭐ 854
A pure python HDFS client
dependent packages 12total releases 63most recent commit a year agopypi snakebite} Downloads
Cdap ⭐ 719
An open source framework for building data analytic applications.
dependent packages 26total releases 62most recent commit 5 days ago
Devops Python Tools ⭐ 675
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
most recent commit a month ago
Flintrock ⭐ 628
A command-line tool for launching Apache Spark clusters.
total releases 13most recent commit 8 days ago
Dist Keras ⭐ 611
Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.
total releases 2most recent commit 5 years ago
Aws Glue Libs ⭐ 568
AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
most recent commit 2 months ago
Bigtop ⭐ 532
Bigtop is an Apache Foundation project for Infrastructure Engineers and Data Scientists looking for comprehensive packaging, testing, and configuration of the leading open source big data components.
most recent commit 2 days ago
Tuiblogs ⭐ 443
优秀的计算机编程类博客和文章 share excellent blogs and sites
most recent commit 5 years ago
Gather Deployment ⭐ 347
Gathers Python deployment, infrastructure and practices.
most recent commit 2 months ago
The 10 Most Depended On Python Hadoop Open Source Projects
Spark ⭐ 36,821
Apache Spark - A unified analytics engine for large-scale data processing
dependent packages 903total releases 46latest release May 09, 2021most recent commit 6 hours ago
Cdap ⭐ 719
An open source framework for building data analytic applications.
dependent packages 26total releases 62latest release November 30, 2018most recent commit 5 days ago
Ibis ⭐ 3,148
The flexibility of Python with the scale and performance of modern SQL.
dependent packages 24total releases 48latest release August 13, 2023most recent commit 6 hours agopypi ibis-framework} Downloads
Qds Sdk Py ⭐ 51
Python SDK for accessing Qubole Data Service
dependent packages 13total releases 69latest release October 06, 2020most recent commit 6 months agopypi qds_sdk} Downloads
Snakebite ⭐ 854
A pure python HDFS client
dependent packages 12total releases 63latest release August 30, 2016most recent commit a year agopypi snakebite} Downloads
Bigdl ⭐ 4,396
Accelerating LLM with low-bit (INT3 / INT4 / NF4 / INT5 / INT8) optimizations using bigdl-llm
dependent packages 10total releases 16latest release April 19, 2021most recent commit 6 hours ago
Skein ⭐ 126
A tool and library for easily deploying applications on Apache YARN
dependent packages 6total releases 23latest release April 05, 2022most recent commit a year agopypi skein} Downloads
The 10 Latest Releases In Python Hadoop Open Source Projects
Ibis ⭐ 3,148
The flexibility of Python with the scale and performance of modern SQL.
dependent packages 24total releases 48latest release August 13, 2023most recent commit 6 hours agopypi ibis-framework} Downloads
Aut ⭐ 128
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
total releases 25latest release June 17, 2022most recent commit 3 months ago
Tensorflowonspark ⭐ 3,851
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
total releases 32latest release April 21, 2022most recent commit 3 months ago
Skein ⭐ 126
A tool and library for easily deploying applications on Apache YARN
dependent packages 6total releases 23latest release April 05, 2022most recent commit a year agopypi skein} Downloads
Tf Yarn ⭐ 86
Train TensorFlow models on YARN in just a few lines of code!
total releases 18latest release March 17, 2022most recent commit 4 months ago
Hadoop Yarn Api Python Client ⭐ 99
Python client for Hadoop® YARN API
dependent packages 1total releases 21latest release November 16, 2021most recent commit 2 years agopypi yarn-api-client} Downloads
Flintrock ⭐ 628
A command-line tool for launching Apache Spark clusters.
total releases 13latest release June 13, 2021most recent commit 8 days ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Classification
Data Science
Scraper
Transaction
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.