Awesome Open Source

Programming Languages

Search results for python mapreduce

119 search results found

Data Science Ipython Notebooks ⭐ 25,668

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Dev Setup ⭐ 5,802

macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

Dpark ⭐ 2,637

Python clone of Spark, a MapReduce alike framework in Python

Mrjob ⭐ 2,584

Run MapReduce jobs on Hadoop or Amazon Web Services

An open source framework for building data analytic applications.

Mincemeatpy ⭐ 467

Lightweight MapReduce in python

Tdigest ⭐ 332

t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark

Hadoopy ⭐ 244

Python MapReduce library written in Cython. Visit us in #hadoopy on freenode. See the link below for documentation and tutorials.

Appengine Mapreduce ⭐ 229

A library for running MapReduce jobs on App Engine

Juicy Bigdata ⭐ 162

🎉🎉🐳 Datawhale大数据处理导论教程 | 大数据技术方向的开篇课程🎉🎉

Cc Mrjob ⭐ 157

Demonstration of using Python to process the Common Crawl dataset with the mrjob framework

Pda_book ⭐ 154

Code Examples Data Science using Python

Data Algorithms With Spark ⭐ 151

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

Python Bigdata ⭐ 128

Data science and Big Data with Python

MapReduce with ZeroMQ

Fast & furious GroupBy operations for dask.array

Introtohadoopandmr__udacity_course ⭐ 103

🐘 Source code for assignments of Udacity course "Introduction to Hadoop and MapReduce"

Python Data Processing library

Spark With Python ⭐ 98

Fundamentals of Spark with Python (using PySpark), code examples

Big Data Engineering Coursera Yandex ⭐ 91

Big Data for Data Engineers Coursera Specialization from Yandex

Annotated Wikiextractor ⭐ 88

Simple Wikipedia plain text extractor with article link annotations and Hadoop support.

Solutions Google Compute Engine Cluster For Hadoop ⭐ 81

This sample app will get up and running quickly with a Hadoop cluster on Google Compute Engine. For more information on running Hadoop on GCE, read the papers at https://cloud.google.com/resources/.

Scalable RNA-seq analysis

A MapReduce job to explore blunders in chess games.

Efficient and scalable parallelism using the message passing interface (MPI) to handle big data and highly computational problems.

Pybigdata ⭐ 56

使用 python 操作大数据的各种组件

Social Graph Analysis ⭐ 56

Social Graph Analysis using Elastic MapReduce and PyPy

Parallel Algorithms in Python for Hadoop/Mapreduce

Prosto is a data processing toolkit radically changing how data is processed by heavily relying on functions and operations with functions - an alternative to map-reduce and join-groupby

Hadoop_vision ⭐ 49

Example code for "Web-Scale Computer Vision using MapReduce for Multimedia Data Mining"

Machine_learning_in_action_py3 ⭐ 44

Important book about the machine learning algorithms, and introduces the application of those who use these algorithms and tools, and how to use them in a real environment. This book and other books, behind the other books are long on machine learning theory knowledge, the book happened to be more discussion on how to use coded machine learning algorithms.

IPDC(InterPlanetary Distributed Computing) is the Distributed Computation service, A peer-to-peer hypermedia protocol to make the computation faster, open, and more scalable.

Telemetry Server ⭐ 41

Server for the Mozilla Telemetry project

Hanhan Spark Python ⭐ 40

Used Spark core python, Spark sql, Spark MLlib, Spark Streaming

Package for Apache Pig support in Sublime Text 2 and 3

Simple python Multiprocesses-Multithreads queue 简易Python多进程-多线程任务队列, 也能做简单的MapReduce, 自用性质,请勿用于生产环境

Pyxtension ⭐ 36

Pure Python extensions library that includes Scala-like streams, Json with attribute access syntax, and other common use stuff

Hadoop_exporter ⭐ 35

A hadoop exporter for prometheus, scrape hadoop metrics (including HDFS, YARN, MAPREDUCE, HBASE. etc.) from hadoop components jmx url.

Pyspark Algorithms ⭐ 33

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

Elastic MapReduce instance optimizer

Webarchive Indexing ⭐ 30

Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.

Compute Hadoop Java Python ⭐ 28

This software demonstrates one way to create and manage a cluster of Hadoop nodes running on Google Compute Engine.

Scipy2013 ⭐ 27

SciPy 2013 Data Processing Tutorial

Mongo Hive ⭐ 24

Load your MongoDB collection into Hive. Supports complex JSON structure.

Real_time_social_media_mining ⭐ 24

DevOps pipeline for Real Time Social/Web Mining

Redisgears Py ⭐ 23

RedisGears python client

Learn Hadoop And Spark ⭐ 22

This repository focuses on gathering and making a curated list resources to learn Hadoop for FREE.

Dancedeets Monorepo ⭐ 22

DanceDeets Codebase: The python server (with React.JS rendering), as well as the React Native mobile app (and their shared code)

Nonnegative matrix factorizations in MapReduce

Coursera Uw Machine Learning Clustering Retrieval ⭐ 21

Mining Frequent Pattern From Search History ⭐ 19

《大数据挖掘技术》@复旦课程项目，试图从搜狗实验室用户查询日志数据（2008）中找出搜索记录中有较高支持度关键词的频繁二项集 Hadoop 集群，并且用 Python 实现了 Parallel FP-Growth 算法中的三个 MapReduce 过程。

Ceteri Mapred ⭐ 19

MapReduce examples

Pymapreduce ⭐ 19

Simple MapReduce implementation in Python, for text file parallel processing

Spark And Mllib Projects ⭐ 18

This repository contains Spark, MLlib, PySpark and Dataframes projects

MapReduce Streaming TSQR Implementation

Hadoop Python Tutorial ⭐ 18

Exercises and examples developed for the Hadoop with Python tutorial

Hadoop And Swift Integration ⭐ 18

API to run Hadoop MapReduce programs over Swift

Disco Slct ⭐ 17

A mapreduce implementation of SLCT (http://ristov.users.sourceforge.net/slct/) using Disco.

Sf Python Meetup Sep 2013 ⭐ 17

Deliroll presentation given at the SF Python Meetup on September 11th 2013

Mongo Bigquery ⭐ 16

Load your MongoDB collection into Google BigQuery. Supports complex JSON structure.

Cs205_ga ⭐ 16

How deep does Google Analytics go? Efficiently tackling Common Crawl using AWS & MapReduce

mrlin is 'MapReduce processing of Linked Data' … because it's magic

Hadoop Mapreduce ⭐ 16

Collection of example and notes on Hadoop and Map Reduce

There are Python 2.7 codes and learning notes for Spark 2.1.1

Pairwise Mapreduce ⭐ 15

Implementation of a pairwise document similarity algorithm using MapReduce.

Bigquery Appengine Datastore Import Sample ⭐ 15

Demonstrates how to extract and transform data from the Datastore into a format suitable for ingestion by Google BigQuery, via the App Engine MapReduce library.

Udacity_hadoop_intro ⭐ 15

Notes and tasks code for Cloudera / Udacity hadoop course

spark (scala and python)

Rasppi Cluster ⭐ 14

An efficient quick-start tool to build a Raspberry Pi (or Debian-based) Cluster with popular ecosystem like Hadoop, Spark

Intro To Hadoop Mapreduce ⭐ 14

Twittercommunitydetection ⭐ 14

Community Detection for Twitter follower network of 40 million users using mapreduce

Hadoop Mapreduce Python Example ⭐ 13

Map Reduce example for Hadoop in Python based on Udacity: Intro to Hadoop and MapReduce

Mirror of YSmart

Yarn Memory Calculator ⭐ 12

Hadoop YARN & MapReduce Memory Calculator

Python Performance ⭐ 12

Repository for the book Fast Python - published by Manning

Fdic Call Reports ⭐ 11

Tools for analysis and review of FDIC call report data

Emr Demo ⭐ 10

Project files for the post: Running PySpark Applications on Amazon EMR: Methods for Interacting with PySpark on Amazon Elastic MapReduce.

Dijkstra Hadoop Spark ⭐ 10

Dijkstra Algorithm - Python Hadoop Streaming and Pyspark

Sentiment Analysis ⭐ 10

Distributed sentiment analysis on GitHub commit comments

Pyrallel ⭐ 10

Yet another easy-to-use python3 parallel library for humans.

Extra-light API for using Hadoop with Python

Cloudstoragelineinputreader ⭐ 10

An appengine-mapreduce line input reader for Google Cloud Storage

Ascii_dna_translator ⭐ 9

DNA as an information storage medium using either 4-base codon or binary representations of 256-ASCII

Mapreduce ⭐ 9

MapReduce scripts written in Python for Hadoop Streaming

Tf Idf Implementation Using Map Reduce Hadoop Python ⭐ 9

Academicrecommendation_dlut ⭐ 9

面向大连理工大学学者，基于大连理工大学机构知识库，推荐潜在主题词，优化科研方向，促进跨学部、学科、领

Demonstration of using Python to process the Common Crawl dataset with the mrjob framework

Appengine Mapreduce Utils ⭐ 9

Extra utils for http://code.google.com/p/appengine-mapreduce/

Hadoop Mapreduce Examples Python ⭐ 8

All the Hadoop Mapreduce examples in python!

Common_crawl ⭐ 8

Simple Python MapReduce jobs for processing the Common Crawl plus command-line utilities

Implementation Of Mapreduce Algorithms Using A Simple Python Mapreduce Framework ⭐ 8

Implements common data processing tasks such as creation of an inverted index, performing a relational join, multiplying sparse matrices and dna-sequence trimming using a simple MapReduce model, on a single machine in python.

Shepherdpy ⭐ 8

A companion library for mincemeatpy that will manage MapReduce clients.

Hadoop And Mapreduce ⭐ 8

Udacity-Intro to Hadoop and MapReduce-Part 1

Inforetrieval ⭐ 8

Inverted Indexer, web crawler, sort, search and poster steamer written using Python for information retrieval.

Python2 Course ⭐ 8

Python2.7教程

Wiki_pagerank ⭐ 7

Playing with the wikipedia link graph

Airflow Plugin ⭐ 7

Plugin for Apache Airflow to execute serverless tasks using Lithops

Vector Space Model Of Information Retrieval ⭐ 7

Search Engine in MapReduce framework

Canopyclusteringpython ⭐ 7

Canopy Clustering using MapReduce [Hadoop]

Suite for phased clusters discovery, comparison, annotation and to identify miRNA triggers - uses MapReduce model [In review]

Related Searches

Python Script (17,004)

Python Dataset (14,792)

Python Machine Learning (14,099)

Python Tensorflow (13,736)

Python Deep Learning (13,092)

Python Jupyter Notebook (12,976)

Python Algorithms (10,033)

Python Amazon Web Services (7,946)

Python Keras (6,821)

Python Pandas (6,193)

1-100 of 119 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.