Awesome Open Source

Programming Languages

Search results for spark kafka

373 search results found

Movie Recommender Demo ⭐ 50

This project walks through how you can create recommendations using Apache Spark machine learning. There are a number of jupyter notebooks that you can run on IBM Data Science Experience, and there a live demo of a movie recommendation web application you can interact with. The demo also uses IBM Message Hub (kafka) to push application events to topic where they are consumed by a spark streaming job running on IBM BigInsights (hadoop).

Machine learning around business processes

Realtime Dashboard ⭐ 49

Real-time report dashboard with Apache Kafka, Apache Spark Streaming and Node.js

Sparkonkudu ⭐ 48

Based off the design of SparkOnHBase. This Repo will support Spark, Spark Streaming, and Spark SQL integration with Kudu.

Columbiaimagesearch ⭐ 44

Columbia Image and Face Search tool for MEMEX

Awesome Recommendation Engine ⭐ 43

The purpose of this tiny project is to put things together with the know how that i learned from the course big data expert from formacionhadoop.com The idea is to show how to play with apache spark streaming, kafka,mongo, spark machine learning algorithms.

Datashark ⭐ 41

dataShark is a Security & Network Event Analytics Framework built on Apache Spark

Hyperdrive ⭐ 41

Extensible streaming ingestion pipeline on top of Apache Spark

Realtime Data Analytics Using Spark ⭐ 39

Realtime social media data analytics with Apache Spark, Python, Kafka, Pandas, etc

Etl Light ⭐ 38

A light Kafka to HDFS/S3 ETL library based on Apache Spark

Awesome Druid ⭐ 38

Bigdata Getting Started ⭐ 37

大数据相关框架实战项目(Hadoop, Spark, Storm, Flink)

Xxhadoop ⭐ 37

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

Real Time Stream Processing Engine ⭐ 37

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

Spark Kafka 0 8 Sql ⭐ 36

Spark Structured Streaming Kafka 0.8 Source Implementation

Killranalytics ⭐ 35

Open source analytics platform powered by Apache Cassandra, Spark, and Kafka

Spark Utils ⭐ 35

💥 👽 ♨️🚀Encapsulated spark 与其他组件的结合api，方便使用，例如 es，hbase，kudu，kafka，mq等

Streaming Analytics platform, built with Apache Flink and Kafka

Apache Spark 2 For Beginners ⭐ 33

Apache Spark 2 for Beginners, published by Packt

Spark Twitter Sentiment Analysis ⭐ 33

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Engineeringteam ⭐ 32

와이빅타 엔지니어링팀의 자료를 정리해두는 곳입니다.

Dockerfiles ⭐ 31

Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )

Structuredstreaminginsql ⭐ 31

sql实现Structured Streaming

Framework Of Bigdata ⭐ 30

大数据面试题，从0到1走向架构师之路。Flink、Spark、Hive、HBase、Hadoop、K

Spark Streaming Direct Kafka ⭐ 30

High Performance Spark Streaming with Direct Kafka in Java

Kafka Spark Streaming Druid ⭐ 29

Takes a kafka stream into spark, apply transformations and sink into Druid. Everything Dockerised.

Real Time Data Warehouse ⭐ 29

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

Spark Secure Kafka App ⭐ 29

Sample Spark Streaming application for secure consumption from Kafka

Kafka Compose ⭐ 28

🎼 Docker compose files for various kafka stacks

Almaren Framework ⭐ 28

The Almaren Framework provides a simplified consistent minimalistic layer over Apache Spark. While still allowing you to take advantage of native Apache Spark features. You can still combine it with standard Spark code.

Rts_practice ⭐ 28

Adenium Normalizer

Iot_demo ⭐ 26

Demo showcasing Spark Streaming, Kafka, Kudu - all in Python

Interview Refresher Java Bigdata ⭐ 26

a one-stop repo to lookup for code snippets of core java concepts, sql, data structures as well as big data. It also consists of interview questions asked in real-life.

Kafka Spark Flink Example ⭐ 26

Kafka streaming with Spark and Flink example

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Cassandra.realtime ⭐ 25

Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink

Bigdata Doc ⭐ 25

大数据学习笔记，学习路线，技术案例整理。

Iot Truck Streaming ⭐ 25

Bookrecommendersystem ⭐ 25

基于大数据的图书推荐系统

Trafficteach ⭐ 25

大数据项目:车流量监控

Recommendsystem ⭐ 24

电影推荐系统

Open Stream Processing Benchmark ⭐ 24

This repository contains the code base for the Open Stream Processing Benchmark.

Open source data infrastructure platform. Designed for developers, built for speed.

Flight_delay_prediction_web_app ⭐ 23

A big data web application to predict USA airline traffic delay with Python, Flask, Apache Spark, Kafka, MongoDB, ElasticSearch, d3.js, scikit-learn, MLlib and Apache Airflow.

Marseille ⭐ 23

A real time streaming implementation of markov chain based fraud detection

Scala Kafka Twitter ⭐ 23

Example integration of Kafka, Avro & Spark-Streaming on live Twitter feed

Ozymandias ⭐ 23

Real-time image processing at scale using Kafka and Spark Streaming

Location Based Restaurants Recommendation System ⭐ 23

Big Data Management and Analysis Final Project

Kafka Spark Streaming ⭐ 22

Project for reading data from kafka and writing to kafka and HBase with kerberos

Fastdata Cluster ⭐ 22

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

Bigdata Tutorial ⭐ 22

Bigdatariver ⭐ 22

Simple demo implementation of Lambda and Kappa architectures using Python, Docker, Kafka, Spark and Cassandra

Mysql Time Machine ⭐ 22

mysql-time-machine project

Kafka Connect Msk Demo ⭐ 21

For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR

Spark And Kafka_iot Data Processing And Analytics ⭐ 21

Final Project for IoT: Big Data Processing and Analytics class in UCSC Extension. Analyzing U.S nationwide temperature from IoT sensors in real-time

Sparktwitterstreamanalysis ⭐ 21

A Spark Streaming App to analyze the popular hashtags based on keywords

Spark Kafka Streaming ⭐ 21

Custom Spark Kafka consumer based on Kafka SimpleConsumer API.

Sample Kafkasparkcassandra ⭐ 21

Introductory sample scala app using Apache Spark Streaming to accept data from Kafka and write a summary to Cassandra.

Streaminglogisticregression ⭐ 21

Logistic Regression in Spark Streaming with Online Updating

Tobetopgopher ⭐ 20

go的成神之路

Idocuments ⭐ 20

收集与 Java 开发相关的文档，包括基础系统服务（大数据、流计算、NoSQL 等）、专业名词、jar 包、开发工具等文档，持续更新……

Maelstrom ⭐ 20

Maelstrom is an open source Kafka integration with Spark that is designed to be developer friendly, high performance (millisecond stream processing), scalable (consumes messges at Spark worker nodes), and is extremely reliable.

Data Engineering Zoomcamp ⭐ 20

Data Engineering examples covering Airflow and Mage for workflows; dbt for BigQuery, Redshift, ClickHouse; Spark and Kafka for Batch/Streaming Processing

Eskimo is a state of the art Big Data Infrastructure and Management Web Console to build, manage and operate Big Data 2.0 Analytics clusters on Kubernetes. This is the git repository of Eskimo Community Edition.

Cda Client ⭐ 19

Cloud Data Access client

Spark Streaming With Google Cloud Example ⭐ 19

an example of integrating Spark Streaming with Google Pub/Sub and Google Datastore

A general purpose metrics monitor implemented with Apache Spark. Kafka source, Elastic sink, aggregate metrics, different analysis, notifications, actions, live configuration update, missing metrics, ...

Realtime Dashboard Example ⭐ 18

This is a real-time dashboard example using Spark Streaming and Node.js

Kubernetes 100days ⭐ 18

Notes from 100 days with Kubernetes

Edge Fuse ⭐ 18

Edge-X S3 API, mount S3 bucket for full POSIX read/write access

Bigdata Book ⭐ 18

上百本大数据电子书，附带下载链接，包括计算机基础，Java，hadoop，spark，flink，k

Data Pipeline Project ⭐ 18

Data pipeline project

Jun_bigdata ⭐ 18

jun_bigdata大数据平台服务框架。实现了Kafka实时数据过滤、清洗、转换、消费，实现了Sp SQL对Redis、MongoDB等非关系型数据库的数据的读写；集成了规则引擎，可基于规则引擎实现客

Spark Kafka Avro ⭐ 17

POC: Spark consumer for bottledwater-pg Kafka Avro topics

Kafka Streams ⭐ 17

Oh you know, just a coupla, two, tree Kafka Streams in Scala

Sparknlp Elasticsearch ⭐ 17

Twitter sentiment analysis using Spark and Stanford CoreNLP and visualization using elasticsearch and kibana

Divolte Examples ⭐ 17

Usage examples for Divolte collector

Qs Hadoop ⭐ 17

大数据生态圈学习

Bigdata Workshop Es ⭐ 17

Workshop Big Data en Español

Java_spark ⭐ 16

Spark 2.x 案例操作：Scala版本与 Java1.8lambda版代码示例。涵盖Spark核心技术操作SparkCore、SparkSql

Kafka Twitter Spark Streaming ⭐ 16

Counting Tweets Per User in Real-Time

Data Mill ⭐ 16

A K8s-based infrastructure for analytics

Real Time Stock Analyzer ⭐ 16

Bigdata Pipeline

Stream Processing Workshop ⭐ 15

Stream Processing Workshop

Log_analysis ⭐ 15

Practical Log Analysis

Visualizingstreamingdata ⭐ 15

Visuallizing a real time streaming data on web app using node.js socket.io and Apache Kafka

Real Time Twitter Sentiment Analysis Product

Tweet Analysis With Kafka And Spark ⭐ 15

A real time analytics dashboard to analyze the trending hashtags and @ mentions at any location using kafka and spark streaming.

Spark Sql Kafka Offset Committer ⭐ 15

Kafka offset committer for structured streaming query

Bigdataguider ⭐ 15

大数据学习笔记: 涉及hadoop,spark,hive,zookeeper等内容

小白大数据学习笔记 ⭐

Twitterstreamingsparkkafkademo ⭐ 15

a demo project to Analyze most popular twitter hashtags using Java 8 Spring-Boot Spark Streaming Kafka & Docker Demo.

A Big Data Platform Prototype Project

Live_twitter_sentiment_analysis ⭐ 14

Live Twitter sentiment analysis using Python, Apache Spark Streaming, Kafka, NLTK, SocketIO

Code repo for Pytheas (formally DDN), a control platform for enabling data-driven control for network applications

Minikube for big data with Scala and Spark

Mongo Kafka Spark ⭐ 14

A Realtime Analytics Engine using Kafka, Spark & MongoDB

Related Searches

Scala Spark (3,279)

Java Kafka (3,237)

Python Spark (2,053)

Java Spark (1,587)

Kafka Zookeeper (1,229)

Apache Spark (1,207)

Spark Hadoop (1,188)

Jupyter Notebook Spark (1,151)

Python Kafka (1,133)

Docker Kafka (1,106)

101-200 of 373 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.