Awesome Open Source

Programming Languages

Search results for kafka spark streaming

spark-streaming x

76 search results found

Risingwave ⭐ 5,799

The distributed streaming database. Engineered to offer the simplest and most cost-efficient way for stream processing and management.

Movie_recommend ⭐ 1,441

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

Real Time Analytics and Data Pipelines based on Spark Streaming

Data Accelerator ⭐ 293

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Azure Event Hubs Spark ⭐ 225

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Sparkstreaming ⭐ 183

💥 🚀 封装sparkstreaming动态调节batch time(有数据就执行计算)；🚀 支持运行过程中增删topic；🚀 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

Spark Streaming With Kafka ⭐ 161

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

Streamline ⭐ 156

StreamLine - Streaming Analytics

Bigdata Playground ⭐ 154

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Spark Kafka ⭐ 136

Low level integration of Spark and Kafka

Example Spark Kafka ⭐ 118

Apache Spark and Apache Kafka integration example

Kafka Spark Streaming Example ⭐ 96

Simple examle for Spark Streaming over Kafka topic

Spark_als ⭐ 68

基于spark-ml,spark-mllib,spark-streaming的推荐算法实现

Recommendmoteur ⭐ 57

电影推荐系统、电影推荐引擎、使用Spark完成的电影推荐引擎

Movie Recommender Demo ⭐ 50

This project walks through how you can create recommendations using Apache Spark machine learning. There are a number of jupyter notebooks that you can run on IBM Data Science Experience, and there a live demo of a movie recommendation web application you can interact with. The demo also uses IBM Message Hub (kafka) to push application events to topic where they are consumed by a spark streaming job running on IBM BigInsights (hadoop).

Fdp Modelserver ⭐ 45

An umbrella project for multiple implementations of model serving

Xxhadoop ⭐ 37

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

Real Time Stream Processing Engine ⭐ 37

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

Spark_app_twitter ⭐ 36

A data engineering project (Twitter monitor app)

Litemall Dw ⭐ 36

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；

Spark Utils ⭐ 35

💥 👽 ♨️🚀Encapsulated spark 与其他组件的结合api，方便使用，例如 es，hbase，kudu，kafka，mq等

Spark Streaming Direct Kafka ⭐ 30

High Performance Spark Streaming with Direct Kafka in Java

Interview Refresher Java Bigdata ⭐ 26

a one-stop repo to lookup for code snippets of core java concepts, sql, data structures as well as big data. It also consists of interview questions asked in real-life.

Cassandra.realtime ⭐ 25

Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Open Stream Processing Benchmark ⭐ 24

This repository contains the code base for the Open Stream Processing Benchmark.

Recommendsystem ⭐ 24

电影推荐系统

Ozymandias ⭐ 23

Real-time image processing at scale using Kafka and Spark Streaming

Kafka Connect Msk Demo ⭐ 21

For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR

Spark And Kafka_iot Data Processing And Analytics ⭐ 21

Final Project for IoT: Big Data Processing and Analytics class in UCSC Extension. Analyzing U.S nationwide temperature from IoT sensors in real-time

Bandar Log ⭐ 20

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

A general purpose metrics monitor implemented with Apache Spark. Kafka source, Elastic sink, aggregate metrics, different analysis, notifications, actions, live configuration update, missing metrics, ...

Realtime Dashboard Example ⭐ 18

This is a real-time dashboard example using Spark Streaming and Node.js

Sparknlp Elasticsearch ⭐ 17

Twitter sentiment analysis using Spark and Stanford CoreNLP and visualization using elasticsearch and kibana

Qs Hadoop ⭐ 17

大数据生态圈学习

Java_spark ⭐ 16

Spark 2.x 案例操作：Scala版本与 Java1.8lambda版代码示例。涵盖Spark核心技术操作SparkCore、SparkSql

Kafka Twitter Spark Streaming ⭐ 16

Counting Tweets Per User in Real-Time

Real Time Stock Analyzer ⭐ 16

Bigdata Pipeline

Real Time Twitter Sentiment Analysis Product

Tweet Analysis With Kafka And Spark ⭐ 15

A real time analytics dashboard to analyze the trending hashtags and @ mentions at any location using kafka and spark streaming.

Twitterstreamingsparkkafkademo ⭐ 15

a demo project to Analyze most popular twitter hashtags using Java 8 Spring-Boot Spark Streaming Kafka & Docker Demo.

Lambda Architecture Demo ⭐ 14

Developing a Lambda Architecture pipeline using Apache Kafka, Spark Structured Streaming, Redshift, S3, Python

Live_twitter_sentiment_analysis ⭐ 14

Live Twitter sentiment analysis using Python, Apache Spark Streaming, Kafka, NLTK, SocketIO

A Big Data Platform Prototype Project

Spark Streaming External Projects ⭐ 13

Nyc_taxi_pipeline ⭐ 12

Design/Implement stream/batch architecture on NYC taxi data | #DE

Kafka Spark Streaming ⭐ 12

An example project for Kafka and Spark Streaming integration

Spark Example ⭐ 11

Spark1.6和spark2.2的示例，包含kafka,flume,structuredstrea

Registry ⭐ 11

Schema Registry

Big Data ⭐ 10

Big data technologies

Confluent Platform Spark Streaming ⭐ 10

Working example of consuming Avro data from Kafka with Spark Streaming

Bigdatademo ⭐ 9

The demo of using Kafka, Spark, Hive, Cassandra, etc by using Docker. It produces the production ready environment for any kinds of big data project relates to Hadoop ecosystem

Realtime parking spot recommender system and crowd-sourcing platform

Freeza Offset ⭐ 9

Spark stream consumption commit in kafka consumer group

Big Data Framework Demos ⭐ 9

It is a assemble to include all Practice Projects about Big Data Topic, includes Hadoop, Spark, Spark Streaming and Kafka

Kafka Scala Api ⭐ 8

Samples for using Kafka within Spark Streaming and Akka Actors, Akka Streams

Realtime Recommender ⭐ 8

Spark-Kafka Realtime recommender Engine.

Kafka Spark Product Recommendation ⭐ 8

Python script demonstrating spark streaming and Kafka implementation using an e-commerce website like product recommendation engine based on item-based collaborative filtering. 🐍. 💥

Kafka Spark Streaming Weblog ⭐ 8

Sparking Stream from Kafka to calculate real time top url click stream from an http weblog using Spark Streaming, Kafka, SpringBoot, Spring Integration tech stack

Firehose - Spark streaming 2.2 + Kafka 0.8_2

Kse Sample ⭐ 7

This project is used for getting data from Kafka and dealing through Spark streaming and finally storing into Elasticsearch.

Openmrs Etl ⭐ 7

openmrs - mysql - debezium - kafka - spark - scala

Kafkaoffshore ⭐ 7

Kafka, CouchDB and Friends: A test flight beyond the shore

Big Data Laboratory ⭐ 7

Big Data laboratory, using Kibana, Elasticsearch, Logstash, Kafka and Spark to process messages coming from OPC UA and DB

Spark Streaming Kafka Cassandra Starter ⭐ 7

Application built on Spark Streaming, Kafka and Cassandra.

Data_visualization_and_analysis_tool_for_telemetry_data ⭐ 6

An naive anomaly detection and data visualization tool for F1 on board telemetry data.

Spark Road Traffic Graph ⭐ 6

divide city to some points as nodes of a traffic graph and process location data from kafka and calculate edge between nodes with spark aggregations

Bigdata Ecosystem Architecture ⭐ 6

Life-cycle: Internal working of HDFS, SQOOP, HIVE, SPARK, HBASE, KAFKA with code.

Spark Kafka Connector ⭐ 6

Reliable spark-kafka connector. Kafka 1.x and Spark 2.x

Twittersentimentanalysis ⭐ 6

A Spark Streaming implementation for Online Twitter Sentiment Analysis.

Insightde ⭐ 6

TrafficAdvisor: a Real-Time Traffic Monitoring System

Spatialsparkapp ⭐ 5

Spark Streaming ⭐ 5

Twitter Spark Streaming using PySpark

Spotless_recommender ⭐ 5

Las Vpe Platform ⭐ 5

ISEE Video Parsing and Evaluation (VPE) Platform.

Spark Streaming Kafka Demo ⭐ 5

spark streaming从kafka读取消息，offset写入Redis，spark计算单词出现频率，最后

Spark Streaming Kafka 0 10 Connector ⭐ 5

A Kafka 0.10 connector for Spark 1.x Streaming

Spark Kafka Example ⭐ 5

Example for Data Reading from and Writing to from Kafka Topic using Apache Spark DataFrame and DataSet

Related Searches

Kafka Java (3,244)

Kafka Zookeeper (1,229)

Docker Kafka (1,106)

Python Kafka (1,066)

Spark Kafka (979)

Scala Kafka (969)

Golang Kafka (919)

Apache Kafka (836)

Stream Kafka (790)

Streaming Kafka (627)

1-76 of 76 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.