Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Spark Kafka Open Source Projects

Open source projects categorized as Spark Kafka
Data Engineering Zoomcamp ⭐ 19,461
Free Data Engineering course!
most recent commit 3 months ago
Bigdata Notes ⭐ 14,872
大数据入门指南 ⭐
most recent commit 4 months ago
Flink Learning ⭐ 13,801
flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《 Flink 实战与性能优化》
most recent commit 7 months ago
Technology Talk ⭐ 13,579
【大厂面试专栏】一份Java程序员需要的技术指南，这里有面试题、系统架构、职场锦囊、主流中间件等，让
most recent commit 6 months ago
Cookbook ⭐ 12,557
The Data Engineering Cookbook
most recent commit 4 months ago
God Of Bigdata ⭐ 8,483
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive.
most recent commit 9 months ago
Risingwave ⭐ 5,799
The distributed streaming database. Engineered to offer the simplest and most cost-efficient way for stream processing and management.
total releases 14most recent commit 3 months ago
Pipeline ⭐ 4,158
PipelineAI
total releases 85most recent commit a year ago
Bigdataguide ⭐ 2,355
大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
most recent commit 4 months ago
Szt Bigdata ⭐ 2,055
深圳地铁大数据客流分析系统🚇🚄🌟
most recent commit 3 months ago
Jvm Profiler ⭐ 1,717
JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter
most recent commit 9 months ago
Movie_recommend ⭐ 1,441
基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统
most recent commit 5 years ago
Seldon Server ⭐ 1,420
Machine Learning Platform and Recommendation Engine built on Kubernetes
total releases 44most recent commit 4 years ago
Bigdata Interview ⭐ 1,397
🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop
most recent commit 3 years ago
Bigdata Growth ⭐ 1,256
大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。
most recent commit 7 days ago
Killrweather ⭐ 1,174
KillrWeather is a reference application (work in progress) showing how to easily integrate streaming and batch data processing with Apache Spark Streaming, Apache Cassandra, Apache Kafka and Akka for fast, streaming computations on time series data in asynchronous event-driven environments.
most recent commit 7 years ago
Dockerfiles ⭐ 1,171
50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak
most recent commit 5 months ago
Kafka Storm Starter ⭐ 729
[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
most recent commit 2 years ago
Pkpmspark ⭐ 697
awesome 三维数据挖掘 数据分析 & 推荐
most recent commit 4 years ago
Kafka Spark Consumer ⭐ 616
High Performance Kafka Connector for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker.
most recent commit 2 years ago
Freestyle ⭐ 615
A cohesive & pragmatic framework of FP centric Scala libraries
dependent packages 34total releases 19most recent commit 5 years ago
Data Engineering Interview Questions ⭐ 554
More than 2000+ Data engineer interview questions.
most recent commit 7 months ago
Sparta ⭐ 526
Real Time Analytics and Data Pipelines based on Spark Streaming
most recent commit 5 years ago
The 10 Most Depended On Spark Kafka Open Source Projects
Freestyle ⭐ 615
A cohesive & pragmatic framework of FP centric Scala libraries
dependent packages 34total releases 19latest release June 07, 2018most recent commit 5 years ago
Abris ⭐ 215
Avro SerDe for Apache Spark structured APIs.
dependent packages 5total releases 17latest release October 06, 2020most recent commit 8 months ago
Aliyun Emapreduce Datasources ⭐ 157
Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.
dependent packages 3total releases 15latest release February 19, 2019most recent commit 2 years ago
Baryon ⭐ 12
Baryon is a library for building Spark Streaming applications that consume data from Kafka.
dependent packages 2total releases 1latest release July 04, 2016most recent commit 7 years ago
Hyperdrive ⭐ 41
Extensible streaming ingestion pipeline on top of Apache Spark
dependent packages 1total releases 8latest release November 20, 2020most recent commit a year ago
The 10 Latest Releases In Spark Kafka Open Source Projects
Zat ⭐ 414
Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark
dependent packages 1total releases 11latest release January 26, 2023most recent commit 3 months agopypi zat} Downloads
Bullet Kafka ⭐ 5
The Apache Kafka PubSub implementation pluggable into the Bullet Service and Backend
total releases 9latest release January 11, 2022most recent commit 2 years ago
Spark Sql Kafka Offset Committer ⭐ 15
Kafka offset committer for structured streaming query
total releases 6latest release December 15, 2020most recent commit 3 years ago
Hyperdrive ⭐ 41
Extensible streaming ingestion pipeline on top of Apache Spark
dependent packages 1total releases 8latest release November 20, 2020most recent commit a year ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.