Awesome Open Source
Awesome Open Source
Combined Topics
stream-processing
x
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
The Top 503 Stream Processing Open Source Projects on Github
Categories
>
Data Processing
>
Stream Processing
Mediapipe
⭐
15,836
Cross-platform, customizable ML solutions for live and streaming media.
Flink Learning
⭐
11,466
flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
Awesome Bigdata
⭐
10,551
A curated list of awesome big data frameworks, ressources and other awesomeness.
Vector
⭐
8,969
A high-performance observability data pipeline.
Faust
⭐
5,955
Python Stream Processing
Signoz
⭐
5,650
SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc. 🔥 🖥. 👉 Open source Application Performance Monitoring (APM) & Observability tool
Ksql
⭐
4,718
The database purpose-built for stream processing applications.
Hazelcast
⭐
4,704
Open-source distributed computation and storage platform
Awesome System Design
⭐
4,603
A curated list of awesome System Design (A.K.A. Distributed Systems) resources.
Benthos
⭐
3,917
Fancy stream processing made operationally mundane
Watermill
⭐
3,651
Building event-driven applications the easy way in Go.
Fluent Bit
⭐
3,286
Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
Hudi
⭐
2,711
Upserts, Deletes And Incremental Processing on Big Data.
Dpark
⭐
2,637
Python clone of Spark, a MapReduce alike framework in Python
Pipelinedb
⭐
2,303
High-performance time-series aggregation for PostgreSQL
Danfojs
⭐
2,090
Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.
Gojay
⭐
1,991
fastest JSON encoder/decoder with powerful stream API for Golang
Awesome Streaming
⭐
1,896
a curated list of awesome streaming frameworks, applications, etc
Riko
⭐
1,573
A Python stream processing engine modeled after Yahoo! Pipes
Wally
⭐
1,453
Distributed Stream Processing
Siddhi
⭐
1,300
Stream Processing and Complex Event Processing Engine
Hazelcast Jet
⭐
956
Distributed Stream and Batch Processing
Manifold
⭐
886
A compatibility layer for event-driven abstractions
Wormhole
⭐
863
Wormhole is a SPaaS (Stream Processing as a Service) Platform
Spring Cloud Dataflow
⭐
840
A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
Go Streams
⭐
813
A lightweight stream processing library for Go
Spring Cloud Stream
⭐
750
Framework for building Event-Driven Microservices
Gearpump
⭐
747
Lightweight real-time big data streaming engine over Akka
Yomo
⭐
731
🦖 Serverless Streaming Framework for Low-latency Edge Computing applications, running atop QUIC protocol, as Metaverse infrastructure, engaging 5G technology.
Kafka Streams
⭐
713
equivalent to kafka-streams 🐙 for nodejs ✨🐢🚀✨
Automi
⭐
655
A stream processing API for Go (alpha)
Json Machine
⭐
629
Efficient, easy-to-use, and fast PHP JSON stream parser
Stream Json
⭐
541
The micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API.
Streaming Readings
⭐
489
Streaming System 相关的论文读物
Awesome Kafka
⭐
462
A list about Apache Kafka
Hstream
⭐
448
The database built for IoT streaming data storage and real-time stream processing.
Kasper
⭐
413
Kasper is a lightweight library for processing Kafka topics.
Flink Sql Cookbook
⭐
396
The Apache Flink SQL Cookbook is a curated collection of examples, patterns, and use cases of Apache Flink SQL. Many of the recipes are completely self-contained and can be run in Ververica Platform as is.
Psi
⭐
361
Platform for Situated Intelligence
Amadeus
⭐
334
Harmonious distributed data analysis in Rust.
Smooks
⭐
331
Extensible data integration Java framework for building XML and non-XML fragment-based applications
Core
⭐
305
Build platforms that flexibly mix SQL, batch, and stream processing paradigms
Incubator Streampipes
⭐
282
Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.
Hudi Resources
⭐
243
汇总Apache Hudi相关资料
Pulsar Flink
⭐
241
Elastic data processing with Apache Pulsar and Apache Flink
Media Stream Library Js
⭐
225
JavaScript library to handle media streams on the command line (Node.js) and in the browser.
Redisgears
⭐
221
Dynamic execution framework for your Redis data
Kafka Tutorials
⭐
201
Kafka Tutorials microsite
Pluck
⭐
194
Pluck text in a fast and intuitive way 🐓
Vectorsql
⭐
191
VectorSQL is a free analytics DBMS for IoT & Big Data, compatible with ClickHouse.
Logrange
⭐
183
High performance data aggregating storage
Kafka Streams In Action
⭐
183
Source code for the Kafka Streams in Action Book
Fero
⭐
173
light, fast, scalable, streaming microservices made easy
Akka Stream Contrib
⭐
173
Add-ons to Akka Stream
Dagger
⭐
159
Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data.
Dolphinbeat
⭐
153
A server that pulls and parses MySQL binlog, pushs change data into different sinks like Kafka.
Memgraph
⭐
153
Build modern, graph-based applications on top of your streaming data in minutes
Neo4j Streams
⭐
145
Neo4j Kafka Integrations, Docs =>
Wayeb
⭐
139
Wayeb is a Complex Event Processing and Forecasting (CEP/F) engine written in Scala.
Xlstream
⭐
136
Turns XLSX into a readable stream.
Log Anomaly Detector
⭐
134
Log Anomaly Detection - Machine learning to detect abnormal events logs
Daggy
⭐
123
Daggy - Data Aggregation Utility and C/C++ developer library for data streams catching
Samsara
⭐
120
Samsara is a real-time analytics platform
Gsf
⭐
117
Grid Solutions Framework
Dspatch
⭐
113
The Refreshingly Simple Cross-Platform C++ Dataflow / Pipelining / Stream Processing / Reactive Programming Framework
Radio Active
⭐
112
Play any radios around the globe right from the terminal ⚡️
Talaria
⭐
109
TalariaDB is a distributed, highly available, and low latency time-series database for Presto
Go Kafka Example
⭐
108
Golang Kafka consumer and producer example
Logisland
⭐
105
Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.
Stream Registry
⭐
103
Stream Discovery and Stream Orchestration
Machine
⭐
101
Machine is a workflow/pipeline library for processing data
Openpdc
⭐
97
Open Source Phasor Data Concentrator
Kafka Spark Streaming Example
⭐
96
Simple examle for Spark Streaming over Kafka topic
Libretto
⭐
91
Declarative concurrency and stream processing library for Scala
Kspp
⭐
87
A high performance/ real-time C++ Kafka streams framework (C++17)
React Player
⭐
83
One audio player built with ReactJS, Web Audio API and Web Worker API
Flinkexperiments
⭐
78
Experiments with Apache Flink.
Kafka Streams Real Time Stream Processing
⭐
76
This is the central repository for all materials related to Kafka Streams : Real-time Stream Processing! Book by Prashant Pandey.
Pulsar Spark
⭐
76
When Apache Pulsar meets Apache Spark
Flink Connectors
⭐
76
Apache Flink connectors for Pravega.
Fs2 Kafka
⭐
75
Kafka client for functional streams for scala (fs2)
Product Sp
⭐
74
An open source, cloud-native streaming data integration and analytics product optimized for agile digital businesses
Function Mesh
⭐
72
The serverless framework purpose-built for event streaming applications.
Real Time Sentiment Tracking On Twitter For Brand Improvement And Trend Recognition
⭐
69
A real-time interactive web app based on data pipelines using streaming Twitter data, automated sentiment analysis, and MySQL&PostgreSQL database (Deployed on Heroku)
Csvplus
⭐
67
csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.
Kafka4s
⭐
66
Functional programming with Kafka and Scala
Kafka Shell
⭐
65
⚡A supercharged, interactive Kafka shell built on top of the existing Kafka CLI tools.
Pg2kafka
⭐
63
Ship changes in Postgres 🐘 to Kafka 📖
Fluxtion
⭐
62
core fluxtion static event processor generator tool and runtime components
Grppi
⭐
61
Generic Reusable Parallel Patterns Interface
Streaming Integrator
⭐
61
A stream processing runtime that allows connecting any streaming data source to any destination and act on it
Flink Commodity Recommendation System
⭐
60
🐳基于 Flink 的商品实时推荐系统。当用户产生评分行为时，数据由 kafka 发送到 flink，根据用户历史评分行为进行实时和离线推荐。实时推荐包括：基于行为和实时热门，离线推荐包括：历史热门、历史优质商品和 itemcf 。
Ksqldb Go
⭐
59
A Golang client for ksqlDB
Rtec
⭐
53
RTEC is an Event Calculus implementation optimised for stream reasoning
Mage
⭐
51
MAGE - Memgraph Advanced Graph Extensions 🔮
Streamdb Readings
⭐
48
Readings in Stream Processing
Data_processing_course
⭐
48
Some class materials for a data processing course using PySpark
Kafka Fraud Detector
⭐
47
🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python
Swirl
⭐
45
High-Performance Erlang Stream Processor
Gostream
⭐
45
Stream Processing Library for Go
1-100 of 503 projects
Next >
Related Projects
Java Stream Processing Projects (130)
Kafka Stream Processing Projects (101)
Python Stream Processing Projects (75)
Go Stream Processing Projects (60)
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
Privacy policy
"GitHub" is a registered trademark of GitHub, Inc. Awesome Open Source is not affiliated with GitHub.
