Awesome Open Source
Awesome Open Source
Combined Topics
stream-processing
x
Advertising
📦 10
All Projects
Application Programming Interfaces
📦 124
Applications
📦 192
Artificial Intelligence
📦 78
Blockchain
📦 73
Build Tools
📦 113
Cloud Computing
📦 80
Code Quality
📦 28
Collaboration
📦 32
Command Line Interface
📦 49
Community
📦 83
Companies
📦 60
Compilers
📦 63
Computer Science
📦 80
Configuration Management
📦 42
Content Management
📦 175
Control Flow
📦 213
Data Formats
📦 78
Data Processing
📦 276
Data Storage
📦 135
Economics
📦 64
Frameworks
📦 215
Games
📦 129
Graphics
📦 110
Hardware
📦 152
Integrated Development Environments
📦 49
Learning Resources
📦 166
Legal
📦 29
Libraries
📦 129
Lists Of Projects
📦 22
Machine Learning
📦 347
Mapping
📦 64
Marketing
📦 15
Mathematics
📦 55
Media
📦 239
Messaging
📦 98
Networking
📦 315
Operating Systems
📦 89
Operations
📦 121
Package Managers
📦 55
Programming Languages
📦 245
Runtime Environments
📦 100
Science
📦 42
Security
📦 396
Social Media
📦 27
Software Architecture
📦 72
Software Development
📦 72
Software Performance
📦 58
Software Quality
📦 133
Text Editors
📦 49
Text Processing
📦 136
User Interface
📦 330
User Interface Components
📦 514
Version Control
📦 30
Virtualization
📦 71
Web Browsers
📦 42
Web Servers
📦 26
Web User Interface
📦 210
The Top 76 Stream Processing Open Source Projects
Categories
>
Data Processing
>
Stream Processing
Mediapipe
⭐
11,620
Cross-platform, customizable ML solutions for live and streaming media.
Awesome Bigdata
⭐
9,840
A curated list of awesome big data frameworks, ressources and other awesomeness.
Flink Learning
⭐
9,752
flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
Vector
⭐
6,884
A high-performance, high-reliability observability data pipeline.
Faust
⭐
5,467
Python Stream Processing
Ksql
⭐
4,179
The event streaming database purpose-built for stream processing applications
Awesome System Design
⭐
3,890
A curated list of awesome System Design (A.K.A. Distributed Systems) resources.
Benthos
⭐
2,994
Declarative stream processing for mundane tasks and data engineering
Watermill
⭐
2,659
Building event-driven applications the easy way in Go.
Dpark
⭐
2,658
Python clone of Spark, a MapReduce alike framework in Python
Fluent Bit
⭐
2,646
Fast and Lightweight Log processor and forwarder for Linux, BSD and OSX
Pipelinedb
⭐
2,382
High-performance time-series aggregation for PostgreSQL
Gojay
⭐
1,965
fastest JSON encoder/decoder with powerful stream API for Golang
Hudi
⭐
1,860
Upserts, Deletes And Incremental Processing on Big Data.
Awesome Streaming
⭐
1,714
a curated list of awesome streaming frameworks, applications, etc
Riko
⭐
1,563
A Python stream processing engine modeled after Yahoo! Pipes
Wally
⭐
1,439
Distributed Stream Processing
Siddhi
⭐
1,194
Stream Processing and Complex Event Processing Engine
Wormhole
⭐
877
Wormhole is a SPaaS (Stream Processing as a Service) Platform
Hazelcast Jet
⭐
873
Distributed Stream and Batch Processing
Spring Cloud Dataflow
⭐
758
A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
Gearpump
⭐
747
Lightweight real-time big data streaming engine over Akka
Spring Cloud Stream
⭐
671
Framework for building Event-Driven Microservices
Kafka Streams
⭐
623
equivalent to kafka-streams 🐙 for nodejs ✨🐢🚀✨
Go Streams
⭐
623
A lightweight stream processing library for Go
Automi
⭐
621
A stream processing API for Go (alpha)
Examples
⭐
571
Demo applications and code examples for Confluent Platform and Apache Kafka
Streaming Readings
⭐
557
Streaming System 相关的论文读物
Stream Json
⭐
474
The micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API.
Kasper
⭐
412
Kasper is a lightweight library for processing Kafka topics.
Awesome Kafka
⭐
409
A list about Apache Kafka
Json Machine
⭐
393
Efficient, easy-to-use, and fast PHP JSON stream parser
Bistro
⭐
333
A general-purpose data analysis engine radically changing the way batch and stream data is processed
Amadeus
⭐
309
Harmonious distributed data analysis in Rust.
Yomo
⭐
305
🦖 Streaming-Serverless Framework for Low-latency Edge Computing applications, running atop QUIC protocol, engaging 5G technology.
Smooks
⭐
296
Extensible Java framework for building XML and non-XML fragment-based applications
Psi
⭐
272
Platform for Situated Intelligence
Hstream
⭐
259
The database built for IoT streaming data storage and real-time stream processing.
Core
⭐
237
Build platforms that flexibly mix SQL, batch, and stream processing paradigms
Incubator Streampipes
⭐
223
Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.
Flink Sql Cookbook
⭐
205
The Apache Flink SQL Cookbook is a curated collection of examples, patterns, and use cases of Apache Flink SQL. Many of the recipes are completely self-contained and can be run in Ververica Platform as is.
Pluck
⭐
202
Pluck text in a fast and intuitive way 🐓
Media Stream Library Js
⭐
194
JavaScript library to handle media streams on the command line (Node.js) and in the browser.
Logrange
⭐
183
High performance data aggregating storage
Log Anomaly Detector
⭐
179
Log Anomaly Detection - Machine learning to detect abnormal events logs
Kafka Streams In Action
⭐
177
Source code for the Kafka Streams in Action Book
Fero
⭐
175
light, fast, scalable, streaming microservices made easy
Akka Stream Contrib
⭐
173
Add-ons to Akka Stream
Vectorsql
⭐
171
VectorSQL is a free analytics DBMS for IoT & Big Data, compatible with ClickHouse.
Dolphinbeat
⭐
164
A server that pulls and parses MySQL binlog, pushs change data into different sinks like Kafka.
Redisgears
⭐
159
Dynamic execution framework for your Redis data
Bats
⭐
153
面向 OLTP、OLAP、批处理、流处理场景的大一统 SQL 引擎
Kafka Tutorials
⭐
152
Kafka Tutorials microsite
Wayeb
⭐
138
Wayeb is a Complex Event Processing and Forecasting (CEP/F) engine written in Scala.
Real Time Sentiment Tracking On Twitter For Brand Improvement And Trend Recognition
⭐
136
A real-time interactive web app based on data pipelines using streaming Twitter data, automated sentiment analysis, and MySQL&PostgreSQL database (Deployed on Heroku)
Pulsar Flink
⭐
134
Elastic data processing with Apache Pulsar and Apache Flink
Samsara
⭐
132
Samsara is a real-time analytics platform
Neo4j Streams
⭐
126
Neo4j Kafka Integrations, Docs =>
Go Kafka Example
⭐
111
Golang Kafka consumer and producer example
Gsf
⭐
109
Grid Solutions Framework
Logisland
⭐
97
Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.
Hudi Resources
⭐
89
汇总Apache Hudi相关资料
Kspp
⭐
80
A high performance/ real-time C++ Kafka streams framework (C++17)
Machine
⭐
79
Machine is a workflow/pipeline library for processing data
Fs2 Kafka
⭐
75
Kafka client for functional streams for scala (fs2)
Pg2kafka
⭐
62
Ship changes in Postgres 🐘 to Kafka 📖
Pulsar Spark
⭐
56
When Apache Pulsar meets Apache Spark
Stream
⭐
45
Elegant stream processing pipeline written entirely in Golang
Storm Dynamic Spout
⭐
41
A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.
Saber
⭐
35
Window-Based Hybrid CPU/GPU Stream Processing Engine
Ksql Recipes Try It At Home
⭐
33
Files needed to try out KSQL Recipes for yourself
Streamsx.messaging
⭐
31
This toolkit is focused on interacting with popular messaging systems such as Kafka, JMS, XMS, and MQTT. After release v5.4.2 the complete toolkit will be deprecated. See the README.md file for hints to alternative toolkits.
Redis Stream Demo
⭐
24
Demo for Redis Streams
Tutorials
⭐
14
A project for developing tutorials for Streams
Streamsx.inet
⭐
11
This toolkit supports common internet protocols, such as HTTP and WebSockets
Tuna
⭐
11
🐟 A streaming ETL for fish
1-76 of 76 projects
Advertising
📦 10
All Projects
Application Programming Interfaces
📦 124
Applications
📦 192
Artificial Intelligence
📦 78
Blockchain
📦 73
Build Tools
📦 113
Cloud Computing
📦 80
Code Quality
📦 28
Collaboration
📦 32
Command Line Interface
📦 49
Community
📦 83
Companies
📦 60
Compilers
📦 63
Computer Science
📦 80
Configuration Management
📦 42
Content Management
📦 175
Control Flow
📦 213
Data Formats
📦 78
Data Processing
📦 276
Data Storage
📦 135
Economics
📦 64
Frameworks
📦 215
Games
📦 129
Graphics
📦 110
Hardware
📦 152
Integrated Development Environments
📦 49
Learning Resources
📦 166
Legal
📦 29
Libraries
📦 129
Lists Of Projects
📦 22
Machine Learning
📦 347
Mapping
📦 64
Marketing
📦 15
Mathematics
📦 55
Media
📦 239
Messaging
📦 98
Networking
📦 315
Operating Systems
📦 89
Operations
📦 121
Package Managers
📦 55
Programming Languages
📦 245
Runtime Environments
📦 100
Science
📦 42
Security
📦 396
Social Media
📦 27
Software Architecture
📦 72
Software Development
📦 72
Software Performance
📦 58
Software Quality
📦 133
Text Editors
📦 49
Text Processing
📦 136
User Interface
📦 330
User Interface Components
📦 514
Version Control
📦 30
Virtualization
📦 71
Web Browsers
📦 42
Web Servers
📦 26
Web User Interface
📦 210
