Awesome Open Source
Awesome Open Source
Combined Topics
stream-processing
x
Advertising
📦 10
All Projects
Application Programming Interfaces
📦 124
Applications
📦 192
Artificial Intelligence
📦 78
Blockchain
📦 73
Build Tools
📦 113
Cloud Computing
📦 80
Code Quality
📦 28
Collaboration
📦 32
Command Line Interface
📦 49
Community
📦 83
Companies
📦 60
Compilers
📦 63
Computer Science
📦 80
Configuration Management
📦 42
Content Management
📦 175
Control Flow
📦 213
Data Formats
📦 78
Data Processing
📦 276
Data Storage
📦 135
Economics
📦 64
Frameworks
📦 215
Games
📦 129
Graphics
📦 110
Hardware
📦 152
Integrated Development Environments
📦 49
Learning Resources
📦 166
Legal
📦 29
Libraries
📦 129
Lists Of Projects
📦 22
Machine Learning
📦 347
Mapping
📦 64
Marketing
📦 15
Mathematics
📦 55
Media
📦 239
Messaging
📦 98
Networking
📦 315
Operating Systems
📦 89
Operations
📦 121
Package Managers
📦 55
Programming Languages
📦 245
Runtime Environments
📦 100
Science
📦 42
Security
📦 396
Social Media
📦 27
Software Architecture
📦 72
Software Development
📦 72
Software Performance
📦 58
Software Quality
📦 133
Text Editors
📦 49
Text Processing
📦 136
User Interface
📦 330
User Interface Components
📦 514
Version Control
📦 30
Virtualization
📦 71
Web Browsers
📦 42
Web Servers
📦 26
Web User Interface
📦 210
The Top 75 Stream Processing Open Source Projects
Categories
>
Data Processing
>
Stream Processing
Mediapipe
⭐
10,648
Cross-platform, customizable ML solutions for live and streaming media.
Awesome Bigdata
⭐
9,626
A curated list of awesome big data frameworks, ressources and other awesomeness.
Flink Learning
⭐
9,226
flink learning blog. http://www.flink-learning.com 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
Vector
⭐
6,133
A high-performance observability data pipeline.
Faust
⭐
5,221
Python Stream Processing
Ksql
⭐
3,993
The event streaming database purpose-built for stream processing applications
Awesome System Design
⭐
3,657
A curated list of awesome System Design (A.K.A. Distributed Systems) resources.
Benthos
⭐
2,796
Declarative streaming ETL for mundane tasks, written in Go
Dpark
⭐
2,642
Python clone of Spark, a MapReduce alike framework in Python
Fluent Bit
⭐
2,464
Fast and Lightweight Log processor and forwarder for Linux, BSD and OSX
Watermill
⭐
2,456
Building event-driven applications the easy way in Go.
Pipelinedb
⭐
2,356
High-performance time-series aggregation for PostgreSQL
Gojay
⭐
1,943
fastest JSON encoder/decoder with powerful stream API for Golang
Hudi
⭐
1,664
Upserts, Deletes And Incremental Processing on Big Data.
Awesome Streaming
⭐
1,634
a curated list of awesome streaming frameworks, applications, etc
Riko
⭐
1,550
A Python stream processing engine modeled after Yahoo! Pipes
Wally
⭐
1,431
Distributed Stream Processing
Siddhi
⭐
1,165
Stream Processing and Complex Event Processing Engine
Wormhole
⭐
842
Wormhole is a SPaaS (Stream Processing as a Service) Platform
Hazelcast Jet
⭐
834
Distributed Stream and Batch Processing
Gearpump
⭐
746
Lightweight real-time big data streaming engine over Akka
Spring Cloud Dataflow
⭐
725
A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
Spring Cloud Stream
⭐
641
Framework for building Event-Driven Microservices
Kafka Streams
⭐
596
equivalent to kafka-streams 🐙 for nodejs ✨🐢🚀✨
Automi
⭐
595
A stream processing API for Go (alpha)
Go Streams
⭐
580
A lightweight stream processing library for Go
Examples
⭐
571
Demo applications and code examples for Confluent Platform and Apache Kafka
Streaming Readings
⭐
546
Streaming System 相关的论文读物
Stream Json
⭐
443
The micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API.
Kasper
⭐
410
Kasper is a lightweight library for processing Kafka topics.
Awesome Kafka
⭐
383
A list about Apache Kafka
Json Machine
⭐
343
Efficient, easy-to-use, and fast PHP JSON stream parser
Bistro
⭐
333
A general-purpose data analysis engine radically changing the way batch and stream data is processed
Smooks
⭐
286
An extensible Java framework for building XML and non-XML (CSV, EDI, Java, etc...) streaming applications
Psi
⭐
229
Platform for Situated Intelligence
Core
⭐
224
Build platforms that flexibly mix SQL, batch, and stream processing paradigms
Amadeus
⭐
216
Harmonious distributed data analysis in Rust.
Pluck
⭐
201
Pluck text in a fast and intuitive way 🐓
Yomo
⭐
198
🦖 Streaming-Serverless Framework for Low-latency Edge Computing applications, running atop QUIC protocol, engaging 5G technology.
Incubator Streampipes
⭐
192
Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.
Logrange
⭐
183
High performance data aggregating storage
Media Stream Library Js
⭐
182
JavaScript library to handle media streams on the command line (Node.js) and in the browser.
Fero
⭐
175
light, fast, scalable, streaming microservices made easy
Akka Stream Contrib
⭐
171
Add-ons to Akka Stream
Vectorsql
⭐
170
VectorSQL is a free analytics DBMS for IoT & Big Data, compatible with ClickHouse.
Dolphinbeat
⭐
159
A server that pulls and parses MySQL binlog, pushs change data into different sinks like Kafka.
Kafka Streams In Action
⭐
158
Source code for the Kafka Streams in Action Book
Log Anomaly Detector
⭐
158
Log Anomaly Detection - Machine learning to detect abnormal events logs
Bats
⭐
150
面向 OLTP、OLAP、批处理、流处理场景的大一统 SQL 引擎
Redisgears
⭐
145
Dynamic execution framework for your Redis data
Wayeb
⭐
136
Wayeb is a Complex Event Processing and Forecasting (CEP/F) engine written in Scala.
Samsara
⭐
130
Samsara is a real-time analytics platform
Kafka Tutorials
⭐
128
Kafka Tutorials microsite
Neo4j Streams
⭐
124
Neo4j Kafka Integrations, Docs =>
Real Time Sentiment Tracking On Twitter For Brand Improvement And Trend Recognition
⭐
118
A real-time interactive web app based on data pipelines using streaming Twitter data, automated sentiment analysis, and MySQL&PostgreSQL database (Deployed on Heroku)
Pulsar Flink
⭐
110
Elastic data processing with Apache Pulsar and Apache Flink
Go Kafka Example
⭐
106
Golang Kafka consumer and producer example
Gsf
⭐
103
Grid Solutions Framework
Logisland
⭐
96
Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.
Flink Sql Cookbook
⭐
91
The Apache Flink SQL Cookbook is a curated collection of examples, patterns, and use cases of Apache Flink SQL. Many of the recipes are completely self-contained and can be run in Ververica Platform as is.
Fs2 Kafka
⭐
75
Kafka client for functional streams for scala (fs2)
Kspp
⭐
75
A high performance/ real-time C++ Kafka streams framework (C++17)
Machine
⭐
61
Machine is a workflow/pipeline library for processing data
Pg2kafka
⭐
60
Ship changes in Postgres 🐘 to Kafka 📖
Hudi Resources
⭐
58
汇总Apache Hudi相关资料
Pulsar Spark
⭐
51
When Apache Pulsar meets Apache Spark
Stream
⭐
45
Elegant stream processing pipeline written entirely in Golang
Storm Dynamic Spout
⭐
40
A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.
Saber
⭐
34
Window-Based Hybrid CPU/GPU Stream Processing Engine
Ksql Recipes Try It At Home
⭐
33
Files needed to try out KSQL Recipes for yourself
Streamsx.messaging
⭐
31
This toolkit is focused on interacting with popular messaging systems such as Kafka, JMS, XMS, and MQTT. After release v5.4.2 the complete toolkit will be deprecated. See the README.md file for hints to alternative toolkits.
Redis Stream Demo
⭐
24
Demo for Redis Streams
Tutorials
⭐
14
A project for developing tutorials for Streams
Streamsx.inet
⭐
11
This toolkit supports common internet protocols, such as HTTP and WebSockets
Tuna
⭐
11
🐟 A streaming ETL for fish
1-75 of 75 projects
Advertising
📦 10
All Projects
Application Programming Interfaces
📦 124
Applications
📦 192
Artificial Intelligence
📦 78
Blockchain
📦 73
Build Tools
📦 113
Cloud Computing
📦 80
Code Quality
📦 28
Collaboration
📦 32
Command Line Interface
📦 49
Community
📦 83
Companies
📦 60
Compilers
📦 63
Computer Science
📦 80
Configuration Management
📦 42
Content Management
📦 175
Control Flow
📦 213
Data Formats
📦 78
Data Processing
📦 276
Data Storage
📦 135
Economics
📦 64
Frameworks
📦 215
Games
📦 129
Graphics
📦 110
Hardware
📦 152
Integrated Development Environments
📦 49
Learning Resources
📦 166
Legal
📦 29
Libraries
📦 129
Lists Of Projects
📦 22
Machine Learning
📦 347
Mapping
📦 64
Marketing
📦 15
Mathematics
📦 55
Media
📦 239
Messaging
📦 98
Networking
📦 315
Operating Systems
📦 89
Operations
📦 121
Package Managers
📦 55
Programming Languages
📦 245
Runtime Environments
📦 100
Science
📦 42
Security
📦 396
Social Media
📦 27
Software Architecture
📦 72
Software Development
📦 72
Software Performance
📦 58
Software Quality
📦 133
Text Editors
📦 49
Text Processing
📦 136
User Interface
📦 330
User Interface Components
📦 514
Version Control
📦 30
Virtualization
📦 71
Web Browsers
📦 42
Web Servers
📦 26
Web User Interface
📦 210
