Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for kafka s3
kafka
x
s3
x
41 search results found
Javakeeper
⭐
1,766
✍️ Java 工程师必备架构体系知识总结:涵盖分布式、微服务、RPC等互联网公司常用架构,以及数据存储、缓存、搜索
Dataengineeringproject
⭐
644
Example end to end data engineering project.
Kafka Connect Ui
⭐
494
Web tool for Kafka Connect |
Automq For Kafka
⭐
304
A cloud native implementation for Apache Kafka, reducing your cloud infrastructure bill by up to 90%.
Storagetapper
⭐
269
StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service
Kafka Connect Storage Cloud
⭐
256
Kafka Connect suite of connectors for Cloud storage (Amazon S3)
Kafka Connect Fs
⭐
106
Kafka Connect FileSystem Connector
Bifrost
⭐
98
Safely archive data from Apache Kafka to S3 with no Hadoop dependencies :)
Streamx
⭐
95
kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)
Automq For Rocketmq
⭐
86
A cloud native implementation for Apache RocketMQ 5.0
S3 Connector For Apache Kafka
⭐
53
Aiven's S3 Sink Connector for Apache Kafka®
Backbeat
⭐
51
Zenko Backbeat is the core engine for asynchronous replication, optimized for queuing metadata updates and dispatching work to long-running tasks in the background.
Kafka Connect S3
⭐
50
A Kafka-Connect Sink for S3 with no Hadoop dependencies.
Kafka Large Message Serde
⭐
49
A Kafka Serde that reads and writes records from and to Blob storage (S3, Azure, Google) transparently.
Tamer
⭐
41
Standalone alternatives to Kafka Connect Connectors
Etl Light
⭐
38
A light Kafka to HDFS/S3 ETL library based on Apache Spark
Spark_app_twitter
⭐
36
A data engineering project (Twitter monitor app)
Collecting Data
⭐
31
This is a HOWTO for collecting data in Ruby and Python applications and sending it to S3 via Kafka.
Stream Loader
⭐
30
Components for building stream loaders from Kafka to arbitrary storages
Gosquito
⭐
27
gosquito ("go" + "mosquito") is a pluggable tool for data gathering, data processing and data transmitting to various destinations.
Data Digger
⭐
24
Dig through structured messages in Kafka, S3, or local files
Nodestream
⭐
23
A Fast, Declarative, and Extensible ETL Framework for Graph Databases.
Inferencedb
⭐
20
🚀 Stream inferences of real-time ML models in production to any data lake
Edge Fuse
⭐
18
Edge-X S3 API, mount S3 bucket for full POSIX read/write access
Lambda_s3_kafka
⭐
15
AWS Lambda function to get events in Kafka topic when files are uploaded to S3
Lambda Architecture Demo
⭐
14
Developing a Lambda Architecture pipeline using Apache Kafka, Spark Structured Streaming, Redshift, S3, Python
Kafka S3 Consumer
⭐
13
Consumes Kafka topics specified in the config, and outputs them in chunks as desired in an S3 Bucket. Keeps track of offsets via S3.
Heapland
⭐
13
Universal interface for data services
Ifood Data Engineering Test
⭐
13
Nyc_taxi_pipeline
⭐
12
Design/Implement stream/batch architecture on NYC taxi data | #DE
Kafka Connect Field And Time Partitioner
⭐
11
Kafka Connect Store Partitioner by custom fields and time
Gucef
⭐
9
This is the main mono-repo for the Galaxy Unlimited Platform. The software contained herein is a collection of libraries and derived services and applications all fairly generic in nature. The overarching purpose of these libraries is to act as a C++ tool chest with which to rapidly develop applications and services
Temporal Land
⭐
9
Integrations as a Service on Temporal for building reliable, durable, and scalable Cloud Platforms and Data Planes.
Apache Kafka Backup And Restore
⭐
9
It will take backup of given topic and store that into either local filesystem or S3.It will also restore from given local filesystem or S3.
Kafka Connect Transform Archive
⭐
8
Kafka Connect transform to assist with archiving to S3.
Kafka Archiver
⭐
8
Kafka log persistence service
Stackbox
⭐
7
Create app stacks loaded with all your favourite clients, services and infra along with code boilerplates in under 5 mins.
Kafka Archiver
⭐
7
Archive kafka topics into SequenceFiles containing chunks of the message stream stored in S3.
Easy S3 Kafka Consumer
⭐
6
Pain-free system for sinking data from kafka to s3. Written in Go for high concurrency. Great for streaming large amounts of web data into s3 for use with mapreduce.
Kafka Connect S3
⭐
5
Kafka Replicator
⭐
5
Kafka replicator is a tool used to mirror and backup Kafka topics across regions
Related Searches
Java Kafka (3,367)
Amazon Web Services S3 (2,551)
Javascript S3 (1,284)
Kafka Zookeeper (1,240)
Docker Kafka (1,191)
Python S3 (1,171)
Amazon S3 (1,059)
Python Kafka (1,053)
Spark Kafka (979)
Scala Kafka (969)
1-41 of 41 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.