Awesome Open Source

Programming Languages

Search results for kafka avro

433 search results found

Confluent Kafka Dotnet ⭐ 2,542

Confluent's Apache Kafka .NET client

Schema Registry ⭐ 1,964

Confluent Schema Registry for Kafka

Examples ⭐ 1,734

Apache Kafka and Confluent Platform examples and demos

Bottledwater Pg ⭐ 1,510

Change data capture from PostgreSQL into Kafka

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

Kafka Storm Starter ⭐ 729

[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Kafkactl ⭐ 561

Command Line Tool for managing Apache Kafka

Data Engineering Interview Questions ⭐ 449

More than 2000+ Data engineer interview questions.

Schema Registry Ui ⭐ 400

Web tool for Avro Schema Registry |

MySQL binary log consumer with the ability to act on changed rows and publish changes to different systems with emphasis on Apache Kafka.

Slimmessagebus ⭐ 362

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

Mongo Kafka ⭐ 294

MongoDB Kafka Connector

Prometheus Kafka Adapter ⭐ 282

Use Kafka as a remote storage database for Prometheus (remote write only)

Divolte Collector ⭐ 275

Divolte Collector

Storagetapper ⭐ 269

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Kafka Connect File Pulse ⭐ 262

🔗 A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka

Avro SerDe for Apache Spark structured APIs.

Trifecta ⭐ 211

Trifecta is a web-based and CLI tool that simplifies inspecting Kafka messages and Zookeeper data. Additionally, the CLI tool provides the capability to import/export data to/from ElasticSearch and MongoDB.

Srclient ⭐ 191

Golang Client for Schema Registry

Dataclasses Avroschema ⭐ 168

Generate Avro Schemas from Python classes. Serialize/Deserialize python instances with avro schemas

Kafka Avro Course ⭐ 156

Learn the Confluent Schema Registry & REST Proxy

Bigdata Playground ⭐ 154

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Kafka Connect Bigquery ⭐ 151

DEPRECATED. PLEASE USE https://github.com/confluentinc/kafka-connect-bigq A Kafka Connect BigQuery sink connector

Kafka Connect Mongodb ⭐ 148

**Unofficial / Community** Kafka Connect MongoDB Sink Connector -> integrated 2019 into the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

Kafka Rest Node ⭐ 144

Node.js client for the Kafka REST proxy

Eel Sdk ⭐ 140

Big Data Toolkit for the JVM

Kafka Serialization ⭐ 124

Lego bricks to build Apache Kafka serializers and deserializers

Confluent Schema Registry ⭐ 122

is a library that makes it easier to interact with the Confluent schema registry

Kafka Connect Datagen ⭐ 121

Connector that generates data for demos

Schema Registry ⭐ 112

A CLI and Go client for Kafka Schema Registry. This repository is deprecated and no longer maintained.

Azure Functions Kafka Extension ⭐ 103

Kafka extension for Azure Functions

Dotnet Avro ⭐ 98

An Avro implementation for .NET

A high performance/ real-time C++ Kafka streams framework (C++17)

Confluent Kafka Certification ⭐ 92

This repos is to keep all the relevant informations for Confluent Certified Developer for Apache Kafka (CCDAK)

Schema Registry Plugin ⭐ 87

Gradle plugin to interact with Confluent Schema-Registry.

Kaufmann_ex ⭐ 86

Kafka backed service library.

Xk6 Kafka ⭐ 85

k6 extension to load test Apache Kafka with support for various serialization formats, SASL, TLS, compression, Schema Registry client and beyond

Generate authentic looking mock data based on a SQL, JSON or Avro schema and produce to Kafka in JSON or Avro format.

Fluent Kafka Streams Tests ⭐ 83

Fluent Kafka Streams Test with Java

Mirror of Linkedin's Camus

Kafka Avro ⭐ 79

Node.js bindings for librdkafka with Avro schema serialization.

Springboot Kafka Connect Jdbc Streams ⭐ 78

The main goal is to play with Kafka Connect and Streams. We have store-api that inserts/updates records in MySQL; Source connectors that monitor inserted/updated records in MySQL and push messages related to those changes to Kafka; Sink connectors that read messages from Kafka and insert documents in ES; Store-streams that listens for messages in Kafka, treats them using Kafka Streams and push new messages back to Kafka.

A convenient Elixir library to work with Avro schemas and Confluent® Schema Registry

Avroconvert ⭐ 76

Rapid Avro serializer for C# .NET

Kafka Connect Twitter ⭐ 75

Kafka Connect Sink/Source for Twitter

Open Bank Mark ⭐ 72

A bank simulation application using mainly Clojure, which can be used to end-to-end test and show some graphs.

Clickhouse Sink Connector ⭐ 72

Replicate data from MySQL, Postgres and MongoDB to ClickHouse

Spark Structured Streaming ⭐ 66

Spark structured streaming with Kafka data source and writing to Cassandra

Orleans.streams.kafka ⭐ 64

An implementation of a PersistentStreamProvider for Microsoft Orleans and Kafka using the Confluent API

Framework for building Kafka and avro based apps in typescript

Faust Docker Compose Example ⭐ 62

Faust dockerized application

Python Kafka Avro ⭐ 62

A simple examle for Python Kafka Avro

Springboot Kafka Connect Debezium Ksqldb ⭐ 61

The goal of this project is to play with Kafka, Debezium and ksqlDB. For this, we have: research-service that inserts/updates/deletes records in MySQL; Source Connectors that monitor change of records in MySQL and push messages related to those changes to Kafka; Sink Connectors and kafka-research-consumer that listen messages from Kafka and insert/update documents in Elasticsearch; finally, ksqlDB-Server that listens to some topics in Kafka, does some joins and pushes new messages to new topics

An Avro Library that emphasizes testability and ease of use.

Web Based Kafka Consumer and Producer

Avro Kafka Storm ⭐ 57

Avro Kafka and Strom integration

Insulator ⭐ 56

A client UI to inspect Kafka topics, consume, produce and much more

Kafka Serde Scala ⭐ 55

Implicitly converts typeclass encoders to kafka Serializer, Deserializer, Serde.

Framework to work with Kafka, Avro and ActiveRecord

Messenger Kafka ⭐ 54

Simple Kafka transport for Symfony Messenger.

Kafka Connect Elasticsearch Source ⭐ 54

Kafka Connect Elasticsearch Source

Ecommerce App ⭐ 50

eCommerce application is a platform where customers can find products, shop around using a cart, check out the products and initiate payments. It is used for the didactic purpose only, as a support project for my training at https://www.ionutbalosin.com/training

Kafka Connect Rest ⭐ 48

Kafka Connect REST connector

Php Kafka Lib ⭐ 48

PHP Kafka producer / consumer library with PHP Avro support, based on php-rdkafka

Kafka Zsh Completions ⭐ 46

ZSH plugin to have Kafka automatic completion for most CLI tools

Java 11 Examples ⭐ 46

JDK 11 examples and demo projects.

Learn You Some Kafka ⭐ 43

Atomic, bite-size, highly-focused interactive tutorials that teach you how to work with Kafka

Kafka Scala Examples ⭐ 42

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Standalone alternatives to Kafka Connect Connectors

Etl Light ⭐ 38

A light Kafka to HDFS/S3 ETL library based on Apache Spark

Kafka Sink Azure Kusto ⭐ 38

Kafka sink for Kusto

Schema Registry Transfer Smt ⭐ 38

Kafka Connect Transform to copy Avro schemas between Schema Registries

JR: Quality Random Data from the Command line

Kotlin Kafka And Kafka Streams Examples ⭐ 36

Kafka with KafkaReactor and Kafka Streams Examples in Kotlin

Kafka Connect Cosmosdb ⭐ 35

Kafka Connect connectors for Azure Cosmos DB

Kafka Connect Hbase ⭐ 33

Kafka Connect to Hbase

Registryless Avro Converter ⭐ 33

An avro converter for Kafka Connect without a Schema Registry

Kafka Tutorial Simple Client ⭐ 32

Kafka Autocomplete ⭐ 32

Bash completion for Kafka command line utilities.

Kafka Message Scheduler ⭐ 31

Scheduler for low-frequency and long-term scheduling of delayed messages to Kafka topics.

Collecting Data ⭐ 31

This is a HOWTO for collecting data in Ruby and Python applications and sending it to S3 via Kafka.

Framework Of Bigdata ⭐ 30

大数据面试题，从0到1走向架构师之路。Flink、Spark、Hive、HBase、Hadoop、K

Buildoop ⭐ 29

Hadoop Ecosystem Builder: Build, package, test and deploy your Hadoop ecosystem project.

Kafka Compose ⭐ 28

🎼 Docker compose files for various kafka stacks

Kafka Tool Avro ⭐ 27

Avro plugin for Kafka Tool

Go Kafka Avro ⭐ 26

Library to work with Apache Kafka using Avro serialization format

Spring Schema Registry ⭐ 26

The purpose of this module is to solve the problem of multiple keystores using Spring Boot + Kafka + Schema Registry + SSL and Spring Schema Resgistry Test

Go Kafka Avro ⭐ 26

A library provides consumer/producer to work with kafka, avro and schema registry

Singlestore Logistics Sim ⭐ 25

Scalable package delivery logistics simulator built using SingleStore and Vectorized Redpanda

Kafka To Avro Writer ⭐ 25

Kafka to Avro Writer based on Apache Beam. It's a generic solution that reads data from multiple kafka topics and stores it on in cloud storage in Avro format.

Pulsar Io Kafka ⭐ 24

Pulsar IO Kafka Connector

Red Pnda ⭐ 24

Minimal viable set of the core components for app developers

Kafkacli ⭐ 24

CLI and Go Clients to manage Kafka components (Kafka Connect & SchemaRegistry)

Affinity ⭐ 24

Library and a Framework for building fast, scalable, fault-tolerant Data APIs based on Akka, Avro, ZooKeeper and Kafka

Spring Cloud Stream Event Sourcing Testcontainers ⭐ 24

Goal: create a Spring Boot application that handles users using Event Sourcing. So, whenever a user is created, updated, or deleted, an event informing this change is sent to Kafka. Also, we will implement another application that listens to those events and saves them in Cassandra. Finally, we will use Testcontainers for end-to-end testing.

Spring Kafka Event Sourcing Sampler ⭐ 24

Showcases how to build a small Event-sourced application using Spring Boot, Spring Kafka, Apache Avro and Apache Kafka

Scala Kafka Twitter ⭐ 23

Example integration of Kafka, Avro & Spark-Streaming on live Twitter feed

Schema Registry ⭐ 23

📙 json & avro http schema registry backed by Kafka

An opinionated Kafka producer/consumer built on top of confluent-kafka-python/librdkafka

Awesome Kafka In Clojure ⭐ 22

Awesome information for using Kafka and Confluent Platform in Clojure.

Related Searches

Java Kafka (3,314)

Kafka Zookeeper (1,229)

Docker Kafka (1,106)

Python Kafka (1,097)

Spark Kafka (979)

Scala Kafka (969)

Golang Kafka (876)

Apache Kafka (838)

Stream Kafka (790)

Streaming Kafka (627)

1-100 of 433 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2023 Awesome Open Source. All rights reserved.