Avro4s Schema Registry Kafka Streams Demo

Demo application for Avro4s/Schema Registry/Kafka Streams integration
Alternatives To Avro4s Schema Registry Kafka Streams Demo
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Parquet4s236215 days ago47December 15, 202212mitScala
Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.
Awesome Kafka158
2 years ago
Everything about Apache Kafka
Jsm.go1023117 days ago166May 27, 20223apache-2.0Go
JetStream Management Library for Golang
Hypercore Protocol97216752 years ago76October 09, 20204mitJavaScript
Stream that implements the hypercore protocol
Target Postgres93434 months ago17April 18, 202028mitPython
A Singer.io Target for Postgres
Confluent Kafka Certification92
a year ago2Java
This repos is to keep all the relevant informations for Confluent Certified Developer for Apache Kafka (CCDAK)
Embedded Kafka Schema Registry922a month ago31July 22, 2022mitScala
A library that provides in-memory instances of both Kafka and Confluent Schema Registry to run your tests against.
Tap Github68
5 months ago40agpl-3.0Python
A Singer tap for extracting data from the GitHub API
Pravega Samples51
6 months ago19apache-2.0Java
Sample Applications for Pravega.
Csvjson49180623 years ago22August 05, 20188mitJavaScript
csv to json
Alternatives To Avro4s Schema Registry Kafka Streams Demo
Select To Compare

Alternative Project Comparisons

Integrating Kafka Streams, Confluent Schema Registry, and Avro4s

This is an example project that demos that it is possible to integrate Kafka Streams, Confluent Schema Registry, and Avro4s.

We have a simple Kafka Streams application, for which serialization to Kafka will be done by the KafkaAvroDeserializer and KafkaAvroSerializer, which leverage the schema validation in Confluent Schema Registry. We use Avro4s to easily add Avro support to our Scala case classes.


During this demo, we assume that an installation of Confluent Platform is installed in $CONFLUENT_HOME


  • Start up Zookeeper
$CONFLUENT_HOME/bin/zookeeper-server-start $CONFLUENT_HOME/etc/kafka/zookeeper-properties
  • Start up Kafka
$CONFLUENT_HOME/bin/kafka-server-start $CONFLUENT_HOME/etc/kafka/server.properties
  • Start up Schema Registry
$CONFLUENT_HOME/bin/schema-registry-start $CONFLUENT_HOME/etc/schema-registry/schema-registry.properties
  • Boot Writer application
  • Start up ordinary console-producer and produce on input topic, and publish some messages in the key;value format
$CONFLUENT_HOME/bin/kafka-console-producer --topic values --broker-list localhost:9092 --property "parse.key=true" --property "key.separator=;"
  • Use avro-console-consumer to consume from output topic
$CONFLUENT_HOME/bin/kafka-avro-console-consumer --topic avro-values --from-beginning --zookeeper localhost:2181
  • Note that the schemas have been registered at Schema Registry
curl localhost:8081/subjects/
  • Boot Reader application
  • Start up ordinary console-consumer to consume from second output topic
$CONFLUENT_HOME/bin/kafka-console-consumer --topic upper-case-values
Popular Stream Projects
Popular Schema Projects
Popular Control Flow Categories

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Kafka Streams