KNet: the Apache Kafka .NET suite

KNet is a comprehensive .NET suite for Apache Kafka APIs providing all features: Producer, Consumer, Admin, Streams, Connect, backends (ZooKeeper and Kafka).

Libraries and Tools

KNet	KNetCLI	KNet.Templates	KNetPS	KNetConnect

Pipelines

Scope of the project

This project aims to create a set of libraries and tools to direct access, from .NET, all the features available in the Apache Kafka binary distribution.

There are many client libraries written to manage communication with Apache Kafka. Conversely, this project use directly the Java packages released from The Apache Foundation giving more than one benefit:

• all implemented features are availables at no extra implementation costs, see KNet usage;
• avoids any third party communication protocol implementation;
• access all features made available from Apache Kafka: the most important are Apache Kafka Streams and Apache Kafka Connect which does not have any C# implementation;
• measured high performance in many operating conditions.

The Apache Kafka packages are downloaded from:

kafka-clients	kafka-streams	kafka-tools	kafka_2.13

connect-runtime	connect-mirror	connect-file	connect-basic-auth-extension

Community and Contribution

Do you like the project?

• Request your free community subscription.

Do you want to help us?

• put a ⭐️ on this project
• open issues to request features or report bugs 🐛
• improves the project with Pull Requests

This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

Summary

• Roadmap
• Actual state
• Performance
• Connect SDK
• KNet usage
• KNet APIs extensibility
• KNet Serializer/Deserializer
• KNet CLI usage
• KNet Connect usage
• KNet PowerShell usage
• KNet Template usage
• How to build from scratch

News

• V1.4.4+: From version 1.4.4 there is a new project, named KNetPS, which permits to write PowerShell client scripts for an Apache Kafka cluster and many other things, here full usage.
• V1.4.7+: From version 1.4.7 there is a new project, named KNetConnect, to execute Apache Kafka Connect related jobs, here full usage.
• V1.5.4+: From version 1.5.4 there are new packages dedicated to KNet Serializer/Deserializer

Runtime engine

KNet uses JNet, and indeed JCOBridge with its features, to obtain many benefits:

• Cyber-security:
  • JVM and CLR, or CoreCLR, runs in the same process, but are insulated from each other;
  • JCOBridge does not make any code injection into JVM;
  • JCOBridge does not use any other communication mechanism than JNI;
  • .NET (CLR) inherently inherits the cyber-security levels of running JVM and Apache Kafka;
• Direct access the JVM from any .NET application:
  • Any Java/Scala class behind Apache Kafka can be directly managed: Consumer, Producer, Administration, Streams, Server-side, and so on;
  • No need to learn new APIs: we try to expose the same APIs in C# style;
  • No extra validation cycle on protocol and functionality: bug fix, improvements, new features are immediately available;
  • Documentation is shared;
• Dynamic code: it helps to write a Java/Scala/Kotlin/etc seamless language code directly inside a standard .NET application written in C#/VB.NET: look at this simple example and KNet APIs extensibility.

JCOBridge resources

Have a look at the following JCOBridge resources:

• Release notes
• Non Profit or University
• Commercial info: Professional or Enterprise
• Community Edition
• Commercial Edition
• Latest release:

KAFKA is a registered trademark of The Apache Software Foundation. KNet has no affiliation with and is not endorsed by The Apache Software Foundation.