High throughput asynchronous task processing on Apache Kafka
Alternatives To Decaton
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
21 hours ago1,034apache-2.0Java
Mirror of Apache Kafka
Sarama10,0231,7597 days ago152August 11, 2022327mitGo
Sarama is a Go library for Apache Kafka.
Debezium8,548543220 hours ago137September 16, 202249apache-2.0Java
Change data capture for a variety of databases. Please log issues at
a day ago274otherC
The Apache Kafka C/C++ library
Kafka Ui5,991
a day ago287apache-2.0Java
Open-Source Web UI for Apache Kafka Management
Strimzi Kafka Operator3,868221 hours ago34September 06, 2022143apache-2.0Java
Apache Kafka® running on Kubernetes
Goka2,1163172 months ago62July 12, 202216bsd-3-clauseGo
Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go.
Spring Kafka1,9041,6912254 days ago160September 19, 202247apache-2.0Java
Provides Familiar Spring Abstractions for Apache Kafka
Kaf1,85825 days ago71July 18, 202254apache-2.0Go
Modern CLI for Apache Kafka, written in Go.
Oryx1,7961622 years ago14November 25, 20181apache-2.0Java
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
Alternatives To Decaton
Select To Compare

Alternative Project Comparisons


Build Status

Decaton is a streaming task processing framework built on top of Apache Kafka. It is designed to enable "concurrent processing of records consumed from one partition" which isn't possible in many Kafka consumer frameworks.

Here is the list of property that Decaton enables by default:

  • Concurrent (either multi-threaded or asynchronous) processing of records consumed from one partition
  • Preserve ordering guarantee by record keys
  • Preserve At-least-once delivery semantics no matter in which order does records gets processed

Decaton is a library rather than a full-stack execution environment so you can easily integrate it into your existing/new JVM applications just by adding few lines of artifact dependencies.

Since it has been designed, optimized and being used for LINE's server system which produces over 1 million, I/O intensive tasks per second for each stream, its internal implementation for enabling concurrent processing of records is highly optimized and can produce ideal throughput with minimal count of servers, maximizing their resource utilization.

Getting Started / Tutorial

Please see Getting Started

When to use (and when to not)

It's good idea to use Decaton when you have requirement for high-throughput and/or low-latency processing capability with your processing logic containing I/O access against external systems (e.g, DB access, Web API access) which tends to apply certain processing latency in each tasks.

It would be better idea to look for other frameworks like Kafka Streams when you need to do complex stream processing/aggregation such as streams join, windowed processing without needing to access external storage/web APIs.

Minimum dependencies

Below is the minimum dependencies to add Decaton artifacts into your Gradle project. It's for people who prefers to try the APIs first by adding it into your project, please see Getting Started for the detailed explanation and proper use of it.

// For task producers
implementation "com.linecorp.decaton:decaton-common:$DECATON_VERSION"
implementation "com.linecorp.decaton:decaton-client:$DECATON_VERSION"
// For processors
implementation "com.linecorp.decaton:decaton-common:$DECATON_VERSION"
implementation "com.linecorp.decaton:decaton-processor:$DECATON_VERSION"


The core feature of Decaton is support for concurrent processing of records consumed from one partition. See Why Decaton part of document for the detail.

Besides that, it got a lot of unique and useful features through its adoption spread for many services at LINE which are all came out of real needs for building services. Below are some examples. See Index for the full list.

  • Retry Queuing - Retry a failed task with back-off without blocking other tasks flow
  • Dynamic Rate Limiting - Apply and update processing rate quota dynamically
  • Task Compaction - Crush preceding tasks which its processing results will be overwritten by following task


High performance is one of the biggest functionalities of Decaton so we're carefully tracking its performance transitions commit-by-commit. You can see performance tracking dashboard from here (Note that the actual performance could be more or less depending on machine resource and configuration).

How to build

We use Gradle to build this project.

./gradlew build

How to contribute


If you believe you have discovered a vulnerability or have an issue related to security, please contact the maintainer directly or send us a email to [email protected] before sending a pull request.


Copyright 2020 LINE Corporation

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
See the License for the specific language governing permissions and
limitations under the License.

See LICENSE for more detail.

Contributor Credit

Below people made great contributions to this project in early days but their name doesn't appear in commit history because we had to erase commit logs when open-sourcing it by security reasons. Thank you so much for your contributions!

Haruki Okada, Wonpill Seo, Szuyung Wang, Vincent Pericart, Ryosuke Hasebe, Maytee Chinavanichkit, Junpei Koyama, Yusuke Imai, Alex Moreno

Popular Kafka Projects
Popular Apache Kafka Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Apache Kafka
Kafka Consumer