Haskell distributed stream processing with exactly-once semantics
Alternatives To Hailstorm
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Tokio Tungstenite1,232272293 days ago23July 13, 202212mitRust
Future-based Tungstenite for Tokio. Lightweight stream-based WebSocket implementation
Node Libspotify1031047 years ago4September 25, 201329C++
Node bindings for the libspotify C library
Bencode Go1019273 years ago1August 13, 20184otherGo
A Go language binding for encodeing and decoding data in the bencode format that is used by the BitTorrent peer-to-peer file sharing protocol.
Reason Nodejs9822 years ago9May 14, 202015mitReason
Node bindings for Reason and Bucklescript
9 years ago1apache-2.0Haskell
Haskell distributed stream processing with exactly-once semantics
Ruby Bencode7666144 years ago13September 04, 2014mitRuby
Ruby bindings for the bencode data serialization format
Node Ogg6323142 years ago12March 01, 20197mitJavaScript
Node.js native binding to libogg
Minizlib59172,98286a year ago20August 14, 20205otherJavaScript
A smaller, faster, zlib stream built on http://npm.im/minipass and Node.js's zlib binding.
Xz2 Rs57148934 months ago8June 06, 202228apache-2.0Rust
Bindings to liblzma in Rust (xz streams in Rust)
Rsoundio43112 years ago6May 03, 20169otherRust
Rust binding for libsound.io
Alternatives To Hailstorm
Select To Compare

Alternative Project Comparisons


Hailstorm is a distributed stream computation system that uses exactly once semantics.

Written by Thomas Dimson (@cosbynator) and Milind Ganjoo (@mganjoo).


The architecture of Hailstorm is based on Apache Storm (which is also the inspiration for the name).

The exactly-one semantics implemented in Hailstorm are based on a high-level description in an essay by @jasonjckn.



Hailstorm requires Apache Zookeeper, its C bindings, and its Haskell bindings hzk to run.

On OSX, the zookeeper package on homebrew contains the binaries and C bindings for Zookeeper. You can install it as follows:

 brew install --c zookeeper

On Ubuntu, we recommend following the official instructions to obtain and set up the Zookeeper binaries. To install the C bindings:

 sudo apt-get install libzookeeper-mt-dev

Finally, to build and install hzk on Mac OS X, run the following command in your cabal sandbox:

 cabal install --extra-include-dirs=/usr/local/include/zookeeper hzk

The above command is required on Mac OS X because of the non-standard include directory location. On Ubuntu, cabal install hzk should work.


Hailstorm requires Apache Kafka to be installed and operating. See the official instructions for details.


Hailstorm uses Haskakafka, the Haskell bindings written by our very own @cosbynator. Haskakfka, in turn, depends on librdkafka (see the Haskakafka project page for installation instructions).

Haskakafka itself is not available yet on Cabal, so install cabalg and c2hs into your sandbox:

cabal install c2hs
cabal install cabalg
.cabal-sandbox/bin/cabalg https://github.com/cosbynator/haskakafka.git

On OS X, you may get 'stdio.h' errors, in which case you should try:

.cabal-sandbox/bin/cabalg https://github.com/cosbynator/haskakafka.git -- --with-gcc=gcc-4.8

And you are done!


First, start an instance of Zookeeper (zkServer start on Mac or zkServer.sh start on Ubuntu).

Before running Hailstorm for the first time, you will have to initialize your topology using the zk_init subcommand:

 hailstorm zk_init

Finally, run a sample topology:

 hailstorm -f data/test.txt run_sample

While it is running, you can extract debug metadata by executing

 hailstorm zk_show
Popular Stream Projects
Popular Bindings Projects
Popular Control Flow Categories

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Stream Processing