$$\ $$\ $$ | $$ | $$$$$$$ | $$$$$$$\ $$\ $$\ $$$$$$$\ $$ __$$ | $$ __$$\ $$ | $$ | $$ _____| $$ / $$ | $$ | $$ | $$ | $$ | \$$$$$$\ $$ | $$ | $$ | $$ | $$ | $$ | \____$$\ \$$$$$$$ | $$$$$$$ | \$$$$$$ | $$$$$$$ | \_______| \_______/ \______/ \_______/
dbus = distributed data bus
It is yet another lightweight versatile databus system that transfer/transform pipeline data between plugins.
dbus works by building a DAG of structured data out of the different plugins: from data input, via filter(optional), to the output.
dbus is not yet a 1.0. We're writing more tests, fixing bugs, working on TODOs.
dbus supports powerful and scalable directed graphs of data routing, transformation and system mediation logic.
To start using dbus, install Go and run
$ go get -u github.com/funkygao/dbus
Please find sample config files in etc/ directory.
$ $GOPATH/dbusd -conf $myfile
dbus uses zookeeper for sharding/balance/election.
More plugins are listed under dbus-plugin.
is it totally data loss tolerant?
if the binlog exceeds 1MB, it will be discarded(lost)
For example, 3 participants with 1 being the leader. Then 1 is network partitioned and zk session expires, [2, 3] found this event and re-elect 2 as new leader. Before 1 regain new zk session,  and  are leaders both. If  and  both found resources changes, they will both rebalance the cluster.
dbus uses epoch to solve this issue.
dbus continues to work, but Ack will not be able to persist