Lychee (ex. reactive-properties)

Lychee is a library to rule all the data.

ToC

• Approach to declaring data
• Properties
• Other data-binding libraries
• Properties sample
• Sample usage in GUI application
• Persistence and Android
• SQL
• HTTP
• FAQ
• Adding to a project

Approach to declaring data

Typically, we declare data using classes:

/*data*/ class Player(
    val name: String,
    val surname: String,
    var score: Int,
)

But there are some mistakes in the example above:

• there aren't any convenient ways to manipulate the properties of arbitrary classes:

  • reflection does not play well with ProGuard/R8 and Graal,
  • kapt is slow and does not play well with separate compilation,
  • both break encapsulation by exposing field names (or values of annotations) as serialization interface,
  • none of them knows precisely how to serialize values, they just try to guess according to field types,
  • there are no standard annotations: every JSON library has its own annotations (Gson: @SerializedName and @TypeAdapter), every ORM, ActiveRecord, or another database-related thing has its own, too,
  • TypeAdapter concept is cursed: every library tries to support all standard types (how often do you need to store AtomicIntegerArray? Gson has built-in support for it, this is crazy!), and tree shakers (a.k.a. dead code eliminators) cannot figure out which ones are actually used (this requires deep understanding of reflection API and Map<Type, TypeAdapter> machinery, 10+ years of ProGuard were not enough to get this deep);

• along with interface (property names and types), this also declares implementation details (backing fields). Thus, you're getting only in-memory representation. (To workaround the issue, Realm, for example, extends your classes so getters&setters are overridden while fields are unused, and rewrites your bare field accesses, if any, to use getters&setters.) Theoretically, this can be fixed by extracting interface:

  interface Player {
      val name: String
      val surname: String
      var score: Int
   // fun copy()? can we also ask to implement equals()? no.
  }
  data class MemoryPlayer(override val …) : Player
  class JsonPlayer(private val json: JsonObject) : Player {
      override val name: String get() = json.getString("name")
      …
  }
  class SqlPlayer(private val connection: Connection) : Player {
      override val name: String get() = connection.createStatement()…
  }
  

  but implementations are 146% boilerplate;

• no mutability control. var score is mutable but not observable;

• hashCode, equals, and toString contain generated bytecode fully consisting of boilerplate;

• data class copy not only consists of boilerplate but also becomes binary incompatible after every primary constructor change;

• data class componentNs are pure evil in 99% cases: destructuring is good with positional things like Pair or Triple but not with named properties.

:persistence module provides the solution. Interface is declared by inheriting Schema:

object Player : Schema<Player>() {
    val Name = "name" let string
    val Surname = "surname" let string
    val Score = "score".mut(i32, default = 0)
}

Here, Player, string, and i32 (not int because it's Java keyword) are all subtypes of DataType. Thus, they declare how to store data both in-memory and on wire. Name, Surname, and Score are field definitions, two immutable and one mutable, based on typed key pattern.

Implementations are subtypes of Struct<SCHEMA>, so they implement storage machinery while staying decoupled from data schema:

val player: StructSnapshot<Player> = Player { p ->
    p[Name] = "John"
    p[Surname] = "Galt"
    // Score gets its default value.
}

StructSnapshot is immutable (and very cheap: it is an Array, not a HashMap) implementation. It can only be read from:

assertEquals(0, player[Player.Score])

Here, Player {} is SCHEMA.invoke(build: SCHEMA.() -> Unit) function which tries to mimic struct literal; p is StructBuilder<SCHEMA>–a fully mutable temporary object. Structs implement hashCode, equals, toString, and copy of this kind: player.copy { it[Score] = 9000 }. It creates new StructBuilder and passes it to the function you provide. (Similar thing is called newBuilder in OkHttp, and buildUpon in android.net.Uri.)

There's also a good practice to implement a constructor function which gives less chance of forgetting to specify required field values:

fun Player(name: String, surname: String) = Player { p ->
    p[Name] = name
    p[Surname] = surname
}

Properties

Properties (subjects, observables) inspired by JavaFX and Vue.js MVVM-like approach are available in :properties module. A Property provides functionality similar to BehaviorSubject in RxJava, or Property in JavaFX, or LiveData in Android Arch.

• Simple and easy-to-use
• Lightweight: persistence + properties + android-bindings define around 1500 methods including easy-to-shrink inline funs and value classes
• zero reflection (the only use of kotlin.reflect is required if you delegate your Kotlin property to a Lychee Property and eliminated by Kotlin 1.3.70+ compiler)
• Extensible: not confined to Android, JavaFX or whatever (want MPP? File an issue with sample use-cases)
• Single-threaded and concurrent (lock-free) implementations
• Ready to use Android bindings like tv.bindTextTo(prop), not ld.observe(viewLifecycleOwner) { tv.text = it }
• Some bindings for JavaFX
• Sweet with View DSLs like Splitties and TornadoFX
• Depends only on Kotlin-stdlib and Kotlin-MPP Collection utils for overheadless EnumSets
• Presentation about properties: initial problem statement and some explanations

With :persistence + :properties, it's also possible to observe mutable fields:

val observablePlayer = ObservableStruct(player)
val scoreProp: Property<Int> = observablePlayer prop Player.Score
someTextView.bindTextTo(scoreProp.map(CharSequencez.ValueOf)) // bind to UI, for example

// both mutate the same text in-memory int value and a text field:
scoreProp.value = 10
observablePlayer[Player.Score] = 20

Other data-binding libraries

to explain why I've rolled my own:

• agrosner/KBinding (MIT): similar to this, Observable-based, Android-only, depends on kotlinx.coroutines

• BennyWang/KBinding (no license): Android-only, uses annotation processing, depends on RXJava 1.3

• LewisRhine/AnkoDataBindingTest (no license): proof of concept solution from Data binding in Anko article, Android-only, depends on Anko and AppCompat

• lightningkite/kotlin-anko-observable (no license): Android-only, supports easy creation of RecyclerView adapters along with data-binding, based on lightningkite/kotlin-anko (depends on Anko and AppCompat) and lightningkite/kotlin-observable (ObservableProperty<T> and ObservableList<T>); UnknownJoe796/kotlin-components-starter (MIT)

• MarcinMoskala/KotlinAndroidViewBindings (Apache 2.0): delegates properties of Views-by-id to to Kotlin properties

Properties sample

val prop: MutableProperty<Int> = propertyOf(1)
val mapped: Property<Int> = prop.map { 10 * it }
assertEquals(10, mapped.value)

prop.value = 5
assertEquals(50, mapped.value)


val tru = propertyOf(true)
val fals = !tru // operator overloading
assertEquals(false, fals.value)

Sample usage in GUI application

Android layout (Splitties Views DSL):

setContentView(verticalLayout {
    padding = dip(16)

    addView(editText {
        id = 1 // let view save its state, focus, etc
        hint = "Email"
        bindTextBidirectionally(vm.emailProp)
        bindErrorMessageTo(vm.emailValidProp.map { if (it) null else "E-mail is invalid" })
    })

    addView(editText {
        id = 2
        hint = "Name"
        bindTextBidirectionally(vm.nameProp)
    })

    addView(editText {
        id = 3
        hint = "Surname"
        bindTextBidirectionally(vm.surnameProp)
    })

    addView(button {
        bindEnabledTo(vm.buttonEnabledProp)
        bindTextTo(vm.buttonEnabledProp.map { if (it) "Save changes" else "Nothing changed" })
        setWhenClicked(vm.buttonClickedProp)
        // ^ set flag on action
    })

}.wrapInScrollView())

JavaFX layout (using JFoenix):

children.add(JFXTextField().apply {
    promptText = "Email"
    textProperty().bindBidirectionally(vm.emailProp)
})

children.add(Label().apply {
    text = "E-mail is invalid"
    bindVisibilityHardlyTo(!vm.emailValidProp)
})

children.add(JFXTextField().apply {
    promptText = "Name"
    textProperty().bindBidirectionally(vm.nameProp)
})

children.add(JFXTextField().apply {
    promptText = "Surname"
    textProperty().bindBidirectionally(vm.surnameProp)
})

children.add(JFXButton("Press me, hey, you!").apply {
    disableProperty().bindTo(!vm.buttonEnabledProp)
    textProperty().bindTo(vm.buttonTextProp)
    setOnAction { vm.buttonClickedProp.set() }
})

Common ViewModel:

class MainVm(
    // user is backed by arbitrary data source: in-memory, database, SharedPreferences, …
    private val user: TransactionalPropertyStruct<User>
) : PersistableProperties {

    // user input

    // clone user into memory
    private val editableUser = ObservableStruct(user, false)

    // expose properties for View
    val emailProp get() = editableUser prop User.Email
    val nameProp get() = editableUser prop User.Name
    val surnameProp get() = editableUser prop User.Surname

    // handle actions

    val buttonClickedProp = propertyOf(false).clearEachAnd {
        // reset flag and perform action—patch user with values from memory
        user.transaction { t ->
            t.setFrom(editableUser, User.Email + User.Name + User.Surname)
        }
    }

    // preserve/restore state of this ViewModel (for Android)
    override fun saveOrRestore(io: PropertyIo) {
        /*
        When saving state, property values are written to io which is PropertyOutput.
        When restoring, property values are assigned from io which is PropertyInput.
        
        Infix function calls:
        */
        io x emailProp
        io x nameProp
        io x surnameProp
    }

    // some feedback for user actions

    val emailValidProp = emailProp.map { it.contains("@") }

    // compare snapshots
    private val usersDifferProp = user.snapshots().mapWith(editableUser.snapshots(), Objectz.NotEqual)

    val buttonEnabledProp = usersDifferProp and emailValidProp

}

Persistence and Android

Things available in :android-bindings:

• SharedPreferenceProperty implements Property interface, and stores data inside SharedPreferences;

• SharedPreferencesStruct is observable struct stored in SharedPreferences;

// this will copy data from player into the given SharedPreferences instance
val storedPlayer = SharedPreferencesStruct(player, getSharedPreferences(…))
val scoreProp = storedPlayer prop Player.Score
val score = storedPlayer[Player.Score]
// and this is different:
storedPlayer.transaction { p ->
    p[Score] = 100500
}

• implementing PersistableProperties helps you to save or restore the state of a ViewModel to ByteArray/Parcel by implementing a single method, without declaring symmetrical, bolierplate, and error-prone writeToParcel and createFromParcel methods and without having Android dependencies:

class SomeViewModel : PersistableProperties {
    …
    override fun saveOrRestore(io: PropertyIo) {    
        io x prop1
        io x prop2
        io x prop3
    }
}

see full save & restore example.

• JSON support built on top of android.util.JsonReader/Writer:

// reading
val jsonPlayer = """{"name":"Hank","surname":"Rearden"}"""
        .reader() // StringReader
        .json() // JsonReader
        .tokens() // TokenStream
        .readAs(Player) // StructSnapshot<Player>

val jsonPlayers = """[ {"name":"Hank","surname":"Rearden"}, ... ]"""
        .reader().json().tokens().readListOf(Player)

// writing
type.tokensFrom(value).writeTo(JsonWriter(…))

TokenStream abstraction is an iterator over tokens and it's helpful for changing schema of provided data (instead of using “mappers”), see sample transform usage.

Android RemoteViews

RemoteViews API differs from normal Views API. Thus, :android-bindings module provides separate API for this. For example,

RemoteViews(packageName, R.layout.notification).bind(
    android.R.id.text1 textTo vm.nameProp,
    android.R.id.text2 textTo vm.emailProp,
)

returns you a Property<RemoteViews>, so you can just observe it and update notification on every change.

SQL (experimental)

:sql module provides Table, a wrapper over Schema:

// trivial table. Primary key column is not mentioned within Schema
val Players = tableOf(Player, "players", "_id", i64)

With Session (implementations: Android-specific SqliteSession, general-purpose JdbcSession DON'T USE THIS SHIT TILL THE NEXT RELEASE), you're getting

• SQL templates:

val selectNameEmailBySmth = Query(
    "SELECT a.name, b.email FROM anywhere a JOIN anything b WHERE smth = ?",
    /* argument */ string,
    // return a list of positionally bound string-to-string tuples:
    structs(projection(string, string), BindBy.Position)
) // Session.(String) -> List<Struct<Tuple<String, …, String, …>>>

val updateNameByEmail = Mutation(
    "UPDATE users SET name = ? WHERE email = ?",
    string, string,
    execute()
) // Transaction.(String, String) -> Unit

(To be clear, the receivers are not exactly Session and Transaction. You can call a mutation just on a session, or query in a transaction.)

• Triggers:

val listener = session.observe(
    UserTable to TriggerEvent.INSERT,
    UserTable to TriggerEvent.DELETE,
) { report ->
    val userChanges = report.of(UserTable)
    println("+" + userChanges.inserted.size)
    println("-" + userChanges.removed.size)
}
…
listener.close() // unsubscribe

HTTP

HTTP is unfortunately the most popular application layer protocol. Its abilities are not restricted to passing binary or JSON bodies: there are headers, query parameters, form fields, multipart, and more.

With :http, you can declare endpoints using some of DataTypes from :persistence:

val user = GET("/user/{role}/",
    Header("X-Token"), Path("role"), Query("id", uuid),
    Response<String>())

This gives you several features:

• HTTP client templates: val getUser = okHttpClient.template(baseUrl, user, deferred(::parseUser)) => (token: String, role: String, id: UUID) -> Deferred<String>. Here you define parseUser yourself. Thus, you are free to handle responses as you want: throw exception for non-2xx responses, or return Either<HttpException, ResponseEntity>, or ignore response code at all and just parse response body;
• server-side type-safe routing: undertowRoutingHandler.add(user, ::respond, ::respondBadRequest) { token, role, id -> "response" };
• link generation: if endpoint declaration uses GET method and does not contain headers, it is possible to build URL:

GET("/user/{role}/", Path("role"), Query("id", uuid))
    .url(baseUrl, "admin", UUID.randomUUID())
    // => //user/admin/?id=0b46b157-84b9-474c-83bb-76c2ddf58e75

Hey, have you just reinvented Retrofit?

Well, yes, but actually no. Retrofit

• works only on client-side,
• requires method return types (Call, Observable, Deferred) to be tied to async framework,
• promotes Service-style interfaces.

Lychee-HTTP, on the other side,

• allows Endpoints to be both invoked from client-side and implemented at server-side,
• decouples async wrapper from return value,
• httpClient.template(endpoint) returns a function, server-side endpoint handler is a funcion, thus, no Services/Controllers.

FAQ

What's the purpose of this library?

The main purpose is MVVM/DataBinding, especially in Android where preserving ViewModel state may be quirky. ViewModel/ViewState can be declared as a set of mappings, where the values of some properties depend on some other ones.

Why not use an existing solution?

• javafx.beans.property.Property

  It was the main source of inspiration. But the class hierarchy is too deep and wide, looks like a complicated solution for a simple problem. Has no support for multithreading. Looks like unsubscription won't take effect during notification.

• android.util.Property

  A very trivial thing for animation. Has ReflectiveProperty subclass which is close to JavaFX concept (every property is a Property) not observable and reflective (and thus sad).

• io.reactivex.BehaviorSubject

  Has no read-only interface. You can either expose an Observable (without get) or a BehaviorSubject (with get and set). Has no single-threaded version. Part of non-modular, poorly designed RxJava.

• LiveData

  Confined to Handler/Looper which limits usage to Android only and complicates testing. It's also an abstract class, thus customization is limited.

• XML data-binding

  Uses XML layouts (inflexible) and code generation (sucks in many ways, still breaks regularly). Ties layouts to hard-coded Java classes thus killing XML reusability.

Why version is 0.0.x?

1.x versions mean stable and compatible API/ABI. Lychee interface is not volatile, but is a subject to change, move, rename. This means that it can be used in production apps (migrations are easy), but libraries should update as fast as Lychee does. If your library does (or going to) depend on Lychee, file an issue: I will take into account which APIs do you use and maybe add a link to your library.

0.1.0 version is to be released after adding mutational and linearization tests, 1.0.0 is planned after dropping workarounds for KT-24981: @JvmSynthetic for classes, KT-24067: type checking and casting of multi-arity function objects ).

Where and how should I dispose subscriptions?

When the property is not being observed, it not observes its source and thus not being retained by it. Consider the following code:

val someGlobalProp = propertyOf(100)
val mappedProp = someGlobalProp.map { it * 10 }
// mappedProp has no listeners and thus not observes someGlobalProp

println(mappedProp.value) // value calculated on demand

mappedProp.addChangeListener { ... }
// mappedProp now listens for someGlobalProp changes
// and not eligble for GC until someGlobalProp is not

someGlobalProp.value = 1
// mappedProp value calculated due to original value change
// mappedProp's listener was notified

All Android bindings are based on bindViewTo which creates a Binding. It is a flyweight observing View attached state, Activity started state, and Property changes. When view gets attached to window, Binding is getting subscribed to Activity lifecycle via Lifecycle-Watcher; when Activity is started, Binding listens for data source. When Activity gets stopped or View gets detached, binding unsubscribes and becomes eligible for garbage collection along with the whole View hierarchy.

How much black magic do you use under the hood?

Some operator overloading, some value classes, several compilation error suppressions, tons of unchecked casts. No reflection, zero annotation processing. If you encounter any problems, they most likely will be related to type inference or Java interop.

Is there anything similar to RxJava's Single?

Nope. Java since v. 1.8 contains CompletableFuture for async computations. It also was backported to Java 6.5 Android a long time ago. Note that it is distributed under “GPL v. 2.0 with classpath exception” which is not as restrictive as GPL itself.

You can mutate concurrent properties from background threads (e.g. in the end of async computations), triggering UI state change as needed and without any callbacks.

ProGuard rules for Android?

Here you are.

Adding to a project

Properties

Persistence

Extended Persistence

Android Bindings

HTTP

SQL (experimental)

Android JSON

Android JSON on JVM

// `allprojects` section of top-level build.gradle || root of module-level build.gradle
repositories {
    ...
    mavenCentral()
    maven { url "https://jitpack.io" } // our dependency, Collection-utils, still lives there
}

// module-level build.gradle
dependencies {
    // I am a bit dumb, so with my first publication on Maven Central I ended up having empty <dependencies> for some of artifacts
    implementation "com.github.Miha-x64.Kotlin-MPP_Collection_utils:Collection-utils-jvm:1.0-alpha05"
  
    def lychee = '0.0.17'
//  val lychee = "0.0.17"
    implementation("su.lychee:properties:$lychee") // observables for both JVM and Android
    implementation("su.lychee:persistence:$lychee") // persistence for JVM and Android
    implementation("su.lychee:extended-persistence:$lychee") // partial structs, tuples, either, unsigned, primitive[], token transforms
    implementation("su.lychee:android-bindings:$lychee") // AAR for Android(x): View bindings, Parcel, SharedPreferences as Struct, Handler as Executor
    implementation("su.lychee:android-json:$lychee") // android.util.JsonReader as TokenStream
    implementation("su.lychee:android-json-on-jvm:$lychee") // implements android.util.JsonReader for server and desktop, use with android-json outside of Android 
    implementation("su.lychee:sql:$lychee") // observable SQL and SQL templates
    implementation("su.lychee:http:$lychee") // RPC over HTTP: client-side HTTP templates, server-side routing, type-safe link generator
}