InMemoryKeyValueStore without serialization/deserialization

KStreams provides an in-memory key value store (backed by a topic).

For instance:

    builder.addGlobalStore(
	Stores.keyValueStoreBuilder(
		Stores.inMemoryKeyValueStore(STORES), 
		Serdes.String(), Serdes.String()),
        STORES_TOPIC,
        Consumed.with(Serdes.String(), Serdes.String()),
        () -> new GlobalStoreUpdater<>(STORES));

declares one backed by STORES_TOPIC and called STORES. Both key and value are strings.

Being in-memory, heap is a limit to the size of the store.

We use it mostly to implement global stores for master data. Any other system keeps master data updated in the topic and we consume it directly into the global store as a convenient way to make key-value searches. It’s not declared read-only and in fact you could perform updates/deletes/inserts on it and everything should work (changes should propagate to the re of the nodes in the topology shortly afterwards).

Behind the scenes, no matter what key and value types you declare, Stores.inMemoryKeyValueStore() stores the data in a TreeMap<Bytes,byte[]>.

Although provides a KeyValueStore<K,V> abstraction, internally it works always as a KeyValueStore<byte[],byte[]>. On every search it requires to serialize provided key and deserialize found value.

We sketched a minimal implementation that doesn’t require serialization/deserialization overhead.

It works for us and maybe will work for you.