To have some elements in my stream read from a cache, i try to build a Graph with the following semantics:
|---> Flow 1 (cache hit - just forward IN)
IN ---> (condition) |---> OUT
|---> Flow 2 (no cache hit - forward to underlying flow)
Proposal Partition
I could do that with Partition, but my requirement is to have the total order to be stable, so I have no clue how to merge it back together without loosing the order (or adding an id to reorder?).
Proposal flatMapConcat & Source.single*
I could also do a flatMapConcat and with two different Source.single in it. The problem is that the underlying flow can never to a mapAsync or grouping as it gets always just one element from Source.single.
Do you have any ideas or suggestions to solve that?
For ex.: you can broadcast and [filter and subflow] in each branch then merge the whole again. (This will only work if you have no async or buffer in the subflows.)
Or if the Flow1 and Flow2 just some kind of map, you can leave out the streams and do the if in that function.
Or you can build a special component for this. (But I think the buffers and asyncs can ruin your ordering anyway.)
If your usecase can handle the id+sorting can be good too, but you need to measure/calculate the memory impact correctly.
The best way to solve the problem is just doping the ordered requirement. Or outsmart it.
What does Flow 2 look like? Perhaps you are in control of the operations in Flow 2 and can simply use an element type all the way that represents either a cached result or the actual input element. The operators in flow 2 would know about this type and only do something for non-cached input elements. That means that you only have one linear flow.
@patriknw to encode if something is cached or not in a datatype is actually a very good idea and simplifies the whole thing. Thanks a lot for this hint!