Why gossip send full state


(Hsnamed) #1

Hello, i faced with problem that each node in two nodes cluster send full state of two ORMap every
distributed-data.gossip-interval.

I used akka 2.5.13, artery for transport, two actors (on each node), each actor send to replicator own changes of ORMap<String, Dgroup>.

Dgroup that type extends AbstractReplicatedData and implements ReplicatedDataSerialization.

Here’s the log

a.r.a.Encoder - sending remote message [Gossip(Map(event.3792209 -> DataEnvelope(ORMap(
group.2043064276264301 -> Dgroup{id=2043064276264301, factValues=PNCounterMap(), version=GCounter(0), factsSet=GSet()},
group.2041891226523599 -> Dgroup{id=2041891226523599, factValues=PNCounterMap(), version=GCounter(0), factsSet=GSet()},
group.2041373232281131 ->Dgroup{id=2041373232281131, factValues=PNCounterMap(), version=GCounter(0), factsSet=GSet()},
group.2042184502568969 -> Dgroup{id=2042184502568969, factValues=PNCounterMap(), version=GCounter(0), factsSet=GSet()}),Map(), VersionVector(UniqueAddress(akka://Test@127.0.0.1:2551,-663867923909857583) -> 1, UniqueAddress(akka://Test@127.0.0.1:2552,5656432729175541792) -> 1)),
event.1524823 -> DataEnvelope(ORMap(
group.209142229049073 -> Dgroup{id=209142229049073, factValues=PNCounterMap(), version=GCounter(0), factsSet=GSet()}),Map(),
VersionVector(UniqueAddress(akka://Test@127.0.0.1:2551,-663867923909857583) -> 1, UniqueAddress(akka://Test@127.0.0.1:2552,5656432729175541792) -> 1))),true)]
to [Actor[akka://Test@127.0.0.1:2552/system/ddataReplicator#55107212]] from [Actor[akka://Test/system/ddataReplicator#1630010406]]

This ORMap doesn’t change in my test but it still send full state.
As i know ORMap should be replicated by delta or full state depends on max-delta-elements .

What wrong with it ?


(Patrik Nordwall) #2

It’s right that it shouldn’t send full state gossip when the state is in sync. It’s important that the serializer produce the same bytes for equivalent content, because it’s a sha1 hash of the bytes that is sent in the Status messages to detect diverging content and need for full gossip. E.g. Elements in a Set must be sorted deterministically in the serializer.

If you can share your dgroup and serializer I could take a look and see if I spot anything that is wrong.


(Hsnamed) #3

Thank you Patrik the StatusMessage hash very helpfull i will check it.

I’ve just reproduced other strange thing , i’ve created a map of Gcounters with keys a,b,c .
I don’t set any special serializers in config all by default.

ORMap<String, GCounter>

Why it sends full state of ORMap if i increment only one counter ?

Gossip(Map(mapkey -> DataEnvelope(ORMap(a -> GCounter(5), b -> GCounter(5), c -> GCounter(7)),Map(),VersionVector(UniqueAddress(akka://Test@127.0.0.1:2551,2140052748990380887) -> 9, UniqueAddress(akka://Test@127.0.0.1:2552,-3068087603267994073) -> 8))),true)

I update ORMap in this way

Update<ORMap<String, GCounter>> upd = new Update<>(
mapDataKey,
ORMap.create(),
Replicator.writeLocal(),
map -> map.updated(node, key, GCounter.create(), (gCounter -> gCounter.increment(node, 1L))));

I caught it , is it sends full state because hash changed ? , what does mean false - in Status message

ActorSelectionMessage(Status(mapkey -> 756296aa7c54a70ebe7edbff60144ddec66820c6),Vector(system, ddataReplicator),false)

Is it possible to avoid sending full state of ORMap when some entries of map updated ?

Can you suggest the right way to implement replication process w/o sending full state of Map structure ?