[Summarization] Don't close on receiving summary ack for which there is no snapshot in storage

[agarwal-navin]

Bug / root cause

There can be scenarios such as DB rollback in storage, where the latest snapshot is lost but the corresponding summary ack is still present because ops are maintained by another service. Currentlty, if a summarizer receives an ack that is newer that the summary it's tracking, it will fetch latest snapshot and close. So, in the above scenario, the summarizer would get the ack, attempt to fetch the snapshot and close. When a new summarizer loads, it will do the same thing because the ack corresponding to the snapshot doesn't exist. The document will be stuck in this loop and reach 10k+ unsummarizer ops after which it won't be able to make any changes.

Fix

When the client gets a newer ack, it fetches the snapshot and if the fetched snapshot is older than the one for the ack, it will ignore this ack and not close. The idea being that the server will send a summary ack only after it has a snapshot present for it, so a snapshot should exist. The only reason it wouldn't exist is that it was deleted later. So, the client assumes this is the case and moves on.

Let's say this is not the case and the snapshot was not deleted but the server did not return it during snapshot fetch for some reason. The next summary upload by this client will fail because the "parent handle" (the last summary id) will be incorrect. The summarizer will close and the next summarizer will most likely not run into this issue because this is a very rare scenario.

The PR has the following changes:

• Container runtime now tracks the details of the last summary ack it has processed. It uses the information from this as the "parent" summary's handle when uploading a summary. Currently, it gets this information from the "SummaryCollection" but in the scenario described above, "SummaryCollection" has already updated the last ack to the one for which there is no snapshot. So, using that will fail the next summary upload.
• If the fetched snapshot's reference seq# is lower than the reference seq# in the ack, simply return.
• Added a property newerSnapshotPresent to the RefreshLatestSummaryAckFetch telemetry which will tell us if there was a newer snapshot present or not. I could add another telemetry when this case is hit, but I think this should be enough.

Testing

• Added a container runtime unit test that mocks the storage layer. It only has one snapshot and it always returns that. The test calls ContainerRuntime::refreshLatestSummaryAck with a newer ref seq# and then calls ContainerRuntime::submitSummary. It validates that both succeed and call storage with right properties.
• Added an end-to-end test. In the test, there is only one snapshot (initial one from detached container). The test then creates a summarizer and calls its SummaryCollection (that handles summary op / ack) with a fake summary op and summary ack whose ref seq# is greater than the initial snapshot. This simulates the scenario because from the client perspective, it got a summary ack for a snapshot that doesn't exist.

AB#8334

[msfluid-bot]

⯅ @fluid-example/bundle-size-tests: +469 Bytes

Metric Name	Baseline Size	Compare Size	Size Diff
aqueduct.js	457.07 KB	457.16 KB	⯅ +91 Bytes
azureClient.js	555.04 KB	555.14 KB	⯅ +105 Bytes
connectionState.js	680 Bytes	680 Bytes	■ No change
containerRuntime.js	258.28 KB	258.35 KB	⯅ +70 Bytes
fluidFramework.js	391.41 KB	391.43 KB	⯅ +14 Bytes
loader.js	134.1 KB	134.11 KB	⯅ +14 Bytes
map.js	42.17 KB	42.17 KB	⯅ +7 Bytes
matrix.js	145.43 KB	145.44 KB	⯅ +7 Bytes
odspClient.js	522.81 KB	522.91 KB	⯅ +105 Bytes
odspDriver.js	97.17 KB	97.19 KB	⯅ +21 Bytes
odspPrefetchSnapshot.js	42.27 KB	42.29 KB	⯅ +14 Bytes
sharedString.js	162.51 KB	162.52 KB	⯅ +7 Bytes
sharedTree.js	381.87 KB	381.88 KB	⯅ +7 Bytes
Total Size	3.26 MB	3.27 MB	⯅ +469 Bytes

---

Baseline commit: 65e6a0b

Generated by 🚫 dangerJS against 1eb9a61

[kian-thompson]

nit: I'd number these sub-bullets instead of having - to make it more readable

[agarwal-navin]

Improved the indentation. It wasn't allowing it until I disabled it.