Offset Keys

The Offset Key is a marker within the Data Partition which:

• Can be represented with 1 more keys but most commonly as a Timestamp

• Indicates where processing is up to acting as a resume point.

• Uniquely exists within the set of Partition Keys

• Is sequential in nature expecting the newer Offset to be greater or equal to its previous

Depending on Data Ingestion needs the following are all good choices for an Offset Key:

• Last Updated - The timestamp on a request indicating when the data was last updated

• Modified Date - The timestamp of a file that of when it was last updated

• Sequence Id - An id within the dataset that is automatically increased over time

• Timestamp - A timestamp within the provided data (e.g. a timeseries)

Avoid choosing an Offset Key that requires inspecting the response payload to determine