85. An incremental pipeline receives valid events several days after their event timestamps. How would you avoid permanently missing them?
MediumAnalysis & visualization
Technical
Show model answer
Distinguish event time from ingestion time and define the supported lateness window. A watermark based only on the largest event timestamp can skip valid late arrivals unless the design explicitly accounts for them.
Reprocess an appropriate overlap or use a reliable ingestion cursor, deduplicating by stable event identity. Define correction behavior beyond the normal window and test retries, duplicates and backfills against trusted totals.
What interviewers listen for
- Separate event and ingestion time
- Combine overlap with stable deduplication
- Define correction and backfill behavior