Skip to interview prep
Apexed

85. An incremental pipeline receives valid events several days after their event timestamps. How would you avoid permanently missing them?

MediumAnalysis & visualization

Technical
Show model answer

Distinguish event time from ingestion time and define the supported lateness window. A watermark based only on the largest event timestamp can skip valid late arrivals unless the design explicitly accounts for them.

Reprocess an appropriate overlap or use a reliable ingestion cursor, deduplicating by stable event identity. Define correction behavior beyond the normal window and test retries, duplicates and backfills against trusted totals.

What interviewers listen for

  • Separate event and ingestion time
  • Combine overlap with stable deduplication
  • Define correction and backfill behavior
An incremental pipeline receives valid events several days after their event timestamps. How would you avoid permanently missing them — Data & analytics intervi