Summary

The realtime/MQTT component was unreachable in production from 2026-07-26 08:07 UTC
to 2026-07-27 00:24 UTC (~16h17m). A production configuration gap left the
real-time messaging service without a working connection. Fixed by correcting the
configuration and verifying sustained recovery. No data loss occurred; the outage
was limited to real-time push notifications.

2026-07-26 08:02 UTC — realtime_mqtt component registered (monitoring enabled)
2026-07-26 08:07 UTC — first health probe fails; component flips to major_outage;
incident auto-opened as draft
2026-07-27 00:13 UTC — connectivity restored, configuration corrected
2026-07-27 00:24 UTC — first healthy probe after the fix; recovery confirmed
2026-07-27 10:35 UTC — component still healthy, zero consecutive probe failures
across 10+ hours of sustained monitoring

No impact to data integrity, transaction processing, or core sync functionality —
the standard sync path was unaffected throughout. Impact was limited to real-time
push signaling: clients fell back to normal periodic sync polling instead of
receiving an instant push trigger during the affected window. No customer data
was lost or at risk.

The real-time messaging service was not properly configured in production, so it
had no working connection available. This was a configuration/deployment gap, not
an application bug — the underlying code for real-time updates had already been
built and tested; production simply lacked a working connection to it.

• Corrected the production configuration to establish a proper, credentialed
  connection to the real-time messaging service.
• Added an internal tool so staff can review, publish, and resolve
  monitoring-detected incidents directly, reducing time-to-detect for gaps
  like this one going forward.
• Scheduled a broader review of production configuration to catch similar
  gaps proactively.