Incident report

Issue with message sending 
(Undelivered messages)

  • 04:12 AM    01:29 PM

Due to Slack API errors and Slaask server issues, some messages (aproximatively 15%) were not delivered to widget visitors.

04:12 AM- Our server monitoring detected an increased rate of undelivered messages on some widgets (less than 5% of all deployed widgets).

01:24 PM- We found an anormaly high connection count to our message queue service (twice as normal). This issue was due to anormal connections from our test servers to our production message queue, we killed those connextions, fixed the bug, and all services are now working 100% normally.

This issue is now resolved.