Resolved -
We have reached complete recovery and issue is already resolved.
Here are the statuses on the recovery of the failed executions during the service provider outage:
Scheduled Reports - complete
Audience Exports - complete
Broadcast Campaigns - complete
Event Exports - complete
Mar 19, 16:34 PDT
Update -
As we continue to see better performance, here are the statuses on the recovery of the failed executions during the service provider outage:
Scheduled Reports - complete
Audience Exports - in progress
Broadcast Campaigns - in progress
Event Exports - complete
Mar 19, 15:34 PDT
Monitoring -
We are seeing major improvements from our service provider and have recovered all active and new instances of the affected areas including:
◦ Report Builder and Scheduled Reports
◦ Audience Targeting/Counts
◦ Broadcast Campaigns
◦ Audience and Event Exports
◦ Snowflake Connections
◦ direct access to run queries in the Snowflake UI
Failed executions during the time of outage are currently being reprocessed for the following:
◦ Scheduled Reports
◦ Audience Exports
◦ Broadcast Campaigns
Failed executions for these service areas have completed reprocessing.
- Event Exports
Mar 19, 14:30 PDT
Update -
Our service provider has seen recovery in service health after some additional mitigation efforts. They are continuing to implement optimizations and monitor the affected environments.
Report Builder access and the Dashboard has now returned.
We are actively recovering the other affected areas of the platform.
Thank you for your patience.
Mar 19, 13:22 PDT
Update -
We are continuously monitoring the ongoing incident with our service provider. At the moment no further updates since our last communication. The provider has identified the issue and is working diligently on a resolution. We will share additional updates as soon as more information becomes available. Thank you for your patience.
Mar 19, 12:43 PDT
Update -
We continue to monitor the ongoing incident with our service provider. There are currently no further updates since our last communication. The provider has identified the issue and is working diligently on a resolution. We will share additional updates as soon as more information becomes available. Thank you for your patience.
Mar 19, 11:41 PDT
Identified -
We are experiencing issues with one of our service providers. The service provider is currently investigating and has identified the source of the issue and are developing and implementing a fix to restore service. We are continuing to evaluate the impact and are monitoring this incident.
The issues are affecting
- Report Builder and Scheduled Reports
- Audience Targeting/Counts
- Most Broadcast Campaigns starting after 9:30 PDT
- Audience and Event Exports
Unaffected items for clarity:
- Previously established tracked segments
- Experience Builder
- File-based Broadcast Campaigns
- High Priority Messaging
- UI workflows for items not listed as affected above.
Mar 19, 10:42 PDT
Investigating -
We are experiencing intermittent latency with one of our service providers. We are presently evaluating the impact and monitoring this incident. The issue seems to be initially affecting Report Builder, Audience Targeting, Counts, and Snowflake Connections.
Mar 19, 10:12 PDT