Categories: Feature Status
RESOLVED: Service disruption for analytics, file upload, and some widget setting changes
Postmortem Google has posted a detailed explanation of the cause and resolution to this issue.
11:45 pm Mountain time We are currently experiencing some features not working as expected. Our core services are not impacted. The Chat Portal, the visitor side live chat, etc. are working as expected. The specific features currently not working are the analytics, the file upload (and download) feature, and changing settings of your widgets. We are actively investigating the issue to find the root cause, and are in contact with our hosting provider.
12:40 am Mountain time Google has acknowledged a service disruption impacting the components required for our analytics, our file upload feature and the API we use to update widgets. They confirmed they are working on the issue but they have not provided an ETA for resolution yet.
1:20 am Mountain time Google, our platform provider, is still actively working the issue. The SnapEngage team is looking into possible work-arounds as well. We apologize for this service disruption. We will post an update in 30 minutes.
1:52 am Mountain time Google, our platform provider, is still actively working the issue. No work-around has been found to bypass the Google Cloud Platform disruption. We will post an update in 30 minutes.
2:22 am Mountain time Google, our platform provider, is still actively working the issue. We are in regular contact with them trying to get a status and ETA. They are going to provide an update at 3:30 am. We will pass it here too, or will update as soon as we have something new.
3:30 am Mountain time Google, our platform provider, is still actively working the issue. They are going to provide an update at 4:30 am. We will pass it here too, or will update as soon as we have something new.
4:28 am Mountain time Google, our platform provider, is now rolling out a fix for the issue. They stated a few minutes ago that they “expect a full resolution in the near future”, with the next update from them at 9:00 am Mountain Time. We are not seeing much improvements with the analytics, file upload and widget style updates yet. We will post back here as soon as we see some significant improvements, or when we have an update within an hour.
7:30 am Mountain time Google, our platform provider, is still actively working the issue. They are going to provide an update at 8:30 am. We will pass it here too, or will update as soon as we have something new.
8:59 am Mountain time Google, our platform provider, is still actively working the issue. We are very sorry the full resolution is taking so much time. As documented earlier, there is no impact on chats. Analytics are unfortunately not accessible because of this platform outage, but no data will be lost. The file upload feature is currently not available, and updating widget style, proactive chat rules, and system messages is not possible. Google is reporting that their “engineering teams are working on a complete resolution at the highest priority”.