SnapEngage Service Status


Integrations with CRMs, Help Desks, and other 3rd parties experiencing delays

On July 13, 2016 / In / by Jerome Mouton

RESOLVED: Integrations with CRMs, Help Desks, and other 3rd parties experiencing delays
All systems are back to normal

 

July 13 7:40am Mountain time The backlog of all chat transcripts and offline messages from the early hours of the incident have been delivered. If you are unable to locate a transcript in your Help Desk or CRM, please come chat with us so that we can assist you locating it.

July 13 3:59am Mountain time Google confirmed they have mitigated the issue at 3:50am (9 minutes ago). We are seeing integration communications being back to normal. All chat transcripts and offline messages of the past hours are being delivered now. Chats and offline messages started prior to 2 hours ago and not yet in your CRM or Help Desk system might take a few additional hours to get delivered. Will we post an update once we can confirm that the backlog of all chat transcripts and offline messages have been delivered.

July 13 3:56am Mountain time Google communicated with us that their networking engineers are working on redirecting the Fetch traffic (used by 3rd party integrations) to a stable environment. They have not provided an ETA yet, but we anticipate a quick resolution now.

July 13 3:35am Mountain time Our hosting provider posted an update on the ongoing platform issue, saying they are still working on the issue, and will post another update at 03:30 US/Pacific.

July 13 2:50am Mountain time Our hosting provider published an update on the ongoing platform issue:

We are currently investigating an intermittent issue with Google App Engine URLFetch API service. Fetch requests to non-Google related services are failing with deadline exceeded errors.
For everyone who is affected, we apologize for any inconvenience you may be experiencing. We will provide an update by 02:30 US/Pacific with current details.

We will post an update as soon as one is available from Google Cloud Platform.

July 13 2:38am Mountain time Our hosting provider, Google Cloud Platform, has localized the issue causing an elevated error rate with the infrastructure we rely on for our integrations. They are working on a fix. We will post an update as soon as one is available from them, or in an hour at the latest.

July 13 1:35am Mountain time Integrations with 3rd party providers are currently experiencing delays (social discovery, weather, help desks, CRMs, knowledge bases). We are investigating the issue with our hosting provider. We will post an update within an hour.

Website content issue

On May 3, 2016 / In / by Jerome Mouton

RESOLVED: Websites content no longer available (no impact on live chat)

May 3 4:14am Mountain time The issue has been resolved. All websites are back online.
May 3 3:26am Mountain time Our website content  hosting provider posted a updated on their service disruption: https://wpenginestatus.com/read-only-on-some-servers-in-one-us-datacenter/ As the issue is not yet resolved and there is no ETA available from our provider, we are redirecting all […]

Read More

Analytics issues

On February 29, 2016 / In / by Jerome Mouton

RESOLVED: We are currently experiencing issues running queries to generate reports in the SnapEngage Analytics

Feb. 29 3:49pm Mountain time The issue has been resolved. As always we will continue to monitor things and will post any follow-up information as it becomes available.
Feb. 29 12:34pm Mountain time The issue with the analytics is subsiding but we’re […]

Read More

Chat Routing Inconsistencies

On February 23, 2016 / In / by Ryan Graham

RESOLVED: Occurrence of chat routing inconsistencies

 
Feb. 24 8:07am Mountain time Our hosting provider has replaced the server component at the root cause of the small bursts of errors 10 minutes ago. If you still experience some chat routing inconsistencies, please come chat with us. We are continuing to monitor things on our end, and everything is looking fine.
Feb. 24 6:29am […]

Read More

Service disruption for the Visitor Chat API

On February 18, 2016 / In / by Jerome Mouton

RESOLVED: Service disruption for the Visitor Chat API*

 
Postmortem: We have identified that our error monitoring did not detect the increased error rate on the API endpoints, requiring our customers to report the issue before we could escalate to our hosting provider. We have taken the corrective actions and have reconfigured our alert policies to be notified as soon as the […]

Read More