API Errors
Incident Report for ScaleFT
Postmortem

For 14 minutes today the ScaleFT API experienced a partial outage due to low disk space on a production database. This caused some API calls to fail, and may have impacted users' experience while authenticating with both server access and web access products.

  • 10:40am PDT 8/17/18 - The engineering team was alerted to an increased rate of HTTP request failures. Upon further investigation it was discovered that a database server had run out of disk space
  • 10:47am PDT 8/17/18 - Engineers increased the amount available disk space for the impacted database
  • 10:54am PDT 8/17/18 - Engineers were able to confirm the failing HTTP requests had stopped

We have added monitoring to ensure this will not happen again, and we are actively performing an audit of our production monitoring to identify any further gaps.

Posted 10 months ago. Aug 17, 2018 - 22:46 UTC

Resolved
The incident has been resolved
Posted 10 months ago. Aug 17, 2018 - 21:38 UTC
Monitoring
This issue should now be resolved, the ScaleFT team is continuing to monitor the status of the system.
Posted 10 months ago. Aug 17, 2018 - 18:00 UTC
Investigating
The ScaleFT API is currently returning an increased rate of error responses. The engineering team is investigating and will update this incident as we learn more.
Posted 10 months ago. Aug 17, 2018 - 17:48 UTC
This incident affected: API and Management Portal.