Terraform Run Failures
Resolved

After monitoring the fix implemented on 29 March 2019, we have confirmed that Terraform runs are executing as expected and all error rates have stabilized.

Mon, Apr 1, 2019, 09:28 PM
(6 years ago)
·
Affected components

No components marked as affected

Updates

Resolved

After monitoring the fix implemented on 29 March 2019, we have confirmed that Terraform runs are executing as expected and all error rates have stabilized.

Mon, Apr 1, 2019, 09:28 PM

Monitoring

We have implemented a fix for this issue that will be gradually rolled out over the next 24 hours. We will continue to monitor Terraform runs and their service ecosystem through the weekend, in order to ensure Terraform Enterprise stabilizes after the full roll out of the fix. We will update this incident at or before 9AM PDT on 1 April 2019 to confirm the efficacy of the fix.

Fri, Mar 29, 2019, 06:08 PM(3 days earlier)

Monitoring

Error rates have stabilized and returned to normal range in the services responsible for executing Terraform runs. We are continuing to monitor these services to ensure that Terraform run functionality is fully restored. We will provide an update once we are confident all Terraform runs will plan and apply as expected.

Fri, Mar 29, 2019, 04:29 PM(1 hour earlier)

Investigating

We are currently investigating a spike of errors affecting a portion of jobs running in Terraform Enterprise. As a result some users may see failed or stalled Terraforrm Runs. We will provide updates as our investigation develops.

Fri, Mar 29, 2019, 03:55 PM(34 minutes earlier)