We have confirmed that HCP services have reach healthy system state and are functioning as intended
Posted Jan 29, 2023 - 22:26 UTC
Update
HCP engineers have implemented a remediation to alleviate high latency with create/update/delete action for the platform and we are observing latency levels return to normal. We will continue to monitor the platform to make sure latency stays within acceptable bounds.
Posted Jan 29, 2023 - 21:24 UTC
Update
HCP engineers have identified the cause of high latency with create/update/delete actions for the platform and are currently working on a remediation.
Posted Jan 28, 2023 - 19:38 UTC
Monitoring
We are seeing successful create/update/delete actions for the platform but high latencies are still present. HCP engineers will continue to monitor the health of the platform.
Posted Jan 28, 2023 - 05:32 UTC
Update
We are seeing successful create/update/delete actions for the platform but high latencies are still present. HCP engineers will continue to monitor the health of the platform.
Posted Jan 28, 2023 - 02:24 UTC
Update
The updates have been applied to our production systems and we are waiting to verify the success of these changes
Posted Jan 28, 2023 - 01:25 UTC
Update
We have begun to make the aforementioned changes in production, and thus HCP create/update/destroy actions will be unavailable. This outage will last 30 minutes
Posted Jan 28, 2023 - 00:30 UTC
Identified
Through our testing in lower environments we have determined that our fix will resolve the degradation of HCP's functionality. This change will result in a 30 minute outage, to be experience in the failures of create/update/delete actions.
The change is to occur at 00:30UTC/16:30PST
Posted Jan 27, 2023 - 23:55 UTC
Update
HCP Engineers are continuing to troubleshoot the issue have discovered the path forward to resolution. To ensure the minimal impact to our service offerings we are testing the resolution in lower environments. An update will be posted when changes are to be made in production.
Customers may experience elevated latency on actions (create/update/delete) for their HCP resources prior to the fix being applied. When the changes are made to production there will be an outage where any create/update/delete action will fail. This outage is expected to last 30 minutes
Posted Jan 27, 2023 - 23:12 UTC
Update
We are currently aware of operations such as create, updates and deletes of clusters within HCP that may take longer to complete, investigation is ongoing. Customer Consul and Vault clusters should remain operational.
Posted Jan 26, 2023 - 23:33 UTC
Investigating
We are currently aware of operations such as create, updates and deletes of clusters within HCP that may take longer to complete. Customer Consul and Vault clusters should remain operational.