Unbabel - Degraded performance due to ongoing AWS outage – Incident details

All systems operational


About This Page


Unbabel's status page is where you can get updates on how our systems are doing.
If there are interruptions to service, we will post a message here.

Degraded performance due to ongoing AWS outage

Resolved
Degraded performance
Started 6 months agoLasted about 5 hours

Affected

Unbabel

Degraded performance from 3:45 PM to 6:13 PM, Operational from 6:13 PM to 8:21 PM

Unbabel Portal

Degraded performance from 3:45 PM to 6:13 PM, Operational from 6:13 PM to 8:21 PM

Unbabel Longform

Degraded performance from 3:45 PM to 6:13 PM, Operational from 6:13 PM to 8:21 PM

Projects

Degraded performance from 3:45 PM to 6:13 PM, Operational from 6:13 PM to 8:21 PM

Client Review

Degraded performance from 3:45 PM to 6:13 PM, Operational from 6:13 PM to 8:21 PM

Salesforce KB

Degraded performance from 3:45 PM to 6:13 PM, Operational from 6:13 PM to 8:21 PM

Updates
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    With the recovery of AWS systems, Unbabel is restoring its capacity. Currently most systems are operational, with some delays still taking place for messages lost in the process and that may need recovering.

    AWS Update: Oct 20 12:15 PM PDT We continue to observe recovery across all AWS services, and instance launches are succeeding across multiple Availability Zones in the US-EAST-1 Regions. For Lambda, customers may face intermittent function errors for functions making network requests to other services or systems as we work to address residual network connectivity issues. To recover Lambda’s invocation errors, we slowed down the rate of SQS polling via Lambda Event Source Mappings. We are now increasing the rate of SQS polling as we experience more successful invocations and reduced function errors. We will provide another update by 1:00 PM PDT.

    https://health.aws.amazon.com/health/status

  • Update
    Update

    AWS update - "We continue to apply mitigation steps for network load balancer health and recovering connectivity for most AWS services. Lambda is experiencing function invocation errors because an internal subsystem was impacted by the network load balancer health checks. We are taking steps to recover this internal Lambda system. For EC2 launch instance failures, we are in the process of validating a fix and will deploy to the first AZ as soon as we have confidence we can do so safely." - https://health.aws.amazon.com/health/status

  • Identified
    Identified

    We are currently working to restore the service.