Poor performance for AI related operations

Incident Report for Rainforest QA

Postmortem

We realized that our provider for AI infrastructure was having a soft outage which was causing some requests to take a very long time (10 minutes) to error.

We worked around this by decreasing the timeout for the requests so that they would error faster and we could then retry them.

Posted Jun 11, 2024 - 06:18 PDT

Resolved

This incident has been resolved.

Posted Jun 07, 2024 - 07:46 PDT

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Jun 07, 2024 - 05:57 PDT

Investigating

Operations involving AI are taking some time to complete. We're investigating the issue.

Posted Jun 07, 2024 - 05:11 PDT

This incident affected: Visual Editor and Automation test execution.