Skip to content

Workflow run failed with Queue callback error while recording step events (+ 403 and 429 error) #1953

@Graffioh

Description

@Graffioh

Hello, we saw a production Workflow run repeatedly fail during step execution.

Details

  • Workflow run ID: wrun_01KQX6AFJHR1247ZHKXMAY7MCD
  • Environment: production
  • Approximate time window: 2026-05-05 23:06 UTC – 2026-05-06 00:11 UTC

What happened

Several POST /.well-known/workflow/v1/step invocations failed with 500. The logged error was an internal Workflow
callback failure when posting run events (Queue callback error):

  WorkflowWorldError: POST /v3/runs/wrun_01KQX6AFJHR1247ZHKXMAY7MCD/events -> HTTP 403: Forbidden                     

Before the repeated 403 errors, we also saw callback failures caused by 429 responses with Vercel mitigation headers:

  TypeError: fetch failed                                                                                             
  cause: RequestRetryError                                                                                            
  statusCode: 429                                                                                                     
  x-vercel-mitigated: challenge

Can you check what was happening?

Edit: another recent wrun ID for the same Queue callback error: wrun_01KR59P7S0TTJ053TDSFACB3Z7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions