Skip to content

fix(storage): percent-encode object key in multipart fallback URL#4872

Merged
waleedlatif1 merged 3 commits into
stagingfrom
waleedlatif1/s3-fallback-url-encoding
Jun 3, 2026
Merged

fix(storage): percent-encode object key in multipart fallback URL#4872
waleedlatif1 merged 3 commits into
stagingfrom
waleedlatif1/s3-fallback-url-encoding

Conversation

@waleedlatif1
Copy link
Copy Markdown
Collaborator

Summary

  • buildObjectFallbackUrl constructed the object location from a raw key. A key containing spaces or reserved characters (+, ?, #) — or the pre-existing AWS branch — would yield a structurally invalid URL
  • Encode the key per path segment (preserving / separators) across all branches: AWS virtual-hosted, custom path-style, and custom virtual-hosted (R2)

Context

Addresses a Greptile review comment on the v0.6.102 release PR (#4871). Not currently triggerable — keys are generated as kb/<uuid>-<sanitizeFileName(name)> and sanitizeFileName strips everything except [a-zA-Z0-9.-], so only / can appear today. This is a robustness fix that future-proofs the location value and corrects the long-standing AWS branch.

Type of Change

  • Bug fix

Testing

  • bun run scripts/check-api-validation-contracts.ts passes; biome clean
  • location is informational (serving uses the encoded path + /api/files/serve), so no behavior change for existing valid keys

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

buildObjectFallbackUrl built the object URL from a raw key. Keys with spaces
or reserved characters (and the pre-existing AWS branch) would produce a
structurally invalid location. Encode the key per path segment (preserving
'/' separators) across all branches (AWS, custom path-style, virtual-hosted).
@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 3, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 3, 2026 9:40pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented Jun 3, 2026

PR Summary

Low Risk
Narrow change to informational fallback URLs when the SDK omits Location; serving still uses the encoded API path, and current KB keys are already sanitized.

Overview
When S3 multipart completion omits Location, buildObjectFallbackUrl now builds the fallback object URL using a percent-encoded key (each path segment encoded, / kept), instead of inserting the raw key. That applies to AWS virtual-hosted URLs and custom endpoint path-style / virtual-hosted branches, so keys with spaces or reserved characters produce valid URLs.

New unit tests cover completeS3MultipartUpload fallback behavior: SDK Location when present, AWS and MinIO/R2-style fallbacks, trailing-slash endpoints, and encoding of special characters in the key.

Reviewed by Cursor Bugbot for commit 3383635. Configure here.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Jun 3, 2026

Greptile Summary

This PR fixes buildObjectFallbackUrl to percent-encode each segment of the object key (using encodeURIComponent per segment, preserving / separators) before embedding it into the fallback URL. The fix applies consistently to all three URL shapes: AWS virtual-hosted, custom path-style (MinIO/Ceph), and custom virtual-hosted (R2).

  • client.ts: Adds a 3-line encodedKey derivation at the top of buildObjectFallbackUrl and substitutes it for the raw key in all three return branches.
  • client.test.ts: Adds completeS3MultipartUpload to the import/mock surface and six new test cases covering every code path — including SDK-provided Location, each fallback branch, trailing-slash stripping, and percent-encoding of a key containing a space.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to the fallback URL builder and does not touch the actual S3 upload path or the /api/files/serve serving path.

The fix is a 3-line addition inside a single private helper. All three return branches are covered by new tests, mock state is reset in beforeEach so tests are order-independent, and the location field is explicitly documented as informational — serving continues to use the separately-encoded path field unchanged.

No files require special attention.

Important Files Changed

Filename Overview
apps/sim/lib/uploads/providers/s3/client.ts Adds per-segment percent-encoding of the object key in buildObjectFallbackUrl, consistently applied across all three URL construction branches.
apps/sim/lib/uploads/providers/s3/client.test.ts Adds mock for CompleteMultipartUploadCommand and six focused tests covering all buildObjectFallbackUrl branches plus the encoding behavior; beforeEach correctly resets shared mock state.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["completeS3MultipartUpload(key, uploadId, parts)"] --> B["s3Client.send(CompleteMultipartUploadCommand)"]
    B --> C{response.Location present?}
    C -- Yes --> D["use response.Location directly"]
    C -- No --> E["buildObjectFallbackUrl(bucket, region, key)"]
    E --> F["encodedKey = key.split('/').map(encodeURIComponent).join('/')"]
    F --> G{S3_CONFIG.endpoint set?}
    G -- No --> H["https://bucket.s3.region.amazonaws.com/encodedKey"]
    G -- Yes --> I{forcePathStyle?}
    I -- Yes --> J["base/bucket/encodedKey (path-style)"]
    I -- No --> K["bucket.hostname/encodedKey (virtual-hosted)"]
    D & H & J & K --> L["return { location, path, key }"]
Loading

Reviews (3): Last reviewed commit: "test(storage): cover multipart fallback ..." | Re-trigger Greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@cursor review

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 16fc733. Configure here.

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@cursor review

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3383635. Configure here.

@waleedlatif1 waleedlatif1 merged commit 5efb47e into staging Jun 3, 2026
14 checks passed
@waleedlatif1 waleedlatif1 deleted the waleedlatif1/s3-fallback-url-encoding branch June 3, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant