What is a presigned S3 upload, and why stop proxying bytes through your monolith?

A presigned S3 upload splits the work. Your monolith becomes the control plane: it authenticates the user, decides whether the upload is allowed, picks the object key itself, and signs a short-lived URL that permits writing to that one key. The client sends the file straight to S3 with that URL. Afterwards, the monolith checks what actually landed before treating it as real data. Your application never touches the file bytes.

Proxying uploads through the app tier is simple, but it gets expensive under load. Each in-flight upload holds a worker or connection for the entire transfer. A user on a weak mobile signal uploading a video can occupy a request slot for minutes while sending very little data. The request body has to be buffered in memory or spooled to temporary disk before you forward it. The same bytes also cross your infrastructure twice: client to app, then app to S3.

Concern Proxied through monolith Direct to S3 (presigned)
Worker time held Entire transfer, paced by the slowest client One short signing request, then one verification call
Memory/disk buffering Body buffered in app memory or temp disk None in the app tier
Bandwidth path Client → app → S3 Client → S3
Where validation happens Inline, while streaming the body Constraints set at signing time, plus a check after upload

This pattern is worth adopting if file sizes, upload concurrency or slow clients are measurably tying up your app tier. If uploads are small and infrequent, proxying may still be the simpler choice.

Who should use presigned S3 uploads?

This architecture suits teams that run a monolith, or a small number of app servers, and already store user files in S3. It fits best when one or more of these is true:

  • Uploads are large. Videos, design files, datasets, scanned documents and backups take long enough to transfer that holding an app worker for the whole upload is noticeable.
  • Uploads are frequent or arrive in bursts. Upload spikes compete with normal page and API requests for the same worker pool.
  • Clients are slow or unreliable. Mobile apps and users on weak connections stretch transfer times, and that cost lands on your app tier when you proxy.
  • The app runs behind request-size or timeout limits. Load balancers, reverse proxies and platform runtimes often cap body size or request duration. Direct uploads avoid fighting those limits.
  • You are multi-tenant. Server-generated, per-tenant key prefixes and a narrowly scoped signing role give each tenant a clear storage boundary.

It is a poor fit if you must inspect or transform bytes while they stream in, if your uploads are small enough that proxying costs nothing measurable, or if your users sit behind networks that block direct browser-to-S3 traffic. Those trade-offs are covered in the last section.

How does it work? The direct-to-S3 upload flow

The flow has three steps. The app makes every decision, but the file bytes travel only between the client and S3.

1. Client requests permission

The client sends the app a small JSON request containing the filename, size and content type. It does not include the file. This lets the app reject the upload before any bytes move. It can check that the user may upload to this tenant, that the size is within limits and that the content type is on an allowlist.

2. App authorizes and signs

If the checks pass, the app generates the object key itself, for example uploads/{tenant}/{uuid}. Do not build keys from client-supplied filenames. Those invite collisions, path tricks and guessable names. Store the original filename in your database instead.

The app then inserts a row with status pending and creates a short-lived URL for that exact key and content type. In boto3 this is generate_presigned_url("put_object", ...). In the JavaScript SDK v3 it is getSignedUrl with a PutObjectCommand. The app returns the URL and an upload ID.

3. Client PUTs, then finalizes

The client sends the file body straight to S3. Any parameter included in the signature must be sent exactly as signed. If you signed Content-Type: image/png, the PUT must carry that exact header. A missing or different value fails with a signature mismatch error.

When the PUT succeeds, the client calls a finalize endpoint with the upload ID. This step is needed because S3 does not tell your app synchronously that the object arrived. Finalize is where the app confirms the object exists and marks the row complete.

import uuid
import boto3
import requests

s3 = boto3.client("s3", region_name="us-east-1")
BUCKET = "example-app-uploads"
ALLOWED_TYPES = {"image/png", "image/jpeg", "application/pdf"}
MAX_BYTES = 25 * 1024 * 1024

def create_upload(tenant_id: str, filename: str, size: int, content_type: str) -> dict:
    if content_type not in ALLOWED_TYPES or size > MAX_BYTES:
        raise ValueError("upload rejected")
    # The server builds the key. The client filename goes to the DB row only.
    key = f"uploads/{tenant_id}/{uuid.uuid4()}"
    url = s3.generate_presigned_url(
        "put_object",
        Params={"Bucket": BUCKET, "Key": key, "ContentType": content_type},
        ExpiresIn=300,  # short-lived: 5 minutes
    )
    return {"upload_id": key, "url": url, "content_type": content_type}

def finalize_upload(key: str) -> int:
    head = s3.head_object(Bucket=BUCKET, Key=key)  # raises ClientError if missing
    return head["ContentLength"]  # compare with the size stored on the pending row

ticket = create_upload("acme", "logo.png", 48_213, "image/png")
with open("logo.png", "rb") as f:
    # Content-Type must match the signed value exactly, or S3 returns 403
    resp = requests.put(ticket["url"], data=f,
                        headers={"Content-Type": ticket["content_type"]})
resp.raise_for_status()
finalize_upload(ticket["upload_id"])  # in practice, a call to the app's finalize endpoint

A boto3 server function that validates the request, generates a server-side key and presigns a PUT. The client then uploads with the exact signed Content-Type, and a finalize step uses head_object to confirm the object exists.

Should you use a presigned PUT or a presigned POST?

S3 supports two presigned upload mechanisms, and they bind different things.

A presigned PUT URL signs the HTTP method, bucket, object key and expiry, plus any headers you choose to include at signing time, such as Content-Type. The client sends the raw bytes as the request body. The signature does not limit the body size, so a client holding a valid URL can upload anything up to the single-PUT maximum. You learn the real size only afterwards, at finalize.

A presigned POST signs a policy document: a JSON list of conditions plus an expiration. S3 checks every condition before it accepts the object. This is the only one of the two that enforces size at upload time.

Presigned PUT Presigned POST
Signature binds Method, key, expiry, signed headers Policy conditions and expiry
Size limit enforced by S3 No Yes, via content-length-range
Key flexibility Exact key Exact key or starts-with prefix
Client request Raw body multipart/form-data form
Best fit Trusted or internal clients Browser uploads

Use PUT for internal services or trusted clients, and check the size in the finalize step. Use POST when browsers upload and an oversized file should be rejected before it is stored.

For example, with boto3, call generate_presigned_post with Bucket, Key, ExpiresIn and a Conditions list such as ["content-length-range", 1, 10485760], ["starts-with", "$key", "uploads/user-123/"] and {"Content-Type": "image/png"}. This caps the upload at 10 MiB. The call returns a URL and a set of form fields, and the browser must submit those fields unchanged along with the file.

import uuid

import boto3

s3 = boto3.client("s3", region_name="eu-west-1")

BUCKET = "example-app-uploads"
MAX_BYTES = 10 * 1024 * 1024  # 10 MiB, enforced by S3 at upload time
user_prefix = "uploads/user-123/"
key = f"{user_prefix}{uuid.uuid4()}.png"  # server picks the key, never the client

post = s3.generate_presigned_post(
    Bucket=BUCKET,
    Key=key,
    # Fields are pre-filled form values returned to the client.
    # boto3 does not add conditions for them automatically,
    # so each one also needs a matching condition below.
    Fields={"Content-Type": "image/png"},
    Conditions=[
        ["content-length-range", 1, MAX_BYTES],  # reject empty or oversized files
        ["starts-with", "$key", user_prefix],    # confine writes to this user's prefix
        {"Content-Type": "image/png"},           # exact content type match
    ],
    ExpiresIn=300,  # policy valid for 5 minutes
)

print(post["url"])
print(post["fields"])

Generating a presigned POST with boto3 that limits uploads to 10 MiB, confines the key to a per-user prefix and requires an exact Content-Type, all checked by S3 before the object is stored.

How do you secure presigned URLs, keys and bucket configuration?

Server-generated keys and short expirations

Never let the client choose the full object key. If the browser sends a key and you sign it, any authenticated user can overwrite another tenant's files or write anywhere the signer can. Build the key on the server, for example uploads/{tenant_id}/{user_id}/{uuid}. Treat the user's original filename as metadata and store it in your database, not in the key. Keep expirations to minutes. The URL only needs to last long enough for the client to start the upload.

Sign with a narrow identity

A presigned URL carries the permissions of the IAM identity that signed it. If you sign with the application's general role, every URL inherits that role's reach. Use a dedicated role whose policy has one statement: Effect: Allow, Action: s3:PutObject, Resource: arn:aws:s3:::your-bucket/uploads/*. If clients run multipart uploads, also allow s3:AbortMultipartUpload on the same prefix. If the role uses temporary credentials, the URL stops working when the session expires, even if the URL's own expiry has not passed.

Treat the URL as a bearer token

Anyone holding the URL can use it. Redact query strings in access logs and APM traces. Set Referrer-Policy: strict-origin-when-cross-origin or stricter, and keep presigned URLs out of analytics events.

Bucket configuration

  • CORS: Add a rule with AllowedOrigins limited to your app's origins and AllowedMethods set to PUT and POST. Set AllowedHeaders to the headers you sign, such as Content-Type. Set ExposeHeaders: ETag so browser code can read part ETags during multipart uploads.
  • Access: Turn on Block Public Access and keep all objects private. Serve downloads through short-lived presigned GET URLs or a delivery path you control.
#!/usr/bin/env bash
set -euo pipefail
BUCKET="example-app-uploads"
ROLE="upload-signer"   # dedicated role used only to sign upload URLs
APP_ORIGIN="${APP_ORIGIN:?set APP_ORIGIN to your app's origin}"  # scheme + host of your frontend

cat > signer-policy.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:PutObject", "s3:AbortMultipartUpload"],
    "Resource": "arn:aws:s3:::example-app-uploads/uploads/*"
  }]
}
EOF
aws iam put-role-policy --role-name "$ROLE" \
  --policy-name presign-upload-only --policy-document file://signer-policy.json

cat > cors.json <<EOF
{
  "CORSRules": [{
    "AllowedOrigins": ["${APP_ORIGIN}"],
    "AllowedMethods": ["PUT", "POST"],
    "AllowedHeaders": ["Content-Type"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }]
}
EOF
aws s3api put-bucket-cors --bucket "$BUCKET" --cors-configuration file://cors.json

aws s3api put-public-access-block --bucket "$BUCKET" \
  --public-access-block-configuration \
  BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Applies a write-only IAM policy scoped to the uploads/ prefix to the dedicated signing role, sets a CORS rule limited to the app origin that exposes ETag, and turns on Block Public Access for the bucket.

How do you verify an upload actually succeeded?

A 200 response from S3 only tells the browser that S3 accepted the bytes. The client's "done" call could be replayed, forged, or sent for an object that was never written. Treat finalize as a request to check, not a confirmation. The server issues a HEAD on the expected key and compares Content-Length and Content-Type with what it authorized when it signed the URL. It updates the database only if they match.

Track each upload as a row with an explicit state:

From To Trigger Guard
pending uploaded finalize call or S3 event HEAD succeeds, size and type match the grant
pending rejected reconciliation job no object after the URL's expiry plus a grace period
uploaded verified processing job completes scan clean, sniffed type allowed
uploaded rejected processing job fails malware found or content mismatch; delete the object

Make finalize idempotent with a conditional update such as WHERE state = 'pending'. A retry or duplicate event then becomes a no-op instead of creating a second record.

Several failures need explicit handling. The upload can succeed while the finalize call fails, or the client can close the tab between the two steps. In both cases the row stays pending while the object already exists. S3 event notifications can catch this, but they may arrive late or more than once. Use them as a second trigger for the same idempotent handler, and run a periodic reconciliation job that lists stale pending rows and checks each key.

The uploaded transition should enqueue the slower work: malware scanning, magic-byte sniffing to replace the declared MIME type, thumbnailing, and transcoding. Users should not see the file or get a download link until it reaches verified.

import boto3
from botocore.exceptions import ClientError

s3 = boto3.client("s3")
BUCKET = "example-app-uploads"

def finalize_upload(conn, upload_id, enqueue_processing):
    # Shared by the client's finalize call, S3 event consumers and the reconciliation job.
    with conn.cursor() as cur:
        cur.execute("SELECT object_key, expected_size, expected_type, state"
                    " FROM uploads WHERE id = %s", (upload_id,))
        row = cur.fetchone()
    if row is None:
        return "not_found"
    key, expected_size, expected_type, state = row
    if state != "pending":
        return state  # duplicate finalize, retry or repeated event: no-op
    try:
        head = s3.head_object(Bucket=BUCKET, Key=key)
    except ClientError as e:
        if e.response["Error"]["Code"] in ("404", "NoSuchKey", "NotFound"):
            return "pending"  # nothing written yet; reconciliation decides later
        raise
    # Compare with what was signed, not with anything the client claims now.
    ok = head["ContentLength"] == expected_size and head["ContentType"] == expected_type
    new_state = "uploaded" if ok else "rejected"
    with conn.cursor() as cur:
        # Conditional update: only one concurrent caller wins the transition.
        cur.execute("UPDATE uploads SET state = %s, updated_at = now()"
                    " WHERE id = %s AND state = 'pending'", (new_state, upload_id))
        won = cur.rowcount == 1
    conn.commit()
    if not won:
        return "already_handled"
    if ok:
        enqueue_processing(upload_id)  # scan, sniff magic bytes, thumbnail; sets 'verified'
    else:
        s3.delete_object(Bucket=BUCKET, Key=key)  # never keep bytes that broke the grant
    return new_state

An idempotent finalize handler that HEADs the object, checks its size and content type against the original grant, and moves the row out of pending with a conditional UPDATE so retries and duplicate S3 events are no-ops.

How do multipart uploads and cleanup rules work for large files?

A single presigned PUT is fragile for large files. One dropped connection near the end throws away the whole transfer, and S3 caps the size of a single PUT. Multipart upload splits the object into independently uploaded parts, so a failure costs one part instead of the whole file.

Server-coordinated flow

  1. The client asks your monolith to start an upload. The server calls CreateMultipartUpload with the staging key and stores the returned UploadId against the pending record.
  2. The server returns presigned UploadPart URLs, one per part number. For very large files, issue them in batches so URLs don't expire before they're used.
  3. The client PUTs each part and records the ETag response header. In browsers, the bucket's CORS configuration must expose ETag, or the client can't read it.
  4. The client sends the part numbers and ETags to the server. The server calls CompleteMultipartUpload and then runs the verification from the previous section.

Choose a part size that keeps the part count well under the limit and makes each retry cheap. Check current S3 documentation for the minimum part size and maximum part count rather than hard-coding them. Upload a few parts in parallel and retry a failed part on its own. To resume after a crash or page reload, the server calls ListParts for the stored UploadId and hands the client URLs only for the missing parts.

Cleanup

Parts from uploads that are never completed or aborted are stored and billed, but they don't show up as objects. Add a lifecycle rule on the staging prefix with AbortIncompleteMultipartUpload and a DaysAfterInitiation value. Separately, expire staging objects that never reached verified, using either a lifecycle expiration rule or a sweep job driven by your database. Promote verified objects in one of two ways: copy them to a final prefix (large objects need a multipart copy), or tag them so that expiry rules skip them.

How can presigned uploads save engineering time?

The time savings come from removing work your team would otherwise keep doing as upload traffic grows, not from the initial build, which takes longer than a proxied endpoint.

  • Less capacity planning for the app tier. When uploads are proxied, worker count, memory and temporary disk must be sized for the slowest and largest uploads you expect. With direct uploads, the app handles one small signing request and one finalize call per file, so upload volume largely stops driving app-tier sizing.
  • Fewer timeout and body-size tuning cycles. Proxied uploads push you to raise request timeouts and body limits on every layer in front of the app. Each change touches several configs and has to be re-checked whenever infrastructure changes. Direct uploads take that path out of the equation.
  • No streaming and buffering code to maintain. You don't write or debug code that spools request bodies to disk, forwards them to S3 and cleans up temporary files after failures.
  • Resumable large uploads without custom protocols. Multipart upload with ListParts gives you resume and per-part retry. Building the same behaviour through a proxy means designing your own chunking scheme.
  • One verification path for every trigger. The idempotent finalize handler serves the client call, S3 events and the reconciliation job. Duplicate and missed notifications are handled by the same code instead of by separate special cases.
  • Fewer incidents caused by upload spikes. When slow uploads stop holding workers, a burst of uploads is less likely to starve normal requests, which means fewer pages and less time spent diagnosing them.

An illustrative estimate shows where the worker time goes. Assume a 200 MB file and a client uplink of 2 Mbit/s. Transferring 1,600 Mbit at 2 Mbit/s takes about 800 seconds, so a proxied upload holds one app worker for roughly 13 minutes. With an assumed pool of 20 workers, 20 such uploads in parallel would occupy the whole pool for that period. With presigned uploads, the same 20 files cost 20 signing requests and 20 finalize calls, each a normal short request. These inputs are assumed for illustration, not measured. Plug in your own file sizes, client bandwidth and worker counts to see whether the difference matters for your system.

When should you avoid presigned uploads, and what are the alternatives?

Presigned uploads move the bytes off your app tier, and they move several responsibilities with them. The monolith can no longer inspect or reject a file while it streams in, so all validation runs after the object already exists. Unverified objects sit in the bucket until a finalize step or a sweep deals with them. The client gets more complex: it has to handle CORS, retries, progress reporting and URL expiry, and an old frontend build can call an endpoint whose contract has changed. You also have more failure states to reconcile, and abandoned multipart uploads keep costing storage until a rule removes them. On some networks, corporate proxies or a strict Content Security Policy block direct browser-to-S3 traffic entirely.

Approach Fits when Main cost
Proxy through the monolith Small, infrequent uploads Ties up workers and request timeouts
Presigned direct-to-S3 Large or frequent uploads Client and reconciliation work
Dedicated upload service Many apps share upload logic Another service to run
Proxied fallback path Restricted networks Two code paths to maintain

The engineering cost comes early. You pay up front for client logic and reconciliation jobs. In return, once upload volume is heavy, you do less capacity planning for the app tier, see fewer timeouts and scale more simply.

For testing, run the full flow against an S3-compatible local endpoint or a dedicated dev bucket. Write explicit tests for expired URLs, a wrong content type, oversized files and a finalize call made when no upload happened. These are the cases that show whether your post-upload checks actually hold.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to presigned s3 upload, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.