Software Project Cost Overrun: Why Late Rework Costs the Most

What a software project cost overrun is made of: underestimation vs. rework

A software project cost overrun is the gap between the hours you estimated and the hours you spent. That gap has two parts:

• Underestimation: work you knew about but sized too small. The migration really did take three days instead of one.
• Rework: work done twice because an assumption was wrong. The queue consumer was built against a message format the upstream team never shipped, so it was rebuilt.

Rework is usually the part teams can reduce most. You cannot add a buffer for an error you don't know you are making. You can reduce rework by finding wrong assumptions earlier.

Illustrative estimate (assumed numbers, not a benchmark)

• Overrun = underestimation hours + rework hours
• Rework share = rework hours ÷ overrun

Assume a 400-hour estimate, 60 hours of underestimation and 140 hours of rework. The overrun is 200 hours (50%), and the rework share is 140 ÷ 200 = 70%.

Some overrun is outside engineering's control: staffing churn, reorganisations, vendor delays and deadlines set before anyone scoped the work. Better practice will not fix those.

This article is for backend and platform engineers who are asked to estimate and deliver work that spans several services, teams or infrastructure layers. In that kind of work, wrong assumptions tend to stay hidden until components meet.

Who should use these practices

The practices in this article are walking skeletons, checked contracts, priced scope changes and short feedback loops. They are most useful to:

• Backend and platform engineers building services that other teams consume, or consuming services that other teams own. Most late rework happens at those boundaries.
• Tech leads and staff engineers who own an estimate for multi-team work and have to explain an overrun afterwards.
• Engineering managers who negotiate scope with product. Pricing scope changes is their job as much as the engineers'.
• Teams inheriting or replacing a system, where the existing behaviour is the real contract and much of it is undocumented.

Product managers benefit indirectly. A change-impact note gives them a cost to weigh up before they ask for something, not a surprise after.

Why rework costs more the later it is found

A wrong assumption is cheap to fix until something is built on top of it. An interface assumption, such as a field's type, whether it can be null, or what an ID means, is cheap to change while it exists only in a design document. Once code depends on it, every caller, test fixture, migration and config entry encodes it too. A fix then means changing the interface and unwinding everything that copied the assumption.

Copy            orders.customer_id  (assumed: integer)
                         │
     ┌──────────┬────────┼─────────┬──────────────┐
     ▼          ▼        ▼         ▼              ▼
  billing   shipping  test      DB migration   deploy config
  service   service   fixtures  + backfill     (feature flags,
     │          │                              env vars)
     ▼          ▼
  reports   partner webhook

The cost grows because the number of dependent artifacts grows, and each additional team adds coordination time.

Illustrative estimate (assumed numbers, not a benchmark)

Formula: cost (hours) = Σ(artifacts × hours per artifact) + (teams × coordination hours per team)

The underlying engineering change is the same in all three rows. Most of the extra cost comes from work around that change: rework in other services, data that has to be corrected, and meetings between teams.

Copy               ┌──────────────────────────────────┐
               │ orders.customer_id               │
               │ assumed type: integer            │
               └─────────────────┬────────────────┘
                                 │
     ┌─────────────┬─────────────┼──────────────┬──────────────┐
     ▼             ▼             ▼              ▼              ▼
┌─────────┐   ┌──────────┐  ┌──────────┐  ┌───────────┐  ┌───────────┐
│ billing │   │ shipping │  │   test   │  │ migration │  │  deploy   │
│ service │   │ service  │  │ fixtures │  │ +backfill │  │  config   │
└────┬────┘   └────┬─────┘  └──────────┘  └───────────┘  └───────────┘
     │             │
     ▼             ▼
┌─────────┐   ┌──────────┐
│ reports │   │ partner  │
│         │   │ webhook  │
└─────────┘   └──────────┘

Every box below the interface copies the integer assumption,
so changing the type means changing each of them as well.

One interface assumption spreads to callers, their downstream consumers, tests, migrations and deploy config, and a late fix has to unwind every one of them.

How it works: moving discovery earlier

The approach in this article does not try to stop wrong assumptions from being made. That isn't possible. It reduces how many artifacts depend on a wrong assumption by the time someone finds it. There are three levers:

1. Exercise boundaries early. A walking skeleton runs a real request through every real dependency before components are built out. Mismatches show up while only a thin slice of code depends on them.
2. Make assumptions executable. A contract test turns "billing expects customer_id to be a string" from a belief into a check that runs in CI. When a provider change breaks a consumer's assumption, the build fails on the day of the change. Nobody has to wait for an integration run weeks later.
3. Price changes when they arrive. A change-impact note makes the downstream artifacts of a scope change visible before the change is accepted. The cost is then a decision someone made, not an overrun that turns up later.

Each lever moves a discovery from a later row of the cost table above into an earlier one. Shorter feedback loops (small merges, early deploys, production-like environments) do the same for problems that contracts and skeletons don't cover.

Integration failures: the main source of late rework in backend systems

In backend systems, the rework that lands late usually comes from integration: two components that each pass their own tests but fail when connected. Four causes account for most of it.

Implied contracts are the root of most of the other problems. If a field's nullability or an enum's allowed values live only in someone's head, nothing fails until the two sides meet.

Failure semantics are where the most expensive mismatches hide. A client that retries on timeout, calling a service that is not idempotent, creates duplicate records. Nobody notices until a slow dependency triggers the retries, and by then the bad data needs correcting too.

Environment drift means the integration you tested is not the one you ship. Service-to-service auth, network policies and egress rules often exist only in production.

Data volume and shape expose queries that worked on hundreds of rows and fail on millions. They also expose encodings, null values and legacy records that no fixture included.

How to implement it: walking skeletons and explicit, checked contracts

Neither problem shows up in a mocked unit test. Both show up quickly in a walking skeleton: the thinnest end-to-end request that passes through every real dependency, such as the gateway, auth, the queue, the database and the downstream services, before any one component is built out. It doesn't need to do much. It might accept one request, write one row, publish one event and return a real response. The value is in the timing. Network policy, credentials, serialization mismatches and schema surprises show up in week one, while changing course is still cheap.

The skeleton finds today's integration problems. Explicit contracts stop new ones from appearing later. Keep each API's schema in version control, versioned alongside the code. Have each consumer state the fields and behaviors it depends on as contract tests, and run those tests against the provider in CI on every change. If the provider removes a field or tightens a type that a consumer relies on, the build fails before the change can merge. Back this up with a short written compatibility policy: additive changes are allowed, while removals and renames need a new version and a deprecation window. With that in place, a production integration failure becomes a red pipeline.

These practices have costs. On a small system owned by a single team, maintaining contract tests can cost more than the risk they remove. Tightly coupled services produce brittle tests that break on every refactor. Building a skeleton takes time upfront, and stakeholders may see that time as slow progress. When a dependency's interface is still changing every week, wait until it settles before writing contracts against it.

Copyopenapi: 3.0.3
info:
  title: Orders API
  version: 1.4.0          # minor bump = additive change only
paths:
  /v1/orders/{orderId}:
    get:
      parameters:
        - name: orderId
          in: path
          required: true
          schema: { type: string }
      responses:
        '200':
          description: A single order
          content:
            application/json:
              schema: { $ref: '#/components/schemas/Order' }
components:
  schemas:
    Order:
      type: object
      # Consumers' contract tests assert on these fields. CI runs those
      # tests against the provider, so dropping one fails the build.
      required: [id, status, totalCents]
      properties:
        id:         { type: string }
        status:     { type: string, enum: [pending, paid, shipped] }
        totalCents: { type: integer }
        giftNote:   # added in 1.4.0 as optional, so existing consumers are unaffected
          type: string
          deprecated: false

A versioned OpenAPI schema that records the compatibility policy inline: required fields that consumer contract tests depend on, and an optional field added in a minor version as a backward-compatible change.

Pricing scope changes and surfacing hidden platform work

Scope creep rarely breaks a budget because requirements change. It breaks the budget because the changes arrive without a price. "Can we also accept bulk uploads?" sounds like one endpoint. It is actually a new payload contract, a queue, retry semantics and a load test, and nobody counts that cost when the answer is "sure."

Make every change state its cost before anyone accepts it. A lightweight change-impact note covers three things:

• Interfaces: which contracts change, and which consumers must update.
• Data migrations: schema changes, backfills, and whether rollback is still possible.
• Test surface: new contract tests, fixtures and environments needed.

The note must end with an explicit trade: name what drops, or move the date. Without that line, the template is just paperwork.

Keep it to one page. If it grows into an approval board, engineers will route around it. If it is used to reject changes, it turns product and engineering into opponents. Some changes are discoveries the project needs, such as a missing failure mode or a consumer nobody knew about. Price those too, but expect to accept them.

Hidden platform work checklist

These items are left out of estimates because they are not features:

• Environments: provisioning, parity and teardown
• Observability: dashboards, alerts, log and trace coverage
• Secrets and config per environment
• CI capacity for the new pipelines and test suites
• On-call readiness: runbooks, ownership, escalation
• Data backfills and their verification
• Decommissioning the old system, including its consumers and stored data

Copychange: "Accept bulk uploads on /orders"
requested_by: product
kind: request            # request | discovery (a discovery the project needs)

interfaces:
  # Which contracts change, and who has to update
  - contract: "POST /orders/bulk (new payload, max 500 items)"
    consumers: [web-client, partner-api]
  - contract: "orders.created event gains batch_id field"
    consumers: [billing-service]

data_migrations:
  - change: "add orders.batch_id (nullable)"
    backfill: false
    rollback_possible: true   # nullable column can be dropped safely

test_surface:
  - "contract tests for bulk payload and partial-failure responses"
  - "retry/idempotency tests for the upload queue"
  - "load test at expected peak batch size"

platform:
  environments: "queue added to staging and prod"
  observability: "queue depth alert, failed-item dashboard"
  secrets_config: "queue credentials per environment"
  ci_capacity: "load test job needs a dedicated runner"
  on_call: "runbook for stuck batches; owner: orders team"
  backfills: "none"
  decommission: "none"

estimate_days: 8

trade:
  option: drop             # drop | move_date
  detail: "CSV export moves to next release"
decision: pending

A one-page change-impact note for the bulk-upload request that prices interfaces, data migrations, test surface and hidden platform work, and ends with an explicit trade.

Alternatives and complements: better estimation vs. faster feedback loops

Even with every hidden task listed, you can try to reduce overruns two ways: predict better, or learn sooner when a prediction was wrong.

Estimating better

Give ranges rather than single dates. Track how much uncertainty remains, not percent complete, because percent complete can read 90% while the hardest unknown is still untouched. Run spikes on the riskiest unknowns first so the range narrows early. A range that isn't narrowing after several weeks is an early warning that some assumption hasn't been tested yet.

The trade-off is that ranges only help if the organization accepts them. If leadership commits to one date anyway, it usually picks the low end, and the range adds nothing.

Shortening feedback loops

Estimation reduces how often you are surprised. Feedback loops reduce how much each surprise costs, because a wrong assumption found after a week takes far less rework than one found at launch.

When to use these practices

They pay off when wrong assumptions are likely and expensive to unwind:

• Work that crosses team or service boundaries. The more consumers an interface has, the more artifacts a late fix touches.
• New integrations with external or unfamiliar systems, such as a partner API, a new message broker or a legacy database whose real behaviour nobody has checked.
• Projects with data migrations or backfills. Bad data written to production is the most expensive kind of rework to reverse.
• Long-running projects with a fixed date. The longer the gap between start and first integration, the more code builds up on top of untested assumptions.
• Projects with a history of overruns, especially when the diagnostic below shows interface bugs or unpriced changes found late.

Start them at the beginning of the project. A walking skeleton built in the first week is cheap. Introducing contracts after five services already depend on each other informally still helps, but you will first have to write down the behaviour those services rely on today.

When to avoid them

Skip or scale them down when their overhead is larger than the risk they remove:

• Small systems owned by one team, where a refactor updates provider and consumer in the same change and a type mismatch fails at compile time or in a single test run.
• Throwaway prototypes and spikes, where the goal is to learn whether an idea works and the code will be discarded.
• Interfaces that are still changing weekly. Contracts written now will be rewritten constantly. Use a walking skeleton to learn, and add contracts once the shape settles.
• Tightly coupled components that always deploy together. Contract tests between them tend to be brittle and duplicate integration tests you already run.
• Organisations that won't act on the output. A change-impact note with no trade, or a range when leadership will only accept a single date, adds process without reducing overrun. Fix that agreement first.

Even in these cases, the hidden platform work checklist is still worth a few minutes. It costs little, and environments, observability and on-call setup are missing from small projects' estimates too.

Diagnostic: find where rework comes from on your current project

Pick a window long enough to include at least one integration milestone, such as the last 90 days. Pull two lists:

• Fix commits: git log --since="90 days ago" --oneline -i --grep="fix". Add --name-only to see which modules each fix touched.
• Bug tickets: in your issue tracker, filter for bug-type issues closed in the same window, and export the title, the date created and any linked change requests.

For each bug, answer three questions. Did it come from an interface, such as a mismatched schema, timeout, retry or auth assumption between services? Or did it come from a wrong assumption about requirements or data? At what stage was it found? Separately, count scope changes that arrived without a recorded trade, meaning nothing was dropped and no date or estimate moved.

Commit messages and labels are inconsistent, and many fixes are never tagged at all. Treat the tally as directional. A clear skew matters. A difference of two or three items does not.

Use the tally to choose which practice to adopt first:

• Interface bugs found in staging or production: start with explicit, checked contracts and a walking skeleton. These bugs are expensive because nothing exercised the boundary early.
• Wrong assumptions found late: shorten feedback loops with earlier demos and thin slices before you invest in better estimation.
• A high unpriced-change count: price scope changes first. The overrun is coming from your process, not your code.

Copy#!/usr/bin/env bash
set -euo pipefail

SINCE="${1:-90 days ago}"   # choose a window that covers at least one integration milestone
OUT="rework-diagnostic"
mkdir -p "$OUT"

git log --since="$SINCE" -i --grep="fix" --date=short --format="%h,%ad,%s" > "$OUT/fix-commits.csv"
echo "Fix commits: $(wc -l < "$OUT/fix-commits.csv")"

echo "Most-fixed modules:"
git log --since="$SINCE" -i --grep="fix" --name-only --format="" \
  | grep -v '^$' | cut -d/ -f1 | sort | uniq -c | sort -rn | head -10

echo -n "Fix subjects with boundary keywords: "
grep -ciE "schema|timeout|retry|auth|contract" "$OUT/fix-commits.csv" || true

if [ ! -f "$OUT/tally.csv" ]; then
  cat > "$OUT/tally.csv" <<'EOF'
id,category,stage_found,notes
BUG-101,interface,integration,order service sent amount without currency field
BUG-117,assumption,production,report assumed one account per customer
CR-042,unpriced_change,n/a,export format added; no date or scope moved
EOF
fi

echo "Tally (category,stage):"
tail -n +2 "$OUT/tally.csv" | cut -d, -f2,3 | sort | uniq -c | sort -rn

This shell script uses standard git, grep, cut, sort and uniq to list recent fix commits and the modules they touched most. It also creates a tally sheet where you classify each bug by category and the stage where it was found, then counts the rows to show which pattern stands out.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to software project cost overrun, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.