Software development methodologies, and how to choose between them
Methodology is not a preference. It is the decision that determines how early you find out you were wrong, and how much that costs. This is how we pick one, where each fits, and where each fails.
Teams argue about methodology as though it were a matter of taste. It is closer to a decision about risk: every method is a different answer to "when do we find out this was wrong".
It sets your feedback interval
Waterfall finds out at the end. Two-week sprints find out every fortnight. Continuous delivery finds out in hours. The interval is the single most consequential number in a delivery method, because it bounds how much work can be built on a wrong assumption before anyone notices.
It decides what you can change late
A method that front-loads specification buys predictability and pays for it in rigidity. One that defers decisions buys adaptability and pays in coordination overhead. Neither is better in the abstract. They are better for different constraints, and the constraint is usually contractual rather than technical.
Mismatch is more expensive than either option
The most common failure is not choosing the wrong method. It is running Scrum ceremonies on a fixed-scope fixed-price contract, or running Kanban on work that genuinely needed a committed date. The ritual survives and the benefit does not, which is worse than either method honestly applied.
It is a communication contract with the business
Methodology decides what stakeholders see and when. A method nobody outside engineering understands produces a delivery that surprises people, and surprised stakeholders intervene, which is where most mid-project chaos actually originates.
§ 02
Decide
A decision framework that takes four questions
Most methodology debates are settled by constraints, not preferences. Answer these honestly and the shortlist is usually one or two.
01Is the scope genuinely fixed?
If a contract or a regulator fixes scope and date, iterative planning cannot change either. Use a staged method with a firm discovery phase, and spend the flexibility you do have on sequencing rather than on scope.
If scope can move, an iterative method will find problems earlier and cost less overall. Agile or Scrum.
02Does the work arrive in planned batches, or continuously?
Batched, plannable work suits Scrum: sprint boundaries give a rhythm and a forecast.
Continuous or interrupt-driven work (support, platform, operations) is actively harmed by sprints. Kanban, with WIP limits and no artificial boundary.
03How complex is the domain, versus the technology?
Complex domain, ordinary technology: the risk is misunderstanding the business. Domain-Driven Design, with real discovery up front.
Simple domain, hard technology: the risk is in the build. Lean on engineering practices, TDD and continuous delivery, and keep planning light.
04Who carries the system at 3am?
If your team operates what it builds, DevOps is not optional and the delivery method has to accommodate operational work in the same backlog.
If a separate team operates it, the handover is your biggest risk, and the method needs an explicit definition of done that includes runbooks and observability.
§ 03
Family
Discovery and design
Deciding what to build, and whether it should be built at all. The cheapest place to change your mind.
Product discovery
Low risk
Discovery is the work of deciding what to build before committing to building it. Its output is not a specification but a reduction in uncertainty: which assumption is riskiest, what would prove it wrong, and what is the cheapest way to find out. Skipping it does not save the time; it moves the cost to the point where it is highest.
Use it when
The problem is understood but the solution is not.
A large investment is about to be committed on the basis of assumption.
Stakeholders disagree about what the product should do, which usually means they disagree about the problem.
Do not use it when
When the answer is genuinely known and validated. Discovery on a solved problem is procrastination with a whiteboard.
As an indefinite phase. Discovery without a decision date becomes a way of avoiding the decision.
Business benefits
Far less money spent building the wrong thing, which is the largest single category of software waste.
A defensible case for the investment, built on evidence rather than confidence.
Scope reduction, which is nearly always the outcome and nearly always welcome.
Technical benefits
Architectural constraints surface before they are expensive: data availability, integration limits, latency budgets.
The riskiest technical assumption gets a spike rather than a guess.
Common mistakes
Producing a 60-page document instead of a decision.
Asking users what they want rather than watching what they do.
Running discovery without anyone empowered to cancel the project, which removes the most valuable possible outcome.
Project size
Any
Team size
2 to 5
Typical timeline
2 to 6 weeks
Architecture fit
Produces architectural constraints rather than consuming them
Technology
Not applicable
Industries
All
Expected outcomes
A prioritised list of assumptions with evidence against the riskiest.
A go, no-go, or reshape decision.
KPIs worth tracking
Assumptions tested per week
Scope reduction from initial ask
Decisions reversed after build starts
In practice: A logistics operator plans a driver app. Two weeks of discovery, mostly spent in cabs, shows drivers cannot reliably use a phone at the points the workflow assumed. The project becomes a hardware integration and a much smaller app.
§ 04
Family
Delivery methodologies
How work is planned, sequenced and shipped once the direction is set.
Agile
Low risk
Agile is not a process. It is a set of priorities: working software over documentation, responding to change over following a plan. Scrum and Kanban are processes that implement those priorities. The distinction matters because most teams that say they are "doing Agile" have adopted a process and left the priorities behind, which produces the ceremonies without the benefit.
Use it when
Requirements are expected to change as you learn, which is nearly always true of a new product.
You have access to the people who will actually use the software.
Scope can flex even if the date cannot.
Do not use it when
A contract fixes scope, date and price simultaneously. Something has to be negotiable, or you get Agile theatre over a waterfall plan.
The work is genuinely well understood and repeated, where planning overhead buys nothing.
Stakeholders cannot give feedback more often than quarterly, which removes the mechanism the whole method depends on.
Business benefits
Wrong assumptions surface in weeks rather than quarters.
Value ships continuously instead of arriving in one release everything depends on.
Priorities can be reordered without renegotiating the entire plan.
Technical benefits
Smaller changes are easier to review, test and roll back.
Integration happens continuously, so the painful merge at the end never arrives.
Technical debt is visible in the same backlog as features, rather than invisible until it stops delivery.
Common mistakes
Running the ceremonies while keeping fixed scope, which converts a flexible method into an expensive status meeting.
Treating the backlog as a to-do list rather than a prioritised bet on value.
Letting "we are Agile" become the reason no one writes anything down. Agile devalues documentation that nobody reads, not documentation that people need.
Project size
Small to large
Team size
3 to 9 per team
Typical timeline
3 months and up
Architecture fit
Works with anything; benefits most where deployment is cheap
Technology
Stack-agnostic
Industries
Startups, Fintech, Retail, Healthcare
Expected outcomes
A working increment at the end of every cycle.
A prioritised backlog that reflects what has been learned, not what was assumed.
KPIs worth tracking
Cycle time
Escaped defect rate
Percentage of shipped work that was reprioritised mid-quarter
In practice: A product team replacing a legacy quoting tool ships the highest-volume quote type in six weeks, discovers from real usage that two of the planned fields are never filled in, and removes them before building the remaining types.
Scrum
Low risk
Scrum is Agile with a fixed cadence and defined roles. Work is committed to in time-boxed sprints, reviewed at the end, and adapted in a retrospective. Its real product is not the software but the forecast: after a few sprints you can predict throughput with some confidence, which is what makes it popular with anyone who has to answer to a board.
Use it when
Work can be planned in batches and does not arrive unpredictably.
You need a defensible forecast for stakeholders.
The team is stable enough for velocity to mean anything.
Do not use it when
Support, operations or anything interrupt-driven. Sprint commitments and unplanned work are structurally incompatible, and the commitment loses every time.
Teams below about four people, where the ceremony overhead is a meaningful share of capacity.
When leadership treats velocity as a productivity target, at which point it stops measuring anything real.
Business benefits
A predictable delivery rhythm and a forecast that improves over time.
A fixed point every sprint where priorities can legitimately change.
Visible progress for stakeholders who do not read code.
Technical benefits
The sprint boundary forces work to be broken into genuinely shippable pieces.
Retrospectives give a standing mechanism for fixing process problems before they calcify.
Common mistakes
Carrying unfinished work across sprints until the boundary means nothing.
Comparing velocity between teams, which incentivises inflating estimates and measures nothing useful.
Skipping the retrospective when busy, which removes the only part of Scrum that improves the others.
Project size
Medium to large
Team size
5 to 9
Typical timeline
6 months and up
Architecture fit
Any; pairs well with modular systems that allow parallel work
Technology
Stack-agnostic
Industries
Enterprise, Fintech, Healthcare, Government
Expected outcomes
A shippable increment every sprint.
A velocity range reliable enough to plan against.
KPIs worth tracking
Sprint goal achievement rate
Velocity stability
Carry-over percentage
In practice: A payments team runs two-week sprints against a regulatory deadline, using the forecast to decide four months out which two of five planned features can realistically ship before the date, rather than discovering it two weeks before.
Kanban
Low risk
Kanban has no iterations. Work flows continuously, limited by explicit work-in-progress caps at each stage. Those limits are the entire method: by capping how much can be in flight, they force the bottleneck to become visible instead of being absorbed as a growing queue.
Use it when
Work arrives continuously or unpredictably: support, platform, operations, maintenance.
Priorities change faster than a sprint boundary can accommodate.
You want to reduce cycle time rather than produce a forecast.
Do not use it when
When stakeholders need committed dates for a defined scope. Kanban optimises flow, not predictability against a plan.
Teams without the discipline to enforce WIP limits, where it degrades into a to-do list with columns.
Business benefits
Urgent work can be pulled in immediately without breaking a commitment.
Bottlenecks become visible as queues rather than hiding as "we are busy".
Shorter time from request to delivery for small items.
Technical benefits
Lower work in progress means less context switching and fewer half-finished branches.
Adopting the board and ignoring the WIP limits, which discards the mechanism and keeps the furniture.
Never reviewing the process because there is no retrospective built into the cadence.
Using it to avoid planning altogether, then being unable to answer when anything will be done.
Project size
Any
Team size
2 to 8
Typical timeline
Ongoing
Architecture fit
Suits systems with independent deployable parts
Technology
Stack-agnostic
Industries
Logistics, Retail, Telecommunications, Enterprise
Expected outcomes
Predictable cycle time for small items.
A visible, measurable bottleneck.
KPIs worth tracking
Cycle time
Throughput
WIP limit breaches
Queue length by stage
In practice: A platform team supporting twelve product squads caps in-progress work at four items. Requests that previously waited three weeks in an invisible queue now either start within days or are explicitly declined, which is information the requester can act on.
Lean software development
Medium risk
Lean is a lens rather than a process: identify what the customer values, then systematically remove everything that does not contribute. In software the waste is rarely typing. It is partially done work, handovers, waiting for approvals, and building features nobody uses, which is by a distance the largest category.
Use it when
Delivery feels slow but nobody can point to where the time goes.
You are validating a new product and want to build the least that proves something.
There are multiple handovers between teams.
Do not use it when
As a cost-cutting justification. Lean removes waste, not capacity, and using it to reduce headcount produces a slower system with fewer people.
In regulated contexts where some "waste" is a legally required control.
Business benefits
Less money spent building things that are never used.
Shorter time from idea to feedback.
Explicit visibility of where the delay actually happens, which is usually waiting rather than working.
Technical benefits
Smaller batches, which are cheaper to test and safer to release.
Fewer long-lived branches and less integration risk.
Common mistakes
Treating documentation and testing as waste. They are waste when nobody uses them and control when someone does.
Optimising the part of the process that is easy to measure rather than the one that is slow.
Project size
Small to medium
Team size
3 to 8
Typical timeline
3 to 12 months
Architecture fit
Favours simple architectures until complexity is justified
Technology
Stack-agnostic
Industries
Startups, Retail, Manufacturing
Expected outcomes
A measured value stream with the real bottleneck identified.
Smaller, more frequent releases.
KPIs worth tracking
Lead time
Feature usage rate after 90 days
Percentage of time work spends waiting
In practice: A team maps its release path and finds that of eleven days from merge to production, nine are spent waiting for a weekly change-approval board. No amount of faster coding would have addressed it.
§ 05
Family
Engineering practices
How the code itself is written, tested and operated. These compose with any delivery method.
DevOps
Medium risk
DevOps is the removal of the wall between the people who build software and the people who run it, usually by making them the same people. The tooling gets the attention, but the tooling is downstream of the actual change: when the team that writes the code is woken up by it, the code changes shape. Systems become observable and failure becomes something designed for rather than something apologised for.
Use it when
Your team can own production, or could be given ownership.
Releases are infrequent and stressful, which is the clearest symptom.
Incidents are recurring and nobody has time to fix the cause.
Do not use it when
As a job title or a separate team. A "DevOps team" between developers and operations recreates the wall it was meant to remove, one layer over.
Where regulation genuinely mandates separation of duties. The practices still apply; the ownership model has to be adapted rather than pretended away.
Business benefits
Releases become routine rather than events requiring a weekend.
Shorter recovery time when things break, which matters more than reducing failure frequency.
Operational cost becomes visible to the people able to change it.
Technical benefits
Automated, repeatable deployment removes an entire class of human error.
Observability built in at design time rather than added during an incident.
Infrastructure defined in code, so environments stop drifting apart.
Common mistakes
Buying tools and changing nothing about who carries the pager.
Automating a broken process, which produces failure faster.
Measuring deployment frequency alone while change failure rate quietly climbs.
In practice: A retail platform moves from monthly release weekends to daily deploys. The headline is deployment frequency, but the change that mattered was that a failed deploy now rolls back in four minutes instead of requiring an overnight restore.
Domain-Driven Design
High risk
DDD is the discipline of making the software model the business rather than the database. Its central claim is that in a complex domain, the expensive mistakes are misunderstandings, not bugs, and that the way to prevent them is to build a shared language between engineers and domain experts and then encode that language directly in the code.
Use it when
The domain is genuinely complex: insurance, clinical, logistics, regulated finance.
Different parts of the business use the same word to mean different things, which is the clearest signal that bounded contexts are needed.
You are decomposing a monolith and need principled boundaries rather than arbitrary ones.
Do not use it when
CRUD applications. A form over a database does not have a domain worth modelling, and DDD adds ceremony with nothing to buy.
When no domain expert is genuinely available. Without one it degenerates into engineers inventing a domain model, which is worse than not having one.
Under time pressure with a simple problem, where it becomes architecture as a hobby.
Business benefits
The software reflects how the business actually works, so changes to the business map to changes in the code.
Fewer requirement misunderstandings, which are the most expensive category of defect.
Boundaries that let teams work independently without constant coordination.
Technical benefits
Explicit context boundaries that stop the model becoming a single tangled object graph.
Business rules live in one place rather than being scattered across controllers and triggers.
Common mistakes
Adopting the tactical patterns (entities, value objects, aggregates) without the strategic work of finding context boundaries. That is the part that pays.
Treating one bounded context per microservice as a rule rather than a coincidence.
Modelling the whole domain up front instead of the part that is currently expensive.
Project size
Large
Team size
5 to 30 across multiple teams
Typical timeline
9 months and up
Architecture fit
Modular monolith or service boundaries drawn on contexts
Technology
Typically strongly typed languages; not a requirement
Industries
Insurance, Fintech, Healthcare, Logistics
Expected outcomes
A shared, precise language between business and engineering.
Boundaries that survive team growth.
KPIs worth tracking
Defects traced to requirement misunderstanding
Cross-team coordination overhead
Time to add a new business rule
In practice: An insurer discovers that "policy" means three different things to underwriting, claims and billing. Splitting those into separate contexts removes a class of bug that had been recurring for years and had previously been treated as a data problem.
AI-assisted development
Medium risk
AI assistance is now a real part of how software gets written, and it changes the shape of engineering work rather than simply speeding it up. Generation is cheap; review, verification and architectural judgement are not, and they are now the constraint. Teams that treat AI as an accelerator on an unchanged process tend to produce more code and more defects at the same time.
Use it when
Well-specified, verifiable work: tests, migrations, boilerplate, refactors with strong test coverage.
Exploring an unfamiliar API or codebase, where it shortens orientation considerably.
Anywhere the output can be checked automatically and quickly.
Do not use it when
Architectural decisions and context boundaries, where the cost of a plausible-but-wrong answer is highest and hardest to detect.
Domains where correctness cannot be cheaply verified, because the failure mode is confident and subtle.
As a substitute for review capacity. Generating faster than you can review moves the bottleneck rather than removing it.
Business benefits
Meaningfully faster delivery on the mechanical majority of engineering work.
Faster onboarding onto unfamiliar systems.
More engineering time available for the judgement work that actually differentiates.
Technical benefits
Test coverage becomes cheap enough that the usual excuse disappears.
Large mechanical refactors become tractable rather than being deferred indefinitely.
Common mistakes
Measuring lines of code or acceptance rate, neither of which correlates with value delivered.
Letting review standards slip because the code "looks fine". Plausibility is precisely the failure mode.
Assuming the productivity gain is uniform. It is large on boilerplate and close to zero on the hard parts.
Project size
Any
Team size
Any
Typical timeline
Ongoing
Architecture fit
Neutral; benefits most where tests are strong
Technology
Model-assisted tooling alongside the existing stack
Industries
All
Expected outcomes
Higher throughput on mechanical work.
Review capacity as the explicit, managed constraint.
KPIs worth tracking
Change failure rate
Review turnaround time
Test coverage on changed lines
In practice: A team adopting assisted development finds throughput up substantially and change failure rate up with it, until review is made the limiting step and coverage gates are enforced. Throughput settles lower than the peak and defects fall below the original baseline.
§ 06
At a glance
Everything else, compared
The practices and methods that compose with the above. Where a term already has a dedicated page here, the name links to it rather than being repeated.
Comparison of software development methodologies and engineering practices
Method
Family
Best for
Team size
What to know
Design thinking
Discovery and design
Framing an ill-defined problem before solutioning
3 to 8
Strongest where the user problem is genuinely unclear; wasteful when it is not.
Discovery workshops
Discovery and design
Aligning stakeholders who disagree
5 to 15
A forcing function for decisions, not a research method.
Product strategy
Discovery and design
Deciding what not to build this year
2 to 6
Sequencing and trade-offs; without it the roadmap becomes a wish list.
Assume the network is hostile; authenticate every call regardless of origin.
§ 07
In practice
How an engagement actually runs
Methodology in the abstract is easy to agree with. This is the concrete sequence, and what has to be true before each phase ends.
01
Technical discovery
What exists, what must not break, what the constraints genuinely are. Ends with a written shape of the work and the risks we would flag.
Gate: Both sides can state the problem the same way.
02
Architecture review
Options and trade-offs, including the ones we set aside and why. Decisions recorded so the reasoning survives the people.
Gate: Reversible and irreversible decisions explicitly separated.
03
Foundation
Environments, pipeline, observability and the first thin slice through the whole system. Deliberately before feature work.
Gate: A trivial change can reach production safely.
04
Delivery
Iterative build against the agreed cadence, with working software reviewable at every boundary.
Gate: Every increment meets the definition of done, not just the acceptance criteria.
05
Hardening
Performance, security and failure behaviour under realistic conditions rather than optimistic ones.
Gate: The system has been observed failing and recovering.
06
Operation
Running it, or handing it over with the runbooks and observability that make handover honest.
Gate: Someone who did not build it can operate it.
§ 08
The contract
Our definition of done
Applied to every increment. Without one, "done" quietly drifts to mean "the happy path works on my machine", and that drift is invisible until production finds it.
Code reviewed by someone who could maintain it.
Automated tests at the level that matches the risk, not a coverage number.
Observable in production: logs, metrics and traces that would let you diagnose it at 3am.
Security implications considered and recorded, including "none, because".
Documentation that a future maintainer would need, and nothing they would not.
Deployed behind a flag or safely rollbackable.
The team that will operate it knows it exists.
§ 09
Discipline
Governance, risk and how we stay honest
Architecture decision records
Significant decisions written down with the alternatives and the reasoning. Six months later the question is always "why did we do it this way", and memory is not a reliable answer.
Risk register, reviewed not filed
A short list of what could actually hurt this project, with an owner each. Reviewed at every phase gate rather than written once and forgotten.
Bad news early
Estimates that move come with the reason they moved, in writing, at the point we know rather than at the point it becomes undeniable.
Continuous improvement with teeth
Retrospectives that produce one change with an owner, not a list of observations. A retrospective that changes nothing is a meeting.
Definition of done as a contract
Applied to every increment. It is the mechanism that stops "done" quietly meaning "the happy path works on my machine".
Questions about methodology
Which software development methodology is best?
None of them, in the abstract. The right one is decided by four constraints: whether scope is genuinely fixed, whether work arrives in plannable batches or continuously, whether the risk sits in the domain or the technology, and who operates the system. Answer those honestly and the shortlist is usually one or two. Anyone who recommends a methodology before asking is selling a preference.
What is the difference between Agile and Scrum?
Agile is a set of priorities; Scrum is a process that implements them with fixed sprints and defined roles. You can be Agile without Scrum, most obviously with Kanban. You can also run every Scrum ceremony while being entirely un-Agile, which is more common, and happens whenever scope is fixed but the rituals are kept.
When should we use Kanban instead of Scrum?
When work arrives continuously rather than in plannable batches. Support, platform and operations teams are actively harmed by sprint commitments, because unplanned work and commitments cannot both win and the commitment always loses. Kanban optimises cycle time; Scrum optimises predictability. Pick the one whose output your stakeholders actually need.
Is Waterfall ever the right choice?
Occasionally, and dismissing it outright is a fashion rather than a judgement. Where scope is genuinely fixed by regulation or contract, where the domain is thoroughly understood, and where integration with a fixed external system dictates the sequence, a staged approach is honest. The failure is not choosing it; the failure is choosing it by default when scope could have moved.
Do methodologies matter for small projects?
Less than people think, and the overhead can exceed the benefit below about four people. What still matters at any size is the engineering practices: version control, continuous integration, review and a definition of done. Those pay from day one. The delivery ceremony can wait until there are enough people that coordination is a real cost.
How does AI change software development methodology?
It moves the constraint. Generation becomes cheap while review, verification and architectural judgement do not, so the bottleneck shifts to review capacity. Teams that bolt AI onto an unchanged process reliably produce more code and more defects simultaneously. The methodological change required is making review the explicit limiting step and strengthening the automated checks that make speed safe.
What is Domain-Driven Design used for?
Making the software model the business rather than the database, in domains complex enough that misunderstanding is the expensive risk. The signal that you need it is different parts of the business using the same word to mean different things. On a CRUD application it adds ceremony with nothing to buy, and the tactical patterns without the strategic boundary work are the most common way it is wasted.
How do you choose between a monolith and microservices?
By counting teams, not by assessing technology. Microservices solve an organisational problem: letting many teams deploy independently. Below roughly three teams the coordination cost they remove is smaller than the operational cost they add, and a modular monolith with genuine internal boundaries gives most of the benefit at a fraction of the price.
What is a definition of done, and why does it matter?
A fixed checklist every increment must satisfy before it counts as complete: reviewed, tested at a level matching the risk, observable in production, security considered, safely rollbackable. It matters because without it "done" silently drifts to mean "the happy path works locally", and that drift is invisible until something fails in production.
How long should discovery take?
Two to six weeks for most engagements, and it must have a decision date. Discovery without one becomes a way of avoiding the decision. Its output is not a specification but a reduction in uncertainty, and the most valuable possible outcome is a decision not to build, which requires someone in the room empowered to make it.
Can you work within our existing methodology?
Yes, and it is usually the right answer. Imposing a delivery process on an organisation that already has one creates a translation layer at every boundary and generates friction that has nothing to do with the software. We adapt to your cadence and your tooling. The engineering practices are where we hold a line, because those are what determine whether the code is safe to change.
What KPIs should we track for software delivery?
For delivery: deployment frequency, lead time for change, change failure rate and mean time to recovery. Those four resist gaming better than most alternatives because they trade off against each other. Avoid velocity as a cross-team measure and avoid lines of code entirely; both reliably measure something other than value.
How do you handle fixed-price, fixed-scope contracts?
By being explicit that fixing scope, date and price simultaneously removes every lever, so the risk has to be priced in somewhere and usually is. Where the commercial structure is genuinely immovable we front-load discovery and architecture, sequence to deliver the highest-risk part first, and keep the change process honest rather than pretending change will not happen.
What is platform engineering, and do we need it?
It is building an internal platform so product teams stop solving the same infrastructure problems repeatedly. It is justified when the duplicated effort across teams exceeds the cost of a team maintaining the platform, which in practice is around twenty engineers. Below that it is a platform team building for two customers, which is an expensive way to add a queue.
How do methodologies affect security?
Decisively, because security is structural. Authentication, tenancy isolation and data boundaries are design-time decisions, and retrofitting them means touching everything, usually under time pressure after something has already gone wrong. Any method that defers security to a phase before launch has already made it expensive.
What happens if the methodology is not working?
You change it, which is the point of retrospectives having teeth. The failure mode to watch for is a team that keeps the ceremonies while the benefit has gone: sprints that always carry over, retrospectives that produce no change, a board with no WIP limits. Those are signals the process has become furniture, and furniture is cheaper to move than people think.
Not sure which of these fits your project?
That is a good hour of conversation, and it is free. Bring the constraints and we will work through the shortlist with you.