MongoDB aggregation pipelines transform documents step by step; filtering, reshaping, grouping, and sorting them. On a collection of 1 million documents, a badly ordered pipeline can run for 45 seconds. Reorder the stages, and the same logic can finishes in under 3 seconds. Aggregation pipelines execute stages sequentially. At small scale, stage order barely matters. Once you hit 500,000+ documents, the sequence becomes the single biggest factor in runtime.

mongodb aggregation pipeline

MongoDB 6.0+ introduced an automatic optimization phase that reshapes your pipeline: it moves $match earlier, coalesces stages, and optimizes projections. Write pipelines with performance in mind anyway; the optimizer helps, but it cannot fix a fundamentally broken stage order.

Pipeline Basics

db.orders.aggregate([
  { $match: { status: "completed" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $limit: 10 }
]);

MongoDB’s Automatic Optimizations

MongoDB applies several transformations before executing your pipeline:

  • Moves independent $match filters before $project and $addFields
  • Coalesces consecutive $match, $skip, or $limit stages
  • Merges $sort + $limit into a bounded sort
  • Coalesces $lookup + $unwind + $match (on the joined array) into a single optimized $lookup with an internal sub-pipeline
  • Trims field projection so only needed fields flow through most stages

Always run explain("executionStats") to see what the optimizer actually did.

Stage Order: Still the Biggest Lever

Even with automatic optimizations, placing $match first is the primary rule.

// Recommended: filter early, then unwind
db.orders.aggregate([
  { $match: { status: "completed" } },
  { $unwind: "$items" },
  { $group: { _id: "$customerId", count: { $sum: 1 } } }
]);

Index Utilization

MongoDB can use indexes at $match and $sort, and sometimes at $group for $first/$last. Early stages benefit the most. See MongoDB Index Optimization: Boost Performance & Reduce Overhead for a deeper look at managing index overhead.

Compound Indexes and the ESR Rule

Equality fields first, Sort fields second, Range fields last.

db.orders.createIndex({ status: 1, region: 1, createdAt: -1 });

MongoDB 8.0 added distinct scans on sparse indexes for $group, delivering up to 200% higher throughput for analytical workloads; especially time-series collections using block processing. For selective $match filters on sparse fields, partial indexes can further reduce index size and scan cost.

Project Fields Early

MongoDB optimizes projection automatically in many cases, but explicitly projecting only the fields you need before expensive stages like $lookup or $unwind reduces memory pressure and serialization cost.

db.users.aggregate([
  { $match: { plan: "pro", active: true } },
  { $project: { name: 1, email: 1, _id: 1 } },
  { $lookup: { from: "subscriptions", localField: "_id", foreignField: "userId", as: "sub" } }
]);

$lookup Optimization

The slot-based execution engine (SBE), introduced in MongoDB 6.0, accelerated $lookup significantly. For maximum SBE utilization, use pipeline-style $lookup:

{
  $lookup: {
    from: "products",
    let: { productId: "$productId" },
    pipeline: [
      { $match: { $expr: { $eq: ["$_id", "$$productId"] } } },
      { $project: { name: 1, price: 1 } }
    ],
    as: "productDetails"
  }
}

SBE activates when all three conditions are met:

  • No sub-pipeline on the foreign collection (or a simple equality sub-pipeline as above)
  • Simple localField/foreignField; no numeric array paths
  • Foreign collection is neither a view nor sharded

When SBE is active, explain output shows EQ_LOOKUP instead of the generic lookup stage. Always index the foreign key field.

MongoDB 8.0: $lookup inside transactions now works with sharded foreign collections.

If you use Mongoose, the native equivalent of $lookup is Mongoose population, which has its own performance tradeoffs worth understanding.

Sharded Collections

  • Push $match filters that include the shard key as early as possible to minimize inter-shard data movement.
  • Avoid $lookup against a sharded from collection if you want SBE; sharded foreign collections disable the faster engine.
  • Use targeted shard-key filters in $lookup sub-pipelines wherever possible.

Memory and Disk

// Required for large sorts and groups that exceed 100MB in-memory limit
db.orders.aggregate([...], { allowDiskUse: true });

// Write results to another collection instead of returning a cursor
{ $merge: "customer_totals" }

Skip $unwind When Possible

MongoDB automatically coalesces $lookup + $unwind + $match into a single stage. You do not always need to unwind manually. When you must unwind:

{ $unwind: { path: "$items", preserveNullAndEmptyArrays: false } }

$facet, $bucket, and $setWindowFields

Use $facet for multi-metric analytics that need a single pass over the collection. Use $bucket for histogram-style grouping. Use $setWindowFields for running totals, moving averages, and rank calculations; all without a $group + $unwind round-trip.

Profiling with Explain

db.orders.explain("allPlansExecution").aggregate([...]);

Recent MongoDB versions add optimizationTimeMillis and slotBasedPlan.stages to the explain output. These fields show exactly what the optimizer changed and whether SBE ran. For a visual alternative, MongoDB Compass has a built-in aggregation pipeline builder with explain plan support and the ability to export pipelines to code.

Common Antipatterns

Server-side JavaScript ($where, $function, $accumulator): bypasses indexes. Deprecated in MongoDB 8.0. Use native operators.

Late $match: filtering after $unwind or $group processes far more documents than necessary.

Unbounded $graphLookup: always set maxDepth.

Unanchored $regex: patterns without a leading anchor (^) cannot use an index.

Real-World Results

1M orders collection

  • Before: 45 seconds
  • After (early $match, optimized $lookup, MongoDB 8.0 $group improvements): 2.5 seconds

10M events dashboard

  • Early $match + $project + $facet + compound index on MongoDB 8.0: sub-second response

11 Rules for Fast Aggregations

  1. Start with $match and let the optimizer help
  2. Build compound indexes following the ESR rule
  3. Project only needed fields before expensive stages
  4. Use pipeline-style $lookup and index the foreign key
  5. Rely on automatic coalescence of $lookup + $unwind + $match
  6. Use $facet for multi-metric analytics
  7. Enable allowDiskUse: true when sorting or grouping large datasets
  8. Prefer $size, $filter, $bucket, and $setWindowFields over manual unwind patterns
  9. Never use deprecated JavaScript operators
  10. Run explain() and verify IXSCAN and EQ_LOOKUP in the output
  11. Test on MongoDB 8.0+; the 200% $group throughput gain and block processing are real production wins

Official reference: MongoDB Aggregation Pipeline Optimization