$ lexprog.com

// notes from an old coder -- php, databases, and the occasional rant

[March 16, 2024] MongoDB

MongoDB Change Streams

MongoDB Change Streams: Tips & Tricks

────────────────────────────────────────────────────────

MongoDB Change Streams: Tips & Tricks

Tip: Watch for Changes

$collection = DB::connection('mongodb')->collection('posts');
$changes = $collection->watch();

foreach ($changes as $change) {
    echo "Operation: {$change['operationType']}\n";
}

Real-time notifications when data changes.

Gotcha: Change Streams Need Replica Set

Like transactions, change streams require a replica set or sharded cluster.

Tip: Filter Changes

$changes = $collection->watch([
    ['$match' => ['operationType' => ['$in' => ['insert', 'update']]]],
]);

Only watch for inserts and updates, not deletes.

Gotcha: Resume Tokens

If the stream disconnects, use the resume token to continue:

$resumeToken = $change['_id'];
$changes = $collection->watch([], ['resumeAfter' => $resumeToken]);

Tip: Full Document in Updates

By default, update events only show changed fields. Get the full document:

$changes = $collection->watch([], ['fullDocument' => 'updateLookup']);

Gotcha: Change Streams Are Blocking

The watch() iterator blocks until a change occurs. Run it in a background process or queue worker.

Tip: Integration with Laravel Events

foreach ($changes as $change) {
    event(new PostChanged($change));
}

Bridge MongoDB changes to Laravel's event system.

Gotcha: Pipeline on Watch

$changes = $collection->watch([
    ['$match' => ['fullDocument.published' => true]],
    ['$project' => ['fullDocument.title' => 1]],
]);

Filter and shape the change events.

Tip: Embed or Reference? The 80/20 Rule

If you always access data together, embed it. If you access it independently, reference it. The 16MB document size limit is the hard boundary — stay under 1MB for most documents.

Tip: Index Your Query Patterns, Not All Fields

Creating indexes on every field wastes RAM. Use explain() to find in-memory sorts and collection scans. Index only what your actual queries filter on.

Gotcha: No Transaction Rollback for Index Builds

Building an index on a large collection can take hours. If it fails midway, the partial index is silently discarded. Plan index builds during maintenance windows.

Senior Insight

Change streams in MongoDB are the equivalent of PostgreSQL's LISTEN/NOTIFY — real-time data change notifications. I've used them for cache invalidation, real-time dashboards, and cross-service synchronization. The gotcha: change streams watch the oplog, which has a limited size (typically 5% of free disk space). If your application is idle for days, the oplog may have cycled, and the change stream cursor will terminate with an 'oplog start point missing' error.

Source: MongoDB Developer Center (https://www.mongodb.com/developer/), MongoDB Engineering Blog (https://www.mongodb.com/blog/channel/engineering-blog), Studio 3T Blog (https://studio3t.com/blog/)

────────────────────────────────────────────────────────
<-- back to posts