---

Reading Time: 10 minutes | Series: Database Dialogues #2

---

It’s 2:03 AM. My hands hover over the keyboard, heart pounding, as I prepare to run a migration on the Prayer Nook production database. One typo, one missed index, and 1,000 users could lose access to their spiritual lifeline. No pressure, right?

If you’ve ever deployed a migration with sweaty palms and a prayer (pun intended), you know the stakes. Downtime isn’t just an inconvenience—it’s a breach of trust. For nonprofits like ours, with no ops team and no “maintenance window,” zero-downtime isn’t a nice-to-have. It’s survival.

Rails 8 finally gives us the tools to make zero-downtime migrations not just possible, but practical. In this post, I’ll walk you through how we migrated critical tables in production—without a single user noticing. I’ll share the real code, the gotchas, and the Ember-approved checklist that now sits taped to my monitor.

Ember’s Opening: “A penguin never abandons the ice mid-molt. Neither should your app during a migration.” 🐧🔥

---

Why Zero-Downtime Matters (Especially for Small Teams)

Let’s be honest: downtime is expensive, but for us the real cost isn’t measured in dollars—it’s measured in lost trust. When a user tries to post a prayer request and gets an error, they might not come back. For faith-based communities and nonprofits, we don’t get a second chance to make a first impression.

No one on my team wants to send out a “scheduled maintenance” email. Our users are global, and “off hours” don’t exist. If you’re running a small team, or you’re the only developer (been there), you need migrations that just work.

---

What’s New in Rails 8 for Safe Migrations

Rails 8 delivers some fantastic migration features straight out of the box:

• Safer Column Changes: The new add_column_with_default adds non-null columns with defaults without locking the table, avoiding those terrifying “migration runs for 12 minutes” moments.
• Background Migrations (Solid Queue): Large data updates can now run asynchronously using Solid Queue, so you don’t block the app or tie up your connection pool.
• Improved Schema Change APIs: Many migration helpers are now idempotent and reversible by default.
• Better Connection Pooling: Rails 8’s improved connection management means your background jobs won’t starve your web traffic.

If you’ve survived Rails 6/7 migrations, these are game-changers.

---

The Step-by-Step: Real Migration, Real Example

The Scenario

We needed to add a privacy_level column to our prayer_requests table. This was a critical table, with 200+ writes per day. Old-school migrations risked locking the table and causing timeouts. Here’s how we did it in Rails 8—without downtime.

---

Step 1: Add the Column Without Default

class AddPrivacyLevelToPrayerRequests < ActiveRecord::Migration[8.0]
  def change
    add_column :prayer_requests, :privacy_level, :string, null: true
  end
end

Why? Adding a nullable column is instant for Postgres. If you add a null: false with a default, Postgres rewrites the whole table—a recipe for downtime.

Ember’s Wisdom: “First, test the water with your flipper. Don’t dive in before you know it’s safe.” 🐧

---

Step 2: Backfill Data Safely (Solid Queue FTW)

In Rails 8, you can use Solid Queue to process background jobs in the database itself. We created a background job to backfill the new column in batches:

class BackfillPrivacyLevelJob < ApplicationJob
  queue_as :default

  def perform(start_id, end_id)
    PrayerRequest.where(id: start_id..end_id).update_all(privacy_level: "public")
  end
end

Then enqueue jobs for each batch (say, 1000 records at a time):

PrayerRequest.pluck(:id).each_slice(1000) do |ids|
  BackfillPrivacyLevelJob.perform_later(ids.first, ids.last)
end

Alternative: You can also use find_in_batches or a dedicated migration gem (e.g., Strong Migrations).

---

Step 3: Enforce the NOT NULL Constraint

Once all records are backfilled and you’ve confirmed via the Rails console:

PrayerRequest.where(privacy_level: nil).count # Should be 0

…add the NOT NULL constraint:

class MakePrivacyLevelNotNull < ActiveRecord::Migration[8.0]
  def change
    change_column_null :prayer_requests, :privacy_level, false
  end
end

Pro tip: Do this in a separate deployment—never combine with data migration!

---

Step 4: Add Application-Level Validation (Before Enforcing at DB)

Update your model to validate presence:

class PrayerRequest < ApplicationRecord
  validates :privacy_level, presence: true
end

Deploy this before enforcing the database constraint, so new writes don’t slip through.

---

Step 5: Add Indexes (CONCURRENTLY)

For large tables, always use algorithm: :concurrently:

add_index :prayer_requests, :privacy_level, algorithm: :concurrently

Note: You must run this index in a separate migration, and disable DDL transactions:

class AddPrivacyLevelIndex < ActiveRecord::Migration[8.0]
  disable_ddl_transaction!

  def change
    add_index :prayer_requests, :privacy_level, algorithm: :concurrently
  end
end

---

Gotchas and Lessons Learned

1. Never Combine Schema and Data Changes

I learned this the hard way in 2022, when an “innocent” migration combining a new column and a backfill locked the table for 8 minutes. Separate your schema changes from your data changes, always.

2. Monitor Job Progress

Background migrations can fail quietly. Use job dashboards (Solid Queue, Sidekiq, etc.) and check counts in the Rails console. Don’t drop NOT NULL until every record is backfilled.

3. Feature Flags for Application Logic

If changing logic based on the new column, use feature flags to gradually enable new behavior—especially if you’re not sure every record is updated.

4. Rollback Plans

Know how to revert every step. For our migration:

• To roll back, drop the column (if no data needed), or remove NOT NULL if you need to temporarily allow nulls.
• Always keep backups. Practice restoring—don’t just assume your backup works.

---

Ember’s Note: “Zero-downtime isn’t just technical—it’s a promise to your users. Plan, test, monitor, and your migrations will slide across the ice like a pro.” 🐧🔥

---

Checklist: Your Next Zero-Downtime Migration

Pre-Migration

• [ ] Back up production database
• [ ] Test migrations on a staging clone
• [ ] Communicate downtime risk (even if you expect none)
• [ ] Break migration into multiple steps

During Migration

• [ ] Add nullable columns first
• [ ] Deploy, then backfill via background jobs
• [ ] Monitor jobs for completion
• [ ] Add NOT NULL constraint in a separate deployment
• [ ] Add indexes concurrently, in separate migrations

After Migration

• [ ] Validate all records in Rails console
• [ ] Monitor logs for errors
• [ ] Celebrate a deployment with zero downtime!

---

Conclusion

Zero-downtime migrations in Rails 8 aren’t magic—they’re the result of careful planning, staged deployments, and new tools that finally make it realistic for small teams. If you’re still running migrations at midnight and hoping for the best, it’s time to embrace these patterns.

The next time you need to migrate a critical table, remember: with the right approach, you can migrate in the middle of the day, drink your coffee while you deploy, and keep your users blissfully unaware that anything changed.

Ember’s Final Wisdom: “Every migration is a journey. The best ones leave no footprints behind.” 🐧🔥

---

Questions? War stories? Drop them in the comments below or reach out on [LinkedIn/Twitter]. Want the code samples from this post? [Download here].

---

---