Technical note

CI/CD for dbt and SQL Projects

  • Analytics Engineering
  • CI/CD
  • dbt
  • Data Quality

CI/CD can sound like infrastructure for software teams.

For analytics engineering, it has a more direct business purpose:

Stop unsafe reporting logic from reaching decision workflows.

Continuous integration checks a change before it is merged. Continuous delivery or deployment controls how that change reaches production models and dashboards.

The value is not the acronym.

The value is moving quality checks earlier, before the leadership meeting finds the problem.

The symptom: production reporting is the first real test

At Brightside Bikes, imagine an analyst updates the model behind weekly trading revenue.

The change handles refunds more correctly, but it also changes the grain of the model. A dashboard still loads. The SQL runs in development. The pull request looks small.

After merge, Monday’s report shows a revenue shift that nobody expected.

Finance asks whether the number moved because of real trading, refund timing, or a code change. Growth pauses spend decisions. The Analytics Engineer starts comparing old and new outputs under pressure.

This is what happens when production is the first serious test.

CI/CD gives the team a safer path.

What CI should check

For dbt, Dataform, and SQL projects, CI should check the things that make reporting unsafe.

At minimum:

  • Project compiles.
  • Changed SQL is syntactically valid.
  • Dependencies resolve.
  • Tests pass for changed models and important downstream models.
  • Source definitions are valid.
  • Documentation metadata is still well formed.
  • Naming and lint rules are respected where useful.

For high-value reporting paths, CI should go further:

  • Compare row counts before and after a change.
  • Compare key metric totals.
  • Run uniqueness and relationship checks.
  • Run reconciliation checks.
  • Identify downstream dashboards or marts affected by the change.
  • Require review when metric definitions change.

The best CI checks are connected to business risk.

For Brightside, a change to fct_weekly_trading_revenue should not be treated the same as a change to an unused exploratory model. The revenue model feeds the Weekly Operating Performance Report. It deserves stronger controls.

A simple CI flow

A practical CI flow might look like this:

  1. Analyst opens a pull request.
  2. CI installs dependencies.
  3. Project compiles.
  4. SQL style or lint checks run.
  5. Changed models are built in a temporary schema.
  6. Tests run on changed and downstream models.
  7. Reconciliation queries compare important totals.
  8. Reviewers see pass/fail results before merge.

For dbt, that might include commands such as:

dbt deps
dbt compile
dbt build --select state:modified+

The exact command depends on the project and warehouse.

The principle is stable: check the changed path before it becomes the official reporting path.

CD should control release, not just automate it

Continuous deployment can be useful, but analytics teams should be careful with sensitive reporting logic.

Some changes can deploy automatically after tests pass. Others need a release window, business sign-off, or communication.

For Brightside:

  • Fixing a typo in documentation can deploy automatically.
  • Adding a new staging field can deploy after tests pass.
  • Changing Shopify Net Revenue logic may need Finance Lead approval.
  • Changing contribution margin logic may need COO and Finance review.
  • Rebuilding the board KPI pack should have a release note.

CD should reflect the reporting risk.

The goal is not to slow every change down. The goal is to keep high-impact changes from surprising the business.

Use environments deliberately

Analytics projects often need at least two environments:

  • Development or pull request environment
  • Production reporting environment

Some teams also use staging.

The important point is separation.

A pull request should not overwrite production models used by executives. Tests should run in an isolated schema or environment. Reviewers should be able to inspect outputs before they become official.

At Brightside, a PR model might build into a temporary schema so the team can compare:

  • Shopify Net Revenue by week
  • Finance bridge variance
  • Contribution margin by product category
  • Active Bike Owner count
  • Weeks of cover by SKU family

If the numbers change, the pull request should explain why.

What to include in a release note

Not every analytics change needs a formal release note.

But important reporting changes should leave a short business-readable trail.

For example:

Changed weekly trading revenue to exclude fully voided Shopify orders.
Impact: prior eight weeks reduced by GBP 1,840 total.
Reason: voided orders were previously excluded from order count but not revenue.
Approved by: Finance Lead.
Dashboard impact: Weekly Operating Performance Report, revenue bridge.

This is not bureaucracy.

It is future debugging support.

When a number changes, the team can see whether the movement was a real business event or an intentional logic release.

What to inspect first

For an existing analytics project, ask:

  • Do pull requests run automated checks?
  • Do checks build only changed paths or the full project?
  • Are critical downstream models included?
  • Are reconciliation checks part of CI?
  • Are production models protected from development runs?
  • Are failures understandable?
  • Are high-risk metric changes reviewed by business owners?
  • Does the team record release notes for important reporting changes?

If tests run but nobody trusts what they mean, CI is still immature.

If deployment is automatic but business owners are surprised by metric changes, CD is too blunt.

What good looks like

Good CI/CD for analytics is not about copying software practices mechanically.

It is about protecting the reporting contract.

For trusted reporting, good looks like this:

  • Every pull request compiles and tests before merge.
  • Changed models build in an isolated environment.
  • Important downstream models are checked.
  • Reconciliation queries protect core metrics.
  • High-risk changes require the right reviewers.
  • Production deployment is controlled.
  • Release notes explain business-impacting changes.
  • Rollback is possible.

CI/CD reduces the Invisible Data Tax because fewer errors reach the meeting.

It does not replace judgement.

It gives judgement a safer place to happen.

Technical note

Use this as supporting context

This article is part of a low-profile technical series behind the Reporting Trust framework. For the executive-level problem, start with the business-facing articles on mismatched numbers and source-to-report lineage.

Start with the business problem