Add Temporary Docker Service

Add this service to your docker-compose.yml (same pattern as the existing dbt-transform, just pointing to ./dbt-2):

# TEMPORARY — for learning exercise only
dbt-transform-2:
  build:
    context: ./dbt-2
    dockerfile: Dockerfile
  container_name: rb-northwind-dbt-2
  depends_on:
    clickhouse:
      condition: service_healthy
  profiles:
    - transform

Run & Test

Build and run all models
docker compose run --build dbt-transform-2 run

dbt resolves the dependency graph and executes models in order: staging views first, then dimensions, then facts, then analytical views. Look for green “OK” markers.

Run tests
docker compose run dbt-transform-2 test

Validates not-null, uniqueness, and referential integrity between fact and dimension tables.

Verify output
# Query the analytical view directly
curl "http://localhost:8123/?query=SELECT+*+FROM+vw_customer_revenue+ORDER+BY+total_freight+DESC+LIMIT+5" \
  --user default:clickhouse

Expected Output

company_name country total_orders total_freight avg_freight
Save-a-lot Markets USA 31 2,386.74 76.99
Ernst Handel Austria 30 2,238.98 74.63
QUICK-Stop Germany 28 1,984.62 70.88
Hungry Owl All-Night Grocers Ireland 19 1,512.45 79.60
Folk och fä HB Sweden 19 1,245.33 65.54

* Actual values depend on the Northwind dataset version loaded via CDC.

🎉
Your full ELT pipeline is working!
CDC mirrors data in real-time → dbt transforms it into analytics-ready tables.

Materialization Strategies

Quick reference for when you build your own models:

table
Drops and recreates the table on every run. Simple and predictable.
Used in this demo — ideal for small to medium datasets.
view
Creates a database view. Always reflects latest data, zero storage cost.
Used for staging models — cheap wrappers over raw tables.
incremental
Only processes new or changed rows since the last run. Dramatically faster for large datasets.
Production-ready — the next step after prototyping with table.

Production: Build vs. Run

Understanding the two commands:

Command What it does When to use
docker compose run --build dbt-transform run Rebuilds the Docker image (copies SQL models into it), then runs dbt First time, or after changing SQL models / Dockerfile (a deployment step)
docker compose run dbt-transform run Runs dbt using the existing Docker image Every recurring run — this is the workhorse command

Cron (simplest)

# Run dbt every hour at :30
30 * * * * cd /path/to/db && docker compose run dbt-transform run >> /var/log/dbt.log 2>&1

No --build in cron — the image is already built from a previous deployment step. Adjust frequency to match your data freshness needs (every 15 minutes, hourly, daily).

Orchestrators (advanced)

Tool What it adds
Apache Airflow DAG-based scheduling, retries, web UI, extensive integrations
Dagster Software-defined assets, native dbt integration, type checking
Prefect Python-native workflows, automatic retries, cloud dashboard

Start with cron. Graduate to an orchestrator when you need retries, alerting, or multi-step dependency management.

Clean Up

Dispose of the learning exercise

Now that you understand the full pipeline, remove the disposable project:

# 1. Delete the disposable project
rm -rf db/dbt-2

# 2. Remove the dbt-transform-2 service block
#    from docker-compose.yml (the one you added above)

# 3. Optionally drop the tables from ClickHouse
curl "http://localhost:8123/" --user default:clickhouse \
  -d "DROP TABLE IF EXISTS dim_customer;
      DROP TABLE IF EXISTS fact_order_summary;
      DROP VIEW IF EXISTS vw_customer_revenue;"

The original db/dbt reference project remains untouched and ready for future use.