If you’re a user of dbt, you’re almost certainly familiar with using the + operator to run/test everything upstream (or downstream) of a model.
But, there’s a common problem with this: what if a downstream model points at something stale?
To illustrate this, let’s consider a simple pipeline:
You make changes to model A, and want to run it and everything downstream of it using dbt run -s model_a+.
A runs, and C runs, but model C points at model B which could have stale data! In this case, we could do dbt run -s model_a+ model_b or simply a dbt run. But, there are noticeable problems with both approaches:
- With option 1,
dbt run -s model_a+ model_b, we need to manually check which models are upstream of our downstream models - With option 2,
dbt run, we could end up running our whole pipeline unnecessarily
So, we want something that says “run my model and everything downstream, and everything upstream of my downstream models”.
Fortunately, dbt has the exact tool for this — the @ operator (link). If we instead did dbt run -s @model_a, it would:
- Select model A, and everything downstream (C)
- Select everything upstream of the downstream models (B)
I must admit, the @ symbol going in front of the selection, rather than dbt run -s model_a@, is a bit confusing!
So hopefully that demystifies the @ operator, and how it solves a common dbt problem.
A complete beginner to dbt, or someone who wants to learn the advanced concepts? My dbt course is now live on Udemy (link), and covers everything from basic to advanced dbt — including building a full project from scratch, 7 hours of video, and 50 downloadable course materials!
