I have updated Wheely's production workloads to a new version of dbt==0.21.0
Along with major improvements to performance, stability, and speed we now have:

– A dbt build command for multi-resource runs
– Handling for column schema changes in incremental models
– Defining configs in all the places you’d expect

An average prod job definition looks like: dbt seed + dbt run + dbt snapshot + dbt test
Now with single dbt build command its going to be really simplified and convenient, building resource by resource, from left to right across your DAG.

New on_schema_change parameter enables additional control when incremental model columns change. Possible strategies are:
– ignore (default): new column will not appear in your target table.
– fail: Triggers an error message when the source and target schemas diverge .
– append_new_columns: Append new columns to the existing table.
– sync_all_columns: Adds any new columns to the existing table, and removes any columns that are now missing.

But nothing comes all good, I’ve faced with a couple of bugs these days:
– One that broke my materialization macro with new dispatch logic
– Serializable isolation violation (unrelated to dbt)

I will describe them in next posts.

Meanwhile read more about upcoming dbt v1.0 in December 2021 !

#dbt #release