DT7: dbt: transformations as tested code
Documentation and lineage
🎯You'll be able to
Document models and columns next to the code, generate a documentation site with lineage, and write descriptions people can act on.
🏠The everyday picture
Medicine packs carry their leaflet inside the box, written when the medicine was made. A separate leaflet kept in a cupboard would soon describe an older version.
📘What it really is
- Descriptions live in
.ymlfiles beside each model and column:
yaml
- name: net_amount_paise
description: The amount if the line was delivered, otherwise 0. Sum this for net revenue.
-
A good description says what a row or value means, what it includes and leaves out, and its unit. "Revenue" isn't enough; "delivered lines only, in paise" is.
-
dbt docs generatebuilds a site (target/index.html, withcatalog.json) with every model, column, test and the lineage graph: which sources and models feed each table.dbt docs serveopens it locally. -
Lineage answers "if this source changes, what breaks?" and "where does this dashboard number come from?"
- Exposures declare the dashboards and reports that use a model, so lineage reaches the people who read the numbers.
- Keep the definitions the dashboard shows (DT9) the same as the column descriptions, so readers and builders mean the same thing.
📝A worked example
Input: in the solution, run dbt docs generate --profiles-dir ..
Expected output: Catalog written to .../warehouse/target/catalog.json, and target/index.html exists. Opening the site with
dbt docs serve --profiles-dir . shows mart_daily_sales with its description, columns and a lineage graph back to
clean.order_lines.
🌍Where it's used
- Data catalogues: tools such as DataHub, OpenMetadata and Atlan read dbt's documentation and lineage.
- Onboarding: new analysts learn the warehouse from the docs site.
- Audits: lineage shows where a regulated figure comes from.
📝Case scenario
A bank's regulator asked how a reported liquidity figure was calculated. The team spent two weeks tracing scheduled queries by hand. A year later, with the figure built in dbt, the answer was a lineage graph and the descriptions of eight models, exported in an hour.
🖱️Try it
Pick three columns in the starter's marts that have no description and write one for each, saying what it includes, what it leaves out and its unit. Success: someone outside the team could use each column correctly from your sentence.
⚠️Common mistakes
- Descriptions that repeat the name. Sign: "revenue: the revenue". Fix: say what's included, excluded and the unit.
- Docs kept elsewhere. Sign: a wiki page describing last year's model. Fix: descriptions in the
.ymlnext to the SQL. - No lineage to dashboards. Sign: a model is changed and a report breaks unexpectedly. Fix: exposures, or at least a list.
🛠️For your project
Project 2 test 10 checks every mart model and every column in the marts has a description.
✅Quick check
What should a column description say?
Show the answer
What it means, what it includes and leaves out, and its unit.
What question does lineage answer?
Show the answer
Where a table's data comes from, and what depends on it.