In the last video I made the case: if it's published, it's prod. If it's not published, it's not scheduled. That's the policy. But a policy you can't enforce is just a suggestion. This video shows how I built the tooling to make it real.
The Publish Lineage Report
The core artifact is a publish lineage report. For every publication in Domo, it maps:
- Which datasets and views are being published
- What upstream dependencies those datasets rely on (webforms, dimension tables, file uploads)
- Which subscriber instance receives the published content
Once you have this list, you know exactly what's production. Everything else is on the chopping block.
Tag-Based Scheduling Enforcement
The report output feeds directly into Domo's enterprise toolkit. The workflow:
- Generate the list of published asset IDs
- Tag each asset with
publishedvia the API - Enforce the policy: if an asset has the
publishedtag, it can be scheduled. If it doesn't, make it manual execution only
This replaces manual naming conventions and periodic "cleansing" events with an automated, enforceable rule. You can script it to run daily. Zero human judgment required.
Page-Level Lineage
Publish lineage is one shape. Another is page-level lineage: given a page, what cards are on it, what datasets feed those cards, and where do those datasets come from upstream (Snowflake, S3, etc.)?
This matters because people pull the same data in multiple times — five, six, twenty-two copies of the same Snowflake table, each feeding a different card. Without lineage tooling, you can't see that duplication. You just see a page and wonder where the data came from.
How I Architected It
The lineage tooling in crew-dcs (the Domo Python SDK) took a couple of months to design. Here's what I learned:
Factory pattern for entity dispatch. Pages, cards, datasets, dataflows — each has a different API for getting lineage. I needed one class and one method that takes an entity type and ID and returns the right object. No if/else chains. The factory pattern handles dispatch.
Separate data access from representation. The lineage object is a list of entities and their relationships. That's the data. The representation is separate — Mermaid diagrams, Neo4j graphs, text reports. You want to be able to swap representations without touching the data layer.
Library first, scripts second. A 150-line script backed by a solid library is more powerful than thousands of lines of inline API calls. The library (crew-dcs) handles authentication, API interaction, and data modeling. The script just orchestrates the workflow.
Using Grill-Me to Design the Architecture
I didn't start by writing code. I used the grill-me skill with Claude to talk through the use case first:
- What's the goal? Generate a lineage diagram for any Domo entity
- What are the nuances? Different entity types have different lineage APIs; "lineage" means different things for pages vs. datasets vs. publications
- What's the desired CLI experience? One command: take this entity, produce this diagram
Claude helped me think through the architecture before I committed to code. That's the value of the grill-me skill — it forces you to articulate what you want before you start building.
Runbooks as Report Shapes
The trace-lineage runbook has three variants, each producing a different report shape for a different team:
- Dataset/card lineage — for data trust teams: "Where does this page's data come from?"
- Publish lineage — for governance teams: "What's scheduled in production? What can I delete?"
- Column lineage — for data trust teams: "Which columns are actually used? Which can I drop from the 100-column dataset?"
Same library, different runbooks, different audiences. That's the pattern: one library for interacting with Domo, multiple runbooks for different workflows.
