Dbt Core Git
You set up a dbt core git workflow by storing your SQL models in a code repo. Then, you use branches and pull requests to test changes before deployment. Most analytics teams start by linking their local project folder to a remote repo. Each developer clones the project and writes transformations on a feature branch. Still, teams often give local profiles and automated runners broad database permissions. If a bad model merges, the runner can overwrite production tables or expose sensitive record
Git Commit Signing (GPG, SSH): the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You set up a dbt core git workflow by storing your SQL models in a code repo. Then, you use branches and pull requests to test changes before deployment. Most analytics teams start by linking their local project folder to a remote repo. Each developer clones the project and writes transformations on a feature branch. Still, teams often give local profiles and automated runners broad database permissions. If a bad model merges, the runner can overwrite production tables or expose sensitive records.
How a dbt core git architecture works
A standard project separates transformation code from database credentials. Your repo holds model files, schema tests, macros, and project settings. The local profile file stays outside version control on the machine of the user. This prevents engineers from pushing plain text passwords or secret keys to the remote server.
When an engineer builds a new data model, they create a separate git branch. They run the command tool on their laptop against a development schema. Once the model passes local checks, the engineer opens a pull request. A continuous integration server detects the pull request and starts an automated test job. This runner pulls the branch code and builds the models in a temporary staging schema. After a teammate approves the code, the branch merges into the main branch. Finally, a scheduled deployment job runs the updated models against the production data warehouse.
Manage branches and environment isolation
Branching strategies keep broken SQL from reaching dashboards and business reports. Most teams protect the main branch so nobody can push code to it directly. Every change must pass automated schema tests and human review first. Teams also split their database into separate targets for development, staging, and production.
Each target uses its own database role and schema name. Developers get write permissions only on their personal scratch schemas. Automated runners get write permissions on the production schema. This split keeps local test runs from changing live company numbers. Still, developers often need to read production tables to build accurate models. Giving developer roles read permissions on production tables creates a major security risk.
Git Commit Signing (GPG, SSH): architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
What native git workflows solve and what they miss
Version control gives your data team a clear history of every file change. You can see who wrote a model, when they changed it, and why. If a new deployment breaks a report, you can revert the commit in seconds. Code reviews also help teams catch bad logic before it hits production.
Still, git only tracks static files before they run. Once a user or runner logs into the database, git has no control over the session. Native database roles decide who can log in, not what individual queries do in flight. A developer querying production data can still pull raw emails, phone numbers, or card numbers. Also, shared service accounts in deployment pipelines hide which person triggered a costly query. And static database keys stored in pipeline secrets can leak if a build server gets compromised.
Add runtime guardrails and masking beside your databases and APIs
Login rules, role grants, and CI checks govern who connects to your stack. Yet any allowed script, engineer, or AI agent still runs commands with the full scope of its role. For workloads on Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes, SSH, or internal HTTP services, hoop.dev sits beside the service as an open-source proxy, a sidecar for your database or API.
Before a query runs, hoop.dev guardrails check the statement against your rules. An AI Analyzer you configure can score intent or hold risky calls for human review. Next, hoop.dev data masking rewrites sensitive fields in responses, and each verdict is logged per listener. Explore the MIT-licensed project on GitHub.
Frequently asked questions
Should I commit my profiles file to the repository?
No, you should never commit your local profiles file to version control. That file often holds database passwords, private keys, and account details. Keep it on your local machine or inject values during automated runs.
How do we test data models in pull requests?
Set up your build server to run modified models in a temporary staging schema. The runner builds only changed models and their downstream tables, runs tests, and drops the schema afterward.
Why do data pipelines risk exposing sensitive fields?
Developers and runners often use database roles with full read permissions on raw source tables. Without field masking in flight, queries return raw emails, phone numbers, and national IDs in query results.