Migration Guide: dbt-af to dmp-af¶

This guide helps you migrate from the original dbt-af project to dmp-af.

Why Migrate?¶

• 🎯 Active maintenance and development
• 🎯 New features and improvements
• 🎯 Community-driven enhancements
• 🎯 Same Apache 2.0 license

Migration Steps¶

1. Update Package Installation¶

Simply replace dbt-af with dmp-af in your dependencies:

- dbt-af
+ dmp-af

[project]
dependencies = [
-   "dbt-af",
+   "dmp-af",
]

install_requires=[
-   "dbt-af",
+   "dmp-af",
]

Then reinstall:

pip install -U dmp-af

2. Update Import Statements¶

Update the imports in your Airflow DAG files from dbt_af to dmp_af:

- from dbt_af.dags import compile_dbt_af_dags
- from dbt_af.conf import Config, DbtDefaultTargetsConfig, DbtProjectConfig
+ from dmp_af.dags import compile_dmp_af_dags
+ from dmp_af.conf import Config, DbtDefaultTargetsConfig, DbtProjectConfig

Common imports to update:

- from dbt_af.dags import compile_dbt_af_dags
+ from dmp_af.dags import compile_dmp_af_dags

- from dbt_af.conf import Config
+ from dmp_af.conf import Config

- from dbt_af.conf import DbtDefaultTargetsConfig
+ from dmp_af.conf import DbtDefaultTargetsConfig

- from dbt_af.conf import DbtProjectConfig
+ from dmp_af.conf import DbtProjectConfig

3. Update Function Calls¶

The main DAG compilation function has been renamed:

- dags = compile_dbt_af_dags(
+ dags = compile_dmp_af_dags(
      manifest_path='/path/to/manifest.json',
      config=config,
  )

4. Update Class Names¶

Several class names have been updated:

  # Graph builder
- from dbt_af.builder import DbtAfGraph
+ from dmp_af.builder import DmpAfGraph

  # Maintenance config
- from dbt_af.parser.dbt_node_model import DbtAFMaintenanceConfig
+ from dmp_af.parser.dbt_node_model import DmpAfMaintenanceConfig

5. Update Configuration Variables¶

If you have any custom configuration or variable names:

- dbt_af_config = Config(...)
- dbt_af_graph = DbtAfGraph.from_manifest(...)
+ dmp_af_config = Config(...)
+ dmp_af_graph = DmpAfGraph.from_manifest(...)

6. Update CLI Scripts¶

If you're using the manifest tests script:

- dbt-af-manifest-tests --manifest_path=./target/manifest.json
+ dmp-af-manifest-tests --manifest_path=./target/manifest.json

7. Update Documentation References¶

Update any documentation or comments in your codebase:

- # Using dbt-af for distributed dbt runs
- # See https://github.com/Toloka/dbt-af for details
+ # Using dmp-af for distributed dbt runs
+ # See https://github.com/dmp-labs/dmp-af for details

8. Update dbt Project Configuration¶

# These settings work with both dbt-af and dmp-af
models:
  sql_cluster: "dev"
  daily_sql_cluster: "dev"
  py_cluster: "dev"
  bf_cluster: "dev"

Testing Your Migration¶

After migrating, test that your DAGs are built correctly:

# Test your DAG file compiles successfully
python your_dmp_af_dag.py

If the script runs without errors, your DAGs are correctly configured.

Verification Steps¶

1. Check Airflow DAGs:

   • Ensure DAGs appear in Airflow UI
   • Verify DAG structure looks correct
   • Test a sample DAG run

2. Check Task Dependencies:

   • Verify cross-domain dependencies are working
   • Check that tests are attached to correct models
   • Validate maintenance tasks are scheduled

3. Run a Test DAG:

   • Execute a small DAG to verify operators work correctly
   • Check logs for any errors or warnings

Breaking Changes¶

From dbt-af 0.14.6 to dmp-af¶

Package Renaming:

Behavioral Changes:

Configuration Changes:

Version Numbering:

• dmp-af continues version numbering from dbt-af (starting at 0.14.6+)

Compatibility¶

• Python: 3.10, 3.11, 3.12
• Airflow: 2.6.3+, tested up to 2.11.0
• dbt-core: 1.7-1.10

Migration Checklist¶

Use this checklist to ensure you've covered all migration steps:

• Updated package in requirements.txt or pyproject.toml
• Reinstalled with pip install -U dmp-af
• Updated all from dbt_af imports to from dmp_af
• Renamed compile_dbt_af_dags() to compile_dmp_af_dags()
• Updated DbtAfGraph to DmpAfGraph (if used)
• Updated DbtAFMaintenanceConfig to DmpAfMaintenanceConfig (if used)
• Updated CLI script calls (if used)
• Updated documentation and comments
• Tested DAG compilation
• Verified DAGs appear in Airflow UI
• Ran test DAG execution

Support¶

If you encounter issues during migration:

1. Check this guide for common scenarios
2. Review the examples directory
3. Open an issue on GitHub

Staying with dbt-af¶

If you prefer to stay with the original dbt-af:

• The original project is available at github.com/Toloka/dbt-af
• You can continue using dbt-af without any changes
• Both projects share the same Apache 2.0 license

Next Steps¶

After completing the migration:

• Review the Quick Start guide to explore dmp-af features
• Check out Tutorials for practical examples
• Read about Configuration Options
• Join the community on GitHub