DQ Validation on Queries in C360 Config Table

Requirement:

 

Prepare a PySpark code to perform DQ comparison between sql queries in c360_config table and the target table as per the attached excel sheet. Display the DQ validation result which should have metric_name,validation, result (PASS/FAIL). Ignore DQ validation for the sql queries whose target table is “NA“ in c360 config table.

 

 

 

Unity Catalog: c360_config