Masking last 4 digits of invoice number

Information:  There are some customers related information that require masking. d_product_revenue table has invoice_number column which we need to mask the last 4 digits by replacing the number with ‘*’.

 

Requirement: Create a PySpark logic to mask the last 4 digits of invoice of the customer with ‘*’  and load the invoice_number column with masked value.

 

 e.g. change the invoice number from 1234234534 to 123423**

 

Prerequisite:

 

1. Drop the table d_product_revenue_clone if exist.
2. Create replica of d_product_revenue table in d_product_revenue_clone table and perform the requirement in the replica table.

 

Unity Catalog Information: purgo_playground.d_product_revenue

 

Excepted Output: Databricks PySpark code