Code With Sauru💻 [ALL]

 Report: Profitability_Actuals_Tableau >> Field Mapping Validation 4 Field Fails >> Join validation No fails >> Filter Validation no fails Derived Field validation no fails >> Data Quality Checks 2 Fails >> Action items

 Report name:  Purchase info Record >> Field Mapping: Planned Delivery Time (in Days) :  Fix: (Hardcoded precision may not match SAP Verify SAP EINE-APLFZ field type) Net Price: Fiz: Hardcoded precision/scale Verify SAP EINE-NETPR field type Price Unit:  Fix: Hardcoded precision may not match SAP Verify SAP EINE-PEINH field type goods_invoice_receipt: Fix:  Last Price Update Fix: Last Close PO Delivery Date Fix: Last Close PO Price Fix:  >> Join Validation  only 2 with WAERM statement >> Filter Validation a) Date Range Filter b) Validity Date Range >> Derived Field Validation a)Last Price Update b)Last Close PO Delivery Date c)Last Close PO Price >> Data Quality Checks a) NULL Value Validation on Key Fields b) Referential Integrity (Join Success Rate) c) Duplicate Record Detection d) Data Completeness Check >> Action Items

 Report name:  Price list with release status >> Field Mapping validation Extra Field: 1) load_date 2) RELEASE_STATUS fiels under A006 PL_SALES_ORG_ID  PL_DISTRIBTN_CHANNEL_ID PL_PRICE_LIST_ID PL_CnTy PL_MATERIAL PL_VALID_START_DATE PL_VALID_END_DATE >> Join validation join name: A006 Pricing Master Fix: Implement explicit JOIN to A006 table on appropriate keys join name: KONP Pricing Conditions Fix: No Fix Required join name: TCURR Currency Conversion Validation Review of Your Comment: NOT JOINED; FDD documents detailed conversion logic In your code: No TCURR table is used No exchange rate logic exists Amount is directly taken from salesorders Why this Fails code: CAST (so.Rate AS decimal ( 17 , 2 )) AS `PL_List Price Amount` so.rate_unit AS `PL_Unit_CURRENCY` This means: Amount is in original transaction currency No conversion to reporting currency If your dataset contains: Currency Amount USD 100 EUR 100 INR 100 Your report wil...

Report: Material Valuation (Validation and Fixes.....)\ 2]Pricing POC_Prod - Material Valuation >> Field Mapping 16 th cell (Field): Material Decription Both Fields mentioned : TXTMD/MAKTX Suggestion Fix=> Verify product_name source matches MAKT.MAKTX Final Status:   In Excel Two fields are mentioned TXTMD/MAKTX but code uses MAKT.MAKTX >> Join Validation A] Product to Sales Org "one product_id have multiple sales org" New_updated Code for Product to Sales Org:  SELECT DISTINCT      product_id,     sales_org_code,   -- ADD THIS     Item_category_group_from_material_master,     Delivering_Plant_Own_or_External  B] Product_Plant to Inventory  # CT_COMMENT=>     # In SAP Inventory comes from MARD (Storage Location / Inventory table)     # but in our code we are using gold.inventory_stock_levels.inventory_stock_levels     # This is a derived gold table, not ...

 POST 1] Unity Catalog Links and Quick Notes By Nishtha Jain and Pratima Jain To Do Now =>          1: Make All Links Proper Here  [  ]          2: Start Learning and Making Notes  [  ]

1] Post: Microsoft Fabrics    Understand what is Microsoft Fabrics in Simple Way... Written Notes:::>>>>   Link2Pdf Reference :>>> Youtube Video  Link2video ________________________________________________________________________________ 2] Post : Celebal Technology Link  Celebal Tracker:  Tracker :>  Link Sample :>  Link Colab :>  Link Curiculum :>  Link

 SQL SKILL : SQL

Most Important Topics:          1. Azure Databricks : [Rajas Playlist  | Data Master Playlist | Partner Academy Data engineer                          Course]            2. UC Migration:  [   Structured Theory  from Learnings  Cloudpandith 4 Videos + Project 1 video of cloud pandith and then naval yemul  ]           3. Leet Code : Solve Daily 2 Coding Problem [Time 5 to 6:30 ]

 Topics: Databricks Naval Yemul Playlist [  ] Rajas DE [  ] Partners Academy Course for Data Engineer :  Link Apache Spark Course By Darshil Parmar [  ] Azure Cloud DataMaster: Naval yemul [  ] UC Migration (Catalog: Hive to Unity ) 1 End to End Project  [  ] Projects Spotify [  ] Uber [  ] UC Migration [  ] Delta Live Table (DLT) Framework S/w Engineer Test [  ]  Sample DLT with SQL Script Naval Yemul Playlist [ Done ] Python  Course by Darshil python for data engineer [  ] Rishabh Pandey Bootcamp [  ] Pyspark Functions [  ] Practice [  ] SQL Practice [  ] Presentation  August Study Presentation in Celebal [Your Review of First Month] [  ]

[STRUCTURED Apache Spark] (Quick in 10 Min) # Reference=> Apache in 10 Min ~ DatawithDarshil  Index Intro Hadoop Apache Spark

 >>>>> Databricks Cluster UI (~ Ref Youtube Video : ~Mr.Ktalks Tech ) # Cluster Diagram Explanation:  Cluster : Nodes and Executors Set of VM 1 Node = 1 Executor Each Executor :  has 1/more Cores 1 Core =  1 Partition Each Core : takes 1 task and used for parallism Driver : Step 1: Write Code in Driver Step 2:  Everything get's Divides into STAGES and TASKS This is Done with help of DAG DAG: divides all jobs in form of stages and task each TASK: executes on executor each EXECUTOR: it made of essential cores each CORE: define degree of parallelism that will happen when JOB RUNS.... #  Cluster: Set of Virtual Machines to do work Creating Computer Resources for processing BigData Types: 1] All Purpose Compute 2] Job Compute 1] All-Purpose Compute [Everything] Analyze data in NB Create, Terminate and Restart Cost: Expensive 2] JOB Compute [Just to run NB as Job with ADF pipeline and Databricks] Just Support Running a NB as Job No restart Cost : L...

# Terms Used by Data engineer >Tables: Only Structured > Volume : All Types  [Structured + Unstructured] > Catalog Binding :  Restricts User Permission diffferent categories: 1. production 2. development 3. testing > Scheme Provisioning : Dont need to explicitly define link/sync azure active directory when resource is leaving we dont need to explicitly remove id IF resource is leaving company is called scheme provsioning > UC:  Storage Credentials  External locations Celebal Tech Utility (UNITY LAUNCHER)  => UCX Similar to unity launcher (UR Utility) > Sync and deep clone > Migrating Notebook to Hive [Steps]: Two level  --> Three level namespace (UC only supports Three Level Namespace) If we use mount : we need to replace with external location If we use RDD replace with dataframe > Goup vs Service Principle Group: Handles by user Service Principle: Handled by Machine > CI/CD and Branching Strategies in Databricks > Q: Ho...