Requirements
The role requires 5+ years of experience in data engineering, Databricks, or related fields, with ideally 2+ years in a Lead or Architect position. Strong hands-on Databricks expertise is essential, including excellent Python and PySpark skills, and knowledge of Databricks Lakehouse, Delta Lake, Unity Catalog, Workflows, Lakeflow, and Databricks SQL. Experience with Azure Databricks services (Data Factory, ADLS Gen2, Azure DevOps), data quality, governance, or remediation solutions, and strong technical leadership and stakeholder management skills are also required.
Description
Databricks Data Quality & Remediation Architect / LeadLocation: Remote - UAE | Abu Dhabi preferredContract: 6 months initially, with strong likelihood of extensionRate: Competitive, depending on experienceStart: ASAPOur client is a leading organisation in the UAE delivering a major enterprise data transformation programme, and they are looking for a Databricks Data Quality & Remediation Architect / Lead Developer to help design and lead the delivery of a new enterprise-scale data quality platform.This is a senior, hands-on role combining Databricks architecture, data engineering and technical leadership.What You'll DoDesign and implement a scalable Data Quality & Remediation Engine on Databricks.Define architecture across data profiling, quality rules, exception management, root-cause analysis and remediation.Build a metadata-driven data-quality framework.Define Bronze, Silver and Gold processing layers.Work with Data Governance teams and Data Stewards on Critical Data Elements and data-quality processes.Support Microsoft Purview integration, governance, lineage and auditability.Lead the development team and provide hands-on technical direction.Own data-quality reporting, including quality KPIs, exceptions and remediation activity.What We're Looking For5+ years experience in data engineering, Databricks or related roles.Ideally 2+ years in a Lead or Architect position.Strong hands-on Databricks experience.Excellent Python and PySpark skills.Strong knowledge of Databricks Lakehouse, Delta Lake, Unity Catalog, Workflows, Lakeflow and Databricks SQL.Experience with Azure Databricks and services such as Data Factory, ADLS Gen2 and Azure DevOps.Experience with data quality, governance or remediation solutions.Strong technical leadership and stakeholder-management skills.DesirableMicrosoft Purview.SAP S/4HANA, SAP BW or Oracle Fusion.MLflow, Mosaic AI or Databricks Asset Bundles.Enterprise data governance and lineage.Why ApplyYou'll have the opportunity to shape the architecture of a major enterprise data platform, lead delivery and work on complex data-quality challenges at scale.The programme is expected to continue beyond the initial six-month contract, offering strong potential for long-term engagement.If you're an experienced Databricks Lead or Architect looking for a technically challenging project where you can make a real impact, we'd love to hear from you. Apply now or get in touch for a confidential discussion.