Databricks Lead – Delivery & Engineering

Lead hands-on Databricks delivery, ensuring adherence to CoE standards and best practices.

Key Responsibilities:

• Lead Databricks engineering teams
• Build and optimize Spark & Delta Lake pipelines
• Implement Databricks Jobs, Workflows, CI/CD
• Optimize performance and DBU usage
• Contribute to Databricks accelerators and standards
• Design and implement end-to-end data platforms using Databricks Lakehouse
• Lead development of:
  • Data ingestion (batch & streaming)
  • Delta Lake data modeling (Bronze/Silver/Gold)
  • Scalable ETL/ELT pipelines
• Optimize Spark workloads for performance and cost
• Ensure reliability, scalability, and observability of data pipelines

Experience:

• 10–12 years in data engineering
• 3–5 years hands-on Databricks experience

Skills:

• Deep expertise in Databricks, Apache Spark, Delta Lake
• Strong hands-on experience with PySpark / Spark SQL (core, SQL, streaming)
• Experience designing Lakehouse architectures
• Strong understanding of data modeling, data quality, and lineage
• Databricks platform features
• Cloud-native data engineering
• Performance tuning & troubleshooting

Certifications (Preferred):

• Databricks Data Engineer Associate and Professional