Senior Data Engineer

We are seeking an experienced Senior Data Engineer to build and optimize scalable data platforms using Microsoft Fabric, Databricks (and/or Snowflake). The role focuses on designing reliable data pipelines, lakehouse and warehouse models, and semantic layers that enable enterprise analytics, BI, and AI/Gen AI use cases. You will work closely with analytics, BI, and data science teams to deliver high-quality, performant, and governed data solutions, while driving best practices in data engineering, optimization, and platform design.

Key Responsibilities

• Design, build, and maintain end-to-end data solutions on Microsoft Fabric, Databricks, including Pipelines, Notebooks, Lakehouse, Data Warehouse, and Semantic Models.
• Implement scalable data ingestion, transformation, and loading (ETL/ELT) using Fabric Pipelines and PySpark.
• Develop robust data models and schemas optimized for analytics, reporting, and AI-driven consumption.
• Create and maintain semantic models to support Power BI and enterprise BI solutions.
• Engineer high-performance data solutions that meet requirements for throughput, scalability, quality, and security.
• Author efficient PySpark and SQL code for large-scale data transformation, data quality management, and business rule processing.
• Build reusable framework components for metadata-driven pipelines and automation.
• Optimize Lakehouse and Data Warehouse performance, including partitioning, indexing, Delta optimization, and compute tuning.
• Develop and maintain stored procedures and advanced SQL logic for operational workloads.
• Design and prepare feature-ready datasets for AI and GenAI applications.
• Collaborate with data scientists and ML engineers to productionize AI pipelines.
• Implement data governance and metadata practices required for responsible AI.
• Leverage Fabric, Databricks capabilities to orchestrate and monitor AI-related data workflows.
• Apply data governance, privacy, and security standards across all engineered assets.
• Implement monitoring, alerting, and observability best practices for pipelines and compute workloads.
• Drive data quality initiatives, including validation frameworks, profiling, and anomaly detection.
• Partner with analytics, BI, data science, and product teams to understand requirements and translate them into technical solutions.
• Mentor junior engineers and contribute to engineering standards and patterns.
• Participate in architecture reviews, technical design sessions, and roadmap planning.
• Develop and deliver dashboards and reports using Power BI and Tableau.