Data Engineer

Job Role:

We are looking for a skilled and delivery-oriented Data Engineering Lead with strong technical expertise in data engineering, hands-on experience with Microsoft Azure, and the ability to manage a small team of data engineers. The candidate will be responsible for translating client requirements into actionable technical tasks, overseeing data pipeline development, ensuring high-quality deliverables, and maintaining alignment with business objectives and delivery timelines. The candidate should also have experience in digital marketing measurement, A/B testing frameworks, and leveraging AI tools to improve productivity and operational efficiency.

Responsibilities:

1. Team Leadership and Delivery Management

• Lead a team of 3–4 data engineers, providing technical guidance and delivery oversight.
• Track progress, manage risks and dependencies, and ensure timely, high-quality delivery.
• Drive team collaboration, accountability, and continuous improvement.

2. Client Engagement

• Engage with clients and stakeholders to understand business requirements and objectives.
• Translate requirements into actionable technical tasks and delivery plans.
• Ensure alignment across clients and engineering teams on scope, priorities, progress, and timelines.
• Proactively communicate delivery status, risks, and dependencies.

3. Data Engineering and Development

• Design, develop, and optimize scalable data pipelines using SQL, Python, and PySpark.
• Implement robust data integration, transformation, and validation processes.
• Troubleshoot pipeline issues and ensure data quality, performance, and reliability.
• Adhere to engineering best practices for coding, testing, and documentation.

4. Azure and Microsoft Fabric

• Develop and manage data engineering solutions using Azure services, including ADLA, ADLS, Azure Synapse Analytics, and Microsoft Fabric.
• Build, monitor, and optimize data pipelines to ensure reliable and timely execution.
• Manage Power BI dashboards and reporting assets within Microsoft Fabric workspaces.
• Collaborate with analytics teams to support business reporting and data accessibility.

5. Digital Marketing Measurement and Experimentation

• Support marketing measurement for digital products involving campaign analytics, and performance reporting.
• Apply knowledge of A/B testing frameworks, experiment design, control groups, and KPI evaluation.
• Ensure data quality and availability for marketing measurement, experimentation, and business insights.
• Translate measurement requirements into scalable data solutions.

6. AI-Enabled Productivity and Innovation

• Leverage AI tools to enhance development efficiency, troubleshooting, and documentation.
• Identify automation opportunities to streamline engineering workflows.
• Ensure responsible AI usage, including data privacy, security, and output validation.

Required Skills:

SQL, Python, PySpark, MS Azure, Azure Data Services, Microsoft Fabric, Power BI.

Base Pay Range:

USD $157,000 – USD $175,500 annually. This range reflects base salary only, excludes incentive and benefits, and is a good-faith estimate for this level and location that may vary based on experience, skills, and where the role is performed.

Communication:

This role requires fluent spoken and written English to communicate directly with client stakeholders, a job-related requirement of the client-facing work itself, not of any candidate's background or origin.