Data Engineer

Job Description:

We are seeking an experienced Data Engineer with hands-on experience in ETL development and AWS-based data engineering solutions. The ideal candidate will have strong expertise in PySpark, Python (NumPy, Pandas), and large-scale distributed data processing, along with proven experience in deploying and managing AWS data services.

You will work closely with IT and business stakeholders to design, build, and optimize scalable data pipelines while ensuring cost efficiency, data quality, and high performance across the data ecosystem.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Py Spark and Python.
• Build and deploy AWS-based data solutions leveraging services such as Glue, S3, Lambda, SNS, and Step Functions.
• Perform large-scale data transformations and processing using distributed data frameworks.
• Implement and manage Delta Lake/Delta Tables within a Data Lake architecture.
• Optimize data pipelines for performance, scalability, and cost efficiency in AWS environments.
• Ensure data quality, reliability, and governance across integrated data systems.
• Collaborate with cross-functional IT and business stakeholders to resolve delivery issues and align on requirements.
• Orchestrate workflows using MWAA (Airflow) and manage end-to-end data pipeline automation is an added advantage

Required Skills

• 4 to 8 years of experience in Data Engineering
• Strong expertise in AWS Services (S3, Lambda, SNS, Step Functions)
• Hands-on experience with PySpark and Python
• Strong proficiency in Python libraries such as NumPy and Pandas
• Experience with large-scale data transformations
• Strong communication and stakeholder management skills

Good to Have

• Metadata Management, Data Lineage & Data Governance & Data Lake Architecture
• Delta Tables / Delta Lake & MWAA (Airflow Orchestration)
• Experience in cost optimization strategies in AWS

Why Join LatentView Analytics?

• Work with world-class clients on cutting-edge data engineering and analytics projects.
• Be part of a high-performing, innovation-driven culture that encourages learning and experimentation.
• Opportunity for professional growth through mentorship, training, and global exposure.
• Collaborative and inclusive workplace that values diversity and individuality.

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status.