Data Engineer
India · Remote · Full-time
- Posted 5d ago
- From LatentView Analytics’s careers page
- Location
- India
- Work mode
- Remote
- Type
- Full-time
- Experience
- 6+ years
- Department
- Data and Analytics
This role is no longer on LatentView Analytics’s careers page.
Let the right jobs find you
In your inbox every Wednesday and SaturdayPersonalised suggestions from verified career pages, matched to your role, location, level and skills.
About the role
Job Role:
We are seeking a seasoned Data Engineer with a robust background in building scalable data pipelines and managing large-scale data architectures. With 6–9 years of experience, you will play a pivotal role in designing and optimizing our data ecosystem, ensuring high performance and reliability. You are an expert in the Spark ecosystem and feel at home working within Databricks to solve complex data challenges.
Responsibilities:
- Pipeline Development: Design, develop, and maintain high-performance ETL/ELT pipelines using PySpark and Databricks.
- Data Modeling: Architect sophisticated data models and schema designs using SQL to support complex analytical requirements.
- Optimization: Perform performance tuning on Spark jobs and SQL queries to ensure efficient processing of massive datasets.
- Automation: Utilize Python for scripting, automation of data workflows, and building custom utility tools.
- Collaboration: Work closely with Data Scientists and Stakeholders to translate business requirements into technical data solutions.
- Mentorship: Provide technical guidance and code reviews for junior team members, ensuring best practices in data engineering.
Added Advantage:
Cybersecurity Domain Expertise: Prior experience working with security datasets (e.g., SIEM logs, network traffic, threat intelligence) or building data platforms for security analytics and anomaly detection. Familiarity with data privacy standards and securing data at rest and in transit is highly preferred.
Required skills:
SQL, PySpark, Databricks, Data Engineering Python, AWS, Airflow
Skills they ask for
Pick one to see other roles that ask for it.
About LatentView Analytics
Data analytics and AI consultingLatentView is a data analytics consulting firm offering industry-specific analytics and AI solutions, with 20+ years of expertise.
See all roles at LatentView AnalyticsLet the right jobs find you
In your inbox every Wednesday and SaturdayPersonalised suggestions from verified career pages, matched to your role, location, level and skills.