Data Engineer
Remote · Full-time
- Posted 2w ago
- From Trantor’s careers page
- Work mode
- Remote
- Type
- Full-time
- Experience
- 7+ years
- Department
- Data and Analytics
Opens the listing on trantorinc.com
Let the right jobs find you
In your inbox every Wednesday and SaturdayPersonalised suggestions from verified career pages, matched to your role, location, level and skills.
About the role
Summary of the Role
We are looking for a highly skilled and motivated Data Engineer with strong expertise in AWS data services to join our data platform team. The ideal candidate will have hands-on experience designing scalable data pipelines, workflow orchestration frameworks, and large-scale data migration solutions.
This role will be responsible for building robust cloud-native data engineering solutions on AWS, migrating datasets from legacy systems and data warehouses, and ensuring secure and efficient data processing pipelines across distributed environments.
Roles and Responsibilities
1. AWS Data Pipeline Development
- Design and implement scalable ETL/ELT data pipelines using AWS Glue, AWS Lambda, and AWS S3.
- Build and maintain high-performance data ingestion frameworks for processing large-scale datasets.
- Implement data pipelines for data warehousing and analytics platforms such as AWS Redshift.
- Optimize storage and querying strategies using AWS S3 data lakes.
2. Data Workflow Orchestration
- Develop and maintain data workflow orchestration frameworks using tools such as Apache Airflow or AWS Step Functions.
- Automate complex workflows including data ingestion, transformation, validation, and loading processes.
- Build reusable and configurable workflows to support multiple data processing use cases.
3. Data Migration and Integration
- Lead data migrations from legacy data warehouse technologies to modern AWS data platforms.
- Perform data migration from RDBMS systems (e.g., MySQL, SQL Server, Oracle) to AWS S3 or AWS Redshift.
- Design scalable migration frameworks for large datasets with minimal downtime.
- Integrate data sources from enterprise applications and external systems.
4. Data Security and Governance
- Implement secure data pipelines using AWS security best practices.
- Manage access control and data governance using AWS IAM and Lake Formation.
- Ensure data encryption, access management, and compliance across all data platforms.
5. Performance Optimization and Monitoring
- Monitor data pipelines and troubleshoot performance issues.
- Optimize ETL workflows for scalability, reliability, and cost efficiency.
- Implement logging, monitoring, and alerting mechanisms for data pipelines.
Must-Have Skills
- 7+ years of experience in Data Engineering or Data Platform development.
- Strong hands-on experience with AWS, AWS Glue, AWS S3, and AWS Lambda.
- Experience with Data Workflow Orchestration tools such as Apache Airflow or AWS Step Functions.
- Experience performing data migrations from other data warehouse technologies.
- Experience performing data migrations from RDBMS systems to AWS S3 or AWS Redshift.
- Strong expertise in Python and SQL for building scalable data pipelines.
- Solid understanding of ETL/ELT concepts, data partitioning, and distributed data processing.
- Experience working with version control systems such as GitLab or Bitbucket.
- Strong debugging, analytical thinking, and problem-solving skills.
- Basic understanding of Object-Oriented Programming concepts.
Industry Knowledge and Experience
- Experience building cloud-native data engineering solutions on AWS.
- Experience with data warehouse architectures and large-scale analytics platforms.
- Hands-on experience with data extraction, transformation, and migration frameworks.
- Experience working in high-volume data environments such as FinTech, analytics platforms, or enterprise data systems.
Good-to-Have Skills
- IBM Cognos
- AWS Athena
- AWS Lake Formation
- AWS Redshift
- AWS Glue Data Catalog
- AWS SageMaker
- AWS IAM
Skills they ask for
Pick one to see other roles that ask for it.
About Trantor
Enterprise technology servicesTrantor modernizes enterprise systems with AI and runs them through Centers of Excellence.
See all 3 roles at TrantorMore roles at Trantor
See all 3- Customer Success EngineerNoida · HybridCustomer Service · HybridNoida, India2w
- Data EngineerRemoteData and Analytics · Remote2w
Let the right jobs find you
In your inbox every Wednesday and SaturdayPersonalised suggestions from verified career pages, matched to your role, location, level and skills.