Lead Data Engineer
United States · Full-time
- Posted 3d ago
- From Elastic’s careers page
- Location
- United States
- Type
- Full-time
- Level
- Senior
- Experience
- 8+ years
- Department
- Data and Analytics
Opens the listing on jobs.elastic.co
Let the right jobs find you
In your inbox every Wednesday and SaturdayPersonalised suggestions from verified career pages, matched to your role, location, level and skills.
About the role
What is The Role
We're looking for a Lead Data Engineer to join our Data Engineering & Architecture team. In this role, you'll help grow a Data Platform that makes it easier for teams across our company to work with data confidently and effectively. Your work will directly support our business goals not just by keeping data flowing, but by turning it into something people can actually use and trust.
This is a great opportunity for someone who enjoys both the technical craft of data engineering and the human side of it: collaborating with partners, sharing knowledge, and helping others get more out of the tools and systems you build together.
What You Will Be Doing
You'll design, maintain, and improve the pipelines and infrastructure that power our data platform. Day to day, that means working across the full data lifecycle — from ingestion and streaming to transformation and delivery — using tools like Apache Airflow, Spark, Iceberg, Kafka, Flink, and dbt Labs.
Beyond the technical work, you'll spend meaningful time with stakeholders to understand what they need and help define how data should flow across systems. That includes co-creating data contracts, writing clear documentation, and guiding teammates on how to get the most out of the platform.
A few other things you'll take on:
- Monitor and refine pipeline performance and refine processes to keep things running smoothly.
- Use Terraform to provision and manage resources on Google Cloud Platform (GCP).
- Help steward the platform by reviewing code and ensuring good practices and standard processes are followed.
- Process access requests and support compliance with platform policies.
What You Bring
- 8+ years of experience in data engineering or related roles
- Experience with Apache Airflow for orchestrating data workflows
- Experience with Spark, Flink, Kafka, and dbt Labs for data processing and transformation
- Understanding of Apache Iceberg for data table formats and data warehouse and lakehouse strategies
- Skilled in using Terraform for infrastructure-as-code deployments
- Proficient in programming languages, particularly Bash, Python, and SQL
- Knowledge of database systems, including OLAP, OLTP, Document, and Vector databases
- Experience with Google Cloud Platform (GCP) for cloud resource management
- Bonus: Proven experience with data security and access controls for SOX Compliant data
- Bonus: Experience and knowledge of the Elastic stack
Skills they ask for
Pick one to see other roles that ask for it.
About Elastic
Search, observability and securityElastic develops search, observability and security products, including Elasticsearch and tools for analyzing application and security data.
See all 130 roles at ElasticMore roles at Elastic
See all 130- Revenue ManagerUnited StatesFinance and AccountingUnited States7h
- Public Sector Account ExecutiveDelhiSalesDelhi, India1d
- Pub Sec Account Executive IIIDelhi · SeniorSales · SeniorDelhi, India1d
- Marketing CoordinatorBengaluruMarketingBengaluru, India1d
Let the right jobs find you
In your inbox every Wednesday and SaturdayPersonalised suggestions from verified career pages, matched to your role, location, level and skills.