Designs and optimizes scalable data ingestion pipelines for product telemetry and enterprise business data. Builds automated pipelines using Fivetran, Python, APIs, S3 lakehouse architectures, Kafka, Spark, and Snowflake. Applies advanced SQL, incremental loading, CDC, micro-batching, and rate-limit handling to support reliable, analytics-ready datasets. Implements CI/CD and DevOps practices, monitors pipeline performance, and partners with analysts and platform teams to rapidly deliver data solutions.
Your role at Dynatrace
We are seeking an experienced Data Engineer to design, build, and optimize scalable data ingestion pipelines for product telemetry and enterprise business applications data. This role operates in a fast‑paced environment, rapidly prototyping data pipelines to enable data exploration, analytics, and downstream consumption.
Responsibilities
- Design and implement modern data ingestion pipelines for business systems data using Fivetran, Python and Bulk APIs, S3-based lakehouse architectures, and streaming ingestion (Kafka, Spark)
- Build end-to-end automated pipelines from source systems to Snowflake, ensuring reliability, scalability, and observability
- Apply advanced SQL in Snowflake for transformation, standardization, and preparation of analytics-ready datasets
- Optimize ingestion performance for large-scale, high-volume datasets using API rate-limit handling, micro-batching, and incremental loading / CDC
- Implement DevOps and CI/CD practices for data pipelines, including version control and deployment automation
- Partner with analysts, SMEs, and platform teams to rapidly prototype pipelines supporting exploration and evolving requirements
- Monitor, troubleshoot, and optimize ingestion jobs to ensure data freshness, accuracy, and performance
- 6+ years of hands-on experience in data engineering and ingestion automation in cloud or hybrid environments
- Proven experience ingesting business application data (CRM, ERP, financial, consumption, renewals systems)
- Advanced proficiency with Python and Fivetran (or similar ETL/ELT tool) for pipelines, automation, orchestration, and validation
- Expertise with Kafka, Spark, and S3 / lakehouse-style storage architectures
- Strong proficiency with Snowflake, ingestion, transformation, and performance optimization
- Deep understanding of ingestion optimization, rate limits, micro-batching, incremental loads, and CDC
- Experience applying DevOps / CI/CD practices to data pipelines
- Ability to work independently and deliver solutions rapidly in a fast-paced, iterative environment
Desirable:
- Experience with additional open-source ingestion frameworks and connectors
- Exposure to event-driven or telemetry-heavy architectures at scale
- Dynatrace is a leader in unified observability and security.
- We provide a culture of excellence with competitive compensation packages designed to recognize and reward performance.
- Our employees work with the largest cloud providers, including AWS, Microsoft, and Google Cloud, and other leading partners worldwide to create strategic alliances.
- You'll get to work at the forefront of innovation with Dynatrace Intelligence, the industry's first agentic operations system. Bringing together deterministic and agentic AI, it helps teams understand what's happening, why it matters, and what to do next, automatically.
- Over 50% of the Fortune 100 companies are current customers of Dynatrace
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
