Top Data Engineer Jobs in Pune

Support, maintain, troubleshoot, and optimize production Databricks applications, Spark jobs, SQL queries, Delta Lake pipelines, clusters, workflows, and workspace configurations. Implement data quality checks, logging, alerting, governance, security, and cost-management practices. Collaborate on enhancements and bug fixes, reduce technical debt, document operational processes, and participate in EST-aligned support rotations.

Develop AI-powered product features for local discovery, including personalized recommendations, content curation, search, and ranking. Build data pipelines by scraping, cleaning, and structuring datasets for machine learning models. Collaborate with product, design, and engineering teams to integrate AI into the platform, while continuously improving machine learning capabilities and user experiences.

Build, monitor, optimize, and maintain cloud-based ETL/ELT pipelines supporting business intelligence and reporting. Develop high-performance SQL transformations, manage data warehouse schemas, tune query and pipeline performance, and optimize cloud costs. Collaborate with analysts, data scientists, actuaries, and architects to translate insurance and SaaS requirements into scalable technical solutions. Participate in code reviews, documentation, troubleshooting, and data integrity initiatives.

Design, build, and support production-grade AWS data pipelines that transform operational data into secure, high-quality, AI-ready datasets. Responsibilities include distributed data processing, Parquet curation, privacy-preserving transformations, orchestration, data quality monitoring, schema management, CI/CD, infrastructure as code, metadata and lineage management, troubleshooting, and reliable backfills.

Build production-grade agentic AI systems that reason, search, and execute workflows. Develop RAG pipelines, AI agents, semantic search layers, data processing solutions, dimensional data models, and scalable AI workloads. Work with structured and unstructured data, LLMs, vector databases, cloud infrastructure, CI/CD, and ML observability. Responsibilities include fine-tuning models, prompt engineering, optimizing context windows, and deploying reliable AI applications.

Designs and operates Databricks data platforms, pipelines, workflows, CI/CD processes, and cloud infrastructure on Azure. Responsibilities include cluster and workspace management, automated testing, Infrastructure as Code, access controls, FinOps, Airflow orchestration, monitoring, incident response, and reusable Terraform and Databricks templates. The role also improves platform reliability, security, developer productivity, and production code promotion processes while partnering with engineering, architecture, and security teams.

Design, develop, and optimize secure, scalable cloud data platforms and ETL/ELT pipelines across AWS, Azure, and GCP. Write Python and SQL, implement data governance and quality controls, optimize performance and costs, and support analytics and BI layers. Contribute to architecture, pre-sales demos, proofs of concept, and client discussions. Mentor junior developers, conduct code reviews, and collaborate with architects, QA, DevOps, and business stakeholders.

Designs and develops scalable cloud data platforms and ETL/ELT pipelines using Python and SQL. Optimizes performance and costs, implements governance, security, quality, and compliance controls, and supports analytics consumption. Collaborates with architects and stakeholders, contributes to pre-sales prototypes and demos, mentors junior developers, conducts code reviews, and recommends improvements across AWS, Azure, and GCP environments.

Develops and supports big data applications and infrastructure using technologies such as Spark, Hadoop, cloud services, and workflow orchestration tools. Responsibilities include coding, code reviews, testing, production incident support, release deployments, root-cause analysis, integration, documentation, and continuous improvements to scalability, reliability, availability, and performance. The role also provides Level 3 technical support, assists junior team members, and collaborates across the software development lifecycle.

Design, develop, and maintain scalable Microsoft Fabric and Azure data solutions. Build Lakehouse, Data Warehouse, OneLake, ETL/ELT pipelines, data transformations, and integrations using Python, PySpark, SQL, and Fabric tools. Implement medallion architecture, optimize data models and workloads, establish data quality monitoring, and support CI/CD deployments. Collaborate with analytics and business teams to deliver enterprise-scale data platforms.

Design and develop secure, optimized data pipelines for real-time and batch processing using Azure and big data technologies. Translate business requirements into scalable data solutions, improve data quality and governance, support machine learning operationalization, and implement validation and monitoring frameworks. Collaborate with stakeholders and data scientists, participate in code reviews and documentation, and mentor junior engineers.

Build and maintain secure, scalable data architectures, pipelines, warehouses, and lakes. Develop data processing and analysis algorithms, support machine learning model deployment, and collaborate with data scientists. The role also involves infrastructure and deployment pipeline evolution, automation, CI/CD, containerization, production troubleshooting, risk management, stakeholder collaboration, and potentially leading or advising teams.

Build and maintain batch and streaming data pipelines across AWS, Azure, and GCP using Databricks, Spark, PySpark, SQL, and Delta Lake. Implement medallion architecture, CDC, SCD, data quality, governance, security, metadata, and lineage controls. Configure storage, orchestration, CI/CD, monitoring, troubleshooting, and performance optimization. Collaborate with architects, data scientists, engineers, analysts, and product teams while documenting pipelines, transformations, testing, and operational procedures.

Design and implement scalable end-to-end data pipelines and storage schemas across cloud and on-prem platforms. Define reference architectures and engineering standards, ensure governance/security/compliance by design, optimize performance for large datasets, and mentor/lead global teams using Python/Java, SQL, Spark, Databricks, and related data platforms.

Design, build, and scale cloud-native data and BI solutions on Databricks and Snowflake. Develop high-performance data pipelines, enable multi-cloud interoperability (AWS/Azure), implement governance and secure access patterns, maintain CI/CD and infrastructure-as-code for platform provisioning, and lead technical design while mentoring engineers.

Design, build, and maintain scalable data pipelines and ETL/ELT workflows using Azure Data Factory, Databricks, PySpark, Python, and SQL. Transform large datasets, optimize data ingestion and queries, monitor pipeline performance, resolve data quality issues, support data modeling, document engineering standards, and contribute to cloud data infrastructure improvements. The role also involves collaboration with architects and senior engineers and requires familiarity with Azure Synapse, Delta Lake, governance, and security.

Lead design, build and modernize enterprise data platforms and pipelines, migrate on-prem processing to AWS, ensure secure, high-quality data delivery, collaborate with data scientists on ML deployment, and provide strategic leadership or subject-matter expertise.

Design, build, and maintain scalable data pipelines, warehouses, and lakes; ensure data quality, security, and operational stability; collaborate with data scientists to deploy ML models; lead strategy, governance, and delivery for data modernization and regulatory requirements.

Build and maintain secure, scalable data architectures, pipelines, warehouses, and lakes. Develop data processing and analysis algorithms, support machine learning model deployment, own platform defects, review code, and improve analytics platform design. Collaborate with quants, architects, technology teams, and stakeholders while ensuring compliance with security, privacy, accessibility, risk, and governance standards. The role also involves advising teams, coordinating resources, and driving continuous engineering and operational improvements.

Principal Data Engineer responsible for developing and supporting Databricks pipelines, integrating heterogeneous healthcare data sources, modeling unified data products, and ensuring data quality and availability. The role partners with business stakeholders, product teams, data engineers, and clinical leaders to gather requirements, create datasets, support ad hoc analytics, enable machine learning and AI use cases, and assist with acquisition integrations. Extensive healthcare data, cloud, SQL, PySpark, and Python experience is required.

Develop and maintain Python-based API data ingestion services for GCP. Build secure, scalable, fault-tolerant pipelines with authentication, incremental loading, validation, reconciliation, orchestration, monitoring, and alerting. Manage data quality, PII masking, CI/CD, production troubleshooting, performance optimization, SLAs, data contracts, technical designs, mappings, and operational documentation. Collaborate with stakeholders to define support models and ensure reliable ingestion from internal and external APIs.

Design, build, and maintain scalable data pipelines, warehouses and lakes; implement processing and analysis algorithms; support production deployments (DEV->UAT->PROD); ensure data quality, security, and governance; collaborate with data scientists to deploy ML models; lead or mentor team members and influence stakeholders to mitigate risk.

Designs and refines data platform blueprints, collaborates with Integration and Data Architects, implements data integration and modeling solutions, ensures data quality for large datasets, mentors junior staff, and proposes improvements and risk mitigations for a cohesive data platform.

Lead and execute acquisitions, disposals and strategic partnerships: identify targets, build pipelines, perform valuations and financial analysis, develop business cases, negotiate deal structures, manage post-merger integration, monitor performance and advise stakeholders. Supervise a team, embed risk controls, collaborate across functions, and maintain strategic alignment with group objectives.

Build and maintain scalable data products using Snowflake, dbt, and Airflow. Responsibilities include dimensional data modeling, SQL transformations, pipeline orchestration, data quality testing, exploratory analysis, SLA monitoring, documentation, and stakeholder requirement gathering. Partner with analytics, product, engineering, and business teams to deliver reliable, production-ready datasets while following software engineering, CI/CD, testing, and deployment practices.