The role involves designing and optimizing data pipelines using Databricks, leading a data engineering team, and ensuring data quality and compliance.
We are looking for a Senior Data Engineer with 6+ years of experience to design, build, and scale cloud‑native data and AI platforms on Azure using Databricks. The role requires strong hands‑on expertise in data engineering, lakehouse architecture, and AI/ML data pipelines to support advanced analytics, machine learning, and business intelligence use cases.
The ideal candidate will lead complex data initiatives, collaborate closely with data scientists and ML engineers, and play a key role in shaping the organization’s data and AI strategy.
ResponsibilitiesKey Responsibilities
- Architect and develop end‑to‑end data pipelines on Azure using Databricks (Spark / PySpark)
- Design and maintain lakehouse architectures using Azure Data Lake + Delta Lake
- Build and optimize batch and streaming pipelines for large‑scale datasets
- Create and manage feature pipelines and curated datasets for AI/ML model training and inference
- Collaborate with data scientists and ML engineers to enable scalable ML workflows
- Support MLOps pipelines, including data versioning, feature stores, and model deployment readiness
- Optimize Databricks workloads for performance, scalability, and cost efficiency
- Implement data quality, validation, monitoring, and observability frameworks
- Ensure data security, governance, and compliance using Azure and Databricks best practices
- Review code, define standards, and mentor junior and mid‑level data engineers
- Lead architectural decisions and contribute to data platform roadmap planning
Required Skills & Qualifications
- 6+ years of hands‑on experience in Data Engineering or Data Platform roles
- Strong proficiency in Python, PySpark, and Spark SQL
- Extensive experience with Databricks (jobs, notebooks, workflows, Delta Live Tables)
- Strong experience with Azure Cloud services, including:
- Azure Data Lake Storage (ADLS Gen2)
- Azure Databricks
- Azure Data Factory / Synapse Pipelines
- Solid understanding of Delta Lake, including optimization and ACID guarantees
- Advanced SQL skills for analytical data modeling
- Experience designing AI/ML data pipelines (training, validation, inference datasets)
- Knowledge of data warehousing, lakehouse, and dimensional modeling concepts
- Familiarity with CI/CD, Git, and DevOps practices
- Strong troubleshooting, performance tuning, and problem‑solving skills
- Knowledge on LangChain , Agent, Agent Architecture.
Preferred / Nice‑to‑Have Skills
- Experience with ML platforms such as Azure Machine Learning or Databricks ML
- Hands‑on experience with Feature Store, MLflow, or experiment tracking
- Streaming data experience (Kafka, Event Hubs, Spark Structured Streaming)
- Experience with dbt, Unity Catalog, or data governance tools
- Knowledge of BI and visualization tools (Power BI preferred)
- Exposure to MLOps best practices and production ML systems
- Prior experience as a technical lead or mentor
About UsHoneywell helps organizations solve the world's most complex challenges in automation, the future of aviation and energy transition. As a trusted partner, we provide actionable solutions and innovation through our Aerospace Technologies, Building Automation, Energy and Sustainability Solutions, and Industrial Automation business segments – powered by our Honeywell Forge software – that help make the world smarter, safer and more sustainable.
Similar Jobs
Aerospace • Security • Energy • Industrial
Design, build, and maintain scalable, secure .NET-based cloud-native applications on Microsoft Azure. Develop automation with PowerShell, design and consume REST APIs, support microservices and containerized deployments (AKS/ACS), implement persistence with SQL/NoSQL, enforce CI/CD and DevOps practices, participate in frontend/backend development (HTML/CSS/JavaScript/Python), and contribute to GenAI experiments and architecture documentation.
Top Skills:
.Net FrameworkAcs (Azure Container Services)Aks (Azure Kubernetes Service)Azure App ServicesAzure Service BusAzure Storage AccountsCi/CdCSSDevOpsGitHTMLJavaScriptMicroservicesAzureNoSQLPowershellPythonRestful ApisSQL
Aerospace • Security • Energy • Industrial
Design, develop, and maintain scalable, secure .NET software and cloud-native solutions on Azure. Build and consume RESTful APIs, support microservices and containerized deployments (AKS/ACS), automate tasks with PowerShell, ensure code quality via GitHub, implement CI/CD and testing, and collaborate on architecture, front-end/back-end components, and GenAI experiments.
Top Skills:
.Net FrameworkAcsAksApp ServicesAzure Container ServicesAzure Kubernetes ServiceCi/CdContainerizationCSSDevOpsGenaiGitHTMLJavaScriptMicroservicesAzureNoSQLPowershellPythonRestful ApisService BusSQLStorage AccountsTesting Frameworks
Aerospace • Security • Energy • Industrial
The Senior Quality Engineer will lead QE initiatives, ensure high-quality product delivery, develop test strategies, and automate testing for various applications, especially on the Salesforce platform.
Top Skills:
JavaJavaScriptJenkinsMongoDBOracleSalesforceSeleniumSQL
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
