The role involves translating business rules to technical specs, managing a data engineering team, and developing scalable data solutions. Responsibilities include performance optimization in Databricks, collaborating with Data Science and BI teams, and maintaining data quality checks.
Experience Required: 8+Years
Mode of work: Remote
Skills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, Spark
Notice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)
- Translate business rules into technical specifications and implement scalable data solutions.
- Manage a team of Data Engineers and oversee deliverables across multiple markets.
- Apply performance optimization techniques in Databricks to handle large-scale datasets.
- Collaborate with the Data Science team to prepare datasets for AI/ML model training.
- Partner with the BI team to understand reporting expectations and deliver high-quality datasets.
- Perform hands-on data modeling, including schema changes and accommodating new data attributes.
- Implement data quality checks before and after data transformations to ensure reliability.
- Troubleshoot and debug data issues, collaborating with source system/data teams for resolution.
- Contribute across project phases: requirement analysis, development, code review, SIT, UAT, and production deployment.
- Utilize GIT for version control and manage CI/CD pipelines for seamless deployment across environments.
- Adapt to dynamic business requirements and ensure timely delivery of solutions.
Requirements
- Strong expertise in Azure Databricks, PySpark, and SQL.
- Proven experience in data engineering leadership and handling cross-market deliverables.
- Solid understanding of data modeling and ETL/ELT pipelines.
- Hands-on experience with performance optimization in big data processing.
- Proficiency in Git, CI/CD pipelines, and cloud-based deployment practices.
- Strong problem-solving and debugging skills with large, complex datasets.
- Excellent communication skills and ability to collaborate with cross-functional teams.
Top Skills
Azure Data Factory
Azure Databricks
Ci/Cd
Eventhub
Git
Kafka
Pyspark
Python
Spark
SQL
Similar Jobs
Cloud • Information Technology • Security • Software
The Staff Software Engineer will manage identity and access synchronization with non-JumpCloud services, develop web services for AD integration, and design architecture for user authentication and migration to alternative directories.
Top Skills:
AWSCC++DockerElk StackGoJavaMongoDBNode.jsPythonRedisSaltstack
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
The role involves building scalable software, mentoring colleagues, integrating AI tools, and managing technical projects, focusing on Java and modern UI frameworks.
Top Skills:
Ai Productivity ToolsAngularApi IntegrationsJavaJavaScriptReactVue
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Develop high-quality, scalable code, manage design to delivery processes, mentor team members, and integrate AI technologies into projects.
Top Skills:
AIClaude CodeCursorJavaJavaScriptRdfSparqlTypescriptWindsurf
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
