Lead healthcare data engineering projects by designing and implementing real-time data pipelines, ensuring compliance, and mentoring junior engineers.
Job Title: SSE – Pyspark, Python, and Azure Databricks (Healthcare Data Project)
Experience: 5 to 8 years
Role Overview:
We are looking for a highly skilled with expertise in Kafka, Python, and Azure Databricks (preferred) to drive our healthcare data engineering projects. The ideal candidate will have deep experience in real-time data streaming, cloud-based data platforms, and large-scale data processing. This role requires strong technical leadership, problem-solving abilities, and the ability to collaborate with cross-functional teams.
Key Responsibilities:
- Lead the design, development, and implementation of real-time data pipelines using Kafka, Python, and Azure Databricks.
- Architect scalable data streaming and processing solutions to support healthcare data workflows.
- Develop, optimize, and maintain ETL/ELT pipelines for structured and unstructured healthcare data.
- Ensure data integrity, security, and compliance with healthcare regulations (HIPAA, HITRUST, etc.).
- Collaborate with data engineers, analysts, and business stakeholders to understand requirements and translate them into technical solutions.
- Troubleshoot and optimize Kafka streaming applications, Python scripts, and Databricks workflows.
- Mentor junior engineers, conduct code reviews, and ensure best practices in data engineering.
- Stay updated with the latest cloud technologies, big data frameworks, and industry trends.
Required Skills & Qualifications:
- 4+ years of experience in data engineering, with strong proficiency in Kafka and Python.
- Expertise in Kafka Streams, Kafka Connect, and Schema Registry for real-time data processing.
- Experience with Azure Databricks (or willingness to learn and adopt it quickly).
- Hands-on experience with cloud platforms (Azure preferred, AWS or GCP is a plus).
- Proficiency in SQL, NoSQL databases, and data modeling for big data processing.
- Knowledge of containerization (Docker, Kubernetes) and CI/CD pipelines for data applications.
- Experience working with healthcare data (EHR, claims, HL7, FHIR, etc.) is a plus.
- Strong analytical skills, problem-solving mindset, and ability to lead complex data projects.
- Excellent communication and stakeholder management skills.
Top Skills
Azure Databricks
Docker
Kafka
Kubernetes
NoSQL
Python
SQL
Similar Jobs
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
The Senior Engineer ISO Documentation Coordinator ensures compliance with ISO standards by managing and auditing technical documentation related to utility engineering operations, preparing for audits, and facilitating process improvements.
Top Skills:
AutocadIso 14001Iso 45001Iso 9001MS Office
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
The position involves providing engineering support for C-17 aircraft, resolving technical issues, conducting analysis, and collaborating with international teams in a military aviation setting.
Top Skills:
Aircraft StructuresAvionics SystemsComputer SystemsMechanical SystemsMilitary Transport Aircraft Maintenance
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
As a Python Developer, you will develop fullstack applications, manage backend systems, and collaborate to drive technical solutions that impact business results.
Top Skills:
C++DockerGoJavaJavaScriptKubernetesNext.JsPythonReactRust
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
