The Technical Lead will develop and maintain ELT/ETL pipelines, focusing on data quality and schema enforcement, while utilizing Databricks and PySpark.
Data Engineer - Databricks,Spark / PySpark,Kafka
• Write efficient Pyspark code and libraries for Databricks notebooks using best practices.
• Build and maintain ELT/ETL pipelines using Medallion Architecture.
• Implement batch pipelines using Databricks,Spark / PySpark,Kafka
• Ensure data quality, schema enforcement, and lineage.
• Support ingestion of healthcare data sources with incremental and CDC logic
• Maintaining the code base, code deployment and infra setup for data pipelines .
• Experience in MDM, Data Mart
• Exposure to incentive planning and compensation.
• Experience in healthcare domain
Top Skills
Databricks
Kafka
Pyspark
Spark
Similar Jobs
Insurance
The Technical Lead will oversee global Operations and Technology teams, focusing on integrating legacy systems with modern applications, cloud digital transformation, and leading operational support for applications and infrastructure.
Top Skills:
Apache CamelAppdynamicsAzureAzure DevopsDockerDynatraceElastic ElkGitlabsGrafanaJenkinsKafkaKubernetesOpensearchPrometheusSplunkSpring Boot
Insurance
The role involves designing and maintaining reporting and data integration solutions, developing ETL pipelines using SSIS, and creating interactive reports with SSRS.
Top Skills:
Azure Data FactoryGitPower BISql Server 2019/2022+SsisSsrsT-SqlTfs
Insurance
The Technical Lead will guide engineering teams, develop enterprise software solutions, ensure compliance with best practices, and mentor team members while working in an Agile environment.
Top Skills:
AngularAzureBootstrapCSSDotnetGraphQLHTMLJavaScriptReactRestSoapTypescript
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
