The Senior Databricks Data Engineer will design, develop, and optimize data pipelines, collaborate with teams for architecture, and implement ETL processes, ensuring high performance for analytics solutions.
Xenon7 is on the forefront of integrating advanced data solutions into business operations. We are looking for a Senior Databricks Data Engineer to join our dynamic team. In this role, you will leverage your expertise in Databricks to build and optimize data pipelines and analytics solutions that drive impactful business insights. You will work closely with data scientists and analysts to ensure data flows seamlessly and efficiently through various systems. Join us to contribute to innovative projects that harness the power of data and artificial intelligence.
Responsibilities:- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
- Collaborate with data teams to define data architecture and design best practices for data processing.
- Implement ETL processes to extract data from various sources and transform it for analysis.
- Optimize data storage and retrieval processes to ensure high performance and reliability.
- Work closely with data scientists to provide clean and accessible data for modeling and analysis.
- Monitor and troubleshoot data pipeline failures, implementing proactive measures to improve reliability.
- Stay updated with industry trends and best practices regarding data engineering and Databricks technologies.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Data Engineer, with at least 7 years of experience, specifically in Databricks.
- Strong proficiency in Apache Spark and Python (or Scala).
- Experience with data modeling, ETL processes, and data warehousing concepts.
- Hands-on experience in designing and implementing data solutions on Databricks platform.
- Familiarity with cloud platforms, preferably AWS or Azure.
- Strong understanding of database technologies, including SQL and NoSQL databases.
- Excellent analytical and problem-solving skills.
- Strong communication and collaboration skills to work effectively in a team environment.
- Experience with data governance and data quality frameworks is a plus.
Top Skills
Spark
AWS
Azure
Databricks
NoSQL
Python
Scala
SQL
Similar Jobs
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
The role involves building high-quality software, mentoring colleagues, and implementing features while collaborating with product owners to fulfill detailed requirements.
Top Skills:
AngularJavaJavaScriptReactVue
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
The Senior Software Engineer will create high-quality code, collaborate with product owners, implement software solutions, and mentor colleagues.
Top Skills:
AIAngularJavaJavaScriptReactVue
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
In this role, you will develop scalable software, mentor teammates, and integrate AI into work processes while ensuring code quality through best practices.
Top Skills:
AngularJavaJavaScriptReactVue
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
