Job Description
We are currently looking for an experienced Lead Data Engineer with demonstrated technical experience within pySpark, Phyton, and Java technologies to join the team in India, within our global Technology department. If this sounds interesting to you and you are skilled in Query engines Postgres/SQL/AWS/Java/Kotlin, we would love to speak with you.
About You – experience, education, skills, and accomplishments:
- Bachelor/master's degree in computer science, computer engineering, information technology, or relevant field
- At least 8+ years’ experience in working in software development
- At least 8+ years’ experience in building massively scalable distributed data processing solutions
- At least 6+ years’ experience of database design & development.
- Alt least 4 years of experience in Apache Spark, ElasticSearch
- Spark, ElasticSearch, Cassandra, Hadoop, Apache Hive, Snowflake, Jupiter notebook, databricks stack
- At least 4 years of experience in PostgreSQL or Oracle DB experience or Oracle 11g+, PostgresSQL 9+, AWS RDS
- At least 4 years of experience in Technologies and Tools like AWS, AWS Glue, Lambda
- Building Data Pipelines & ETL jobs using cloud-native technologies & design patterns
- Experience in designing resilient systems & creating disaster recovery plans
- Working in Master Data Management & designing CMSes, or evaluating 3rd parawty CMS products
- Working in Agile Scrum or Kanban teams & deploying solutions using Continuous Delivery best practices
- Using automated database migration tools & have strong opinions on version control best practices for SQL scripts
It would be great if you also had:
- Experience in designing resilient systems & creating disaster recovery plans
- Working in Master Data Management & designing CMSes, or evaluating 3rd parawty CMS products
- Working in Agile Scrum or Kanban teams & deploying solutions using Continuous Delivery best practices
- Using automated database migration tools & have strong opinions on version control best practices for SQL scripts
What will you be doing in this role?
- Provide technical thought leadership, compare different technologies to meet business requirements and cost control drivers.
- Work with Business and IT groups to design and deliver a data lake platform.
- Produce & maintain the overall solution design for the entire Data Lake Platform.
- Execution of data strategy, help in the design and architecture of solutions on the platform
- Enforce technical best practices for Big Data management and solutions, from software selection to technical architectures and implementation processes.
- Document and publish best practices, guidelines, and training information.
- Ensures all functional solutions and components of the Data Lake platform service are designed and implemented in a way to always meet SLAs.
- Contributes to the continuous improvement of the support & delivery functions by maintaining awareness of technology developments and making appropriate recommendations to enhance application services.
- Focus on data quality throughout the ETL & data pipelines, driving improvements to data management processes, data storage, and data security to meet the needs of the business and customers
About the Team
The existing team is spread across India with strong technical and analytical skills. We have a great skill set in Pyspark, AWS Glue, Data bricks to name a few of the Tools and technologies.
Hours of Work
This is a permanent position with Clarivate. You will be working in IST time zone. 9 hours of work per day including 1-hour lunch break. We have hybrid work model, flexible and employee friendly work environment.
At Clarivate, we are committed to providing equal employment opportunities for all qualified persons with respect to hiring, compensation, promotion, training, and other terms, conditions, and privileges of employment. We comply with applicable laws and regulations governing non-discrimination in all locations.
