Quantiphi Logo

Quantiphi

Senior Data Engineer

Posted 4 Days Ago
Be an Early Applicant
In-Office
2 Locations
Senior level
In-Office
2 Locations
Senior level
Design, build, and operate large-scale batch and real-time data pipelines using Spark, AWS (Glue, EMR, S3, Redshift). Implement observability, alerts, and drive technical deliverables while mentoring junior engineers.
The summary above was generated by AI

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Must have skills:

  • 3+ years of hands-on experience in building large-scale data pipelines.

  • Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads

  • Proficiency in Pyspark/Python, including building Kafka producers for data ingestion

  • Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.

  • Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager

  • Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)

  • Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog

  • Strong SQL skills for validating data accuracy and developing job-level metrics

  • Experience with building alerting mechanisms for pipeline failures and performance issues

  • Strong collaboration and communication skills

  • Proven ability to take ownership of technical deliverables, driving execution from design to deployment

  • Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices

  • AWS Data Engineering - Associate Certified.

Good to have skills : 

  • Experience with orchestration tools like Apache Airflow or AWS Step Functions

  • Exposure to Bigdata stacks like Sqoop,HDFS,Hive,Nifi

  • Exposure to Terraform for infrastructure automation

  • Familiarity with CI/CD pipelines for data workflows

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Top Skills

Spark Streaming,Aws Glue,Emr,Pyspark,Python,Kafka,Confluent Kafka,S3,Redshift,Cloudwatch,Secrets Manager,Datadog,Sql,Oracle,Medallion Architecture,Apache Airflow,Aws Step Functions,Sqoop,Hdfs,Hive,Nifi,Terraform,Ci/Cd

Similar Jobs

7 Days Ago
Hybrid
Pune, Maharashtra, IND
Senior level
Senior level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design and develop complex, large-scale data pipelines and end-to-end solutions across enterprise Big Data environments. Ensure data reliability, performance, and adherence to engineering best practices while collaborating in Agile teams.
Top Skills: Apache Spark,Scala,Python,Java,Hadoop,Object Storage,Sql,Oracle,Netezza,Hive,Apache Nifi,Data Warehouse,Data Lake,Lakehouse
2 Hours Ago
In-Office
Pune, Maharashtra, IND
Senior level
Senior level
Big Data • Cloud • Digital Media • Machine Learning • Mobile • Software • Industrial
Design, implement, and manage data security solutions; perform security assessments and monitoring; enforce encryption and key management; lead breach investigations and incident response; collaborate with IT/dev teams; provide training and report security posture to senior management.
Top Skills: Python,Pyspark,Sql,Aws,Azure,Google Cloud,Firewalls,Ids/Ips,Encryption,Iam,Penetration Testing,Vulnerability Scanning,Risk Assessment,Digital Forensics,Scripting/Automation
13 Days Ago
In-Office
Mumbai, Maharashtra, IND
Senior level
Senior level
Fintech • Software • Financial Services
The Senior Data Engineer is responsible for designing, building, and maintaining the data infrastructure and architecture. They ensure data accuracy and facilitate data accessibility for analytics and business intelligence purposes.

What you need to know about the Pune Tech Scene

Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account