Design and develop data ingestion frameworks and real-time processing solutions while optimizing and operationalizing data analytics tools in GCP.
It's fun to work in a company where people truly BELIEVE in what they are doing!
We're committed to bringing passion and customer focus to the business.
Responsibilities
- Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks.
- Hands-on on technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python, Hadoop Platform, Hive, Presto, Druid, airflow
- Deep understanding of BigQuery architecture, best practices, and performance optimization.
- Proficiency in LookML for building data models and metrics.
- Experience with DataProc for running Hadoop/ Spark jobs on GCP.
- Knowledge of configuring and optimizing DataProc clusters.
- Offer system support as part of a support rotation with other team members.
- Operationalize open source data-analytic tools for enterprise use.
- Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification.
- Understand and follow the company development lifecycle to develop, deploy and deliver the solutions.
Minimum Qualifications:
• Bachelor's degree in Computer Science, CIS, or related field
• Experience on project(s) involving the implementation of software development life cycles (SDLC) GCP DATA ENGINEER
Required Skills & Qualifications:
- Education: Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
- 7+ years of experience in data engineering, cloud data solutions, and pipeline development.
- GCP Expertise: Hands-on experience with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Composer (Airflow), Vertex AI, and IAM Policies.
- Programming: Proficiency in Python, SQL, and Apache Beam (Java or Scala is a plus).
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
Not the right fit? Let us know you're interested in a future opportunity by clicking Introduce Yourself in the top-right corner of the page or create an account to set up email alerts as new job postings become available that meet your interest!
Top Skills
Airflow
Spark
BigQuery
Dataproc
Druid
Hadoop
Hive
Java
Kafka
Presto
Python
Scala
Similar Jobs
Software • Consulting
The Senior DevOps Engineer will architect and manage scalable infrastructure on GCP, automate provisioning with Terraform, and enhance CI/CD pipelines while focusing on developer productivity and tooling.
Top Skills:
Ci/CdElkGCPGithub ActionsGitlab CiGoGrafanaJenkinsKubernetesPrometheusPythonTerraform
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
As Engineering Manager, lead the performance test team in developing and validating endpoint security solutions, focusing on sensor performance quality across platforms.
Top Skills:
C/C++JenkinsLogscalePythonSplunkSQL
Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Cybersecurity • Data Privacy
The Software Engineer will design, develop, and maintain software applications using C++. Responsibilities include troubleshooting, collaborating with teams, and writing documentation.
Top Skills:
C++PythonScala
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
