Design, develop, and maintain scalable GCP data platforms and batch or streaming pipelines using Dataflow, Apache Beam, BigQuery, Python, SQL, Cloud Composer, and Airflow. Build data warehouses, dimensional models, ETL/ELT processes, and analytical solutions. Ensure data quality, security, governance, monitoring, reliability, and cost efficiency while troubleshooting production issues. Collaborate with technical and business stakeholders, support CI/CD and cloud modernization, and independently lead complex data engineering initiatives.
𝗧𝗵𝗶𝘀 𝗿𝗼𝗹𝗲 𝗶𝘀 𝗳𝗼𝗿 𝗼𝗻𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗪𝗲𝗲𝗸𝗱𝗮𝘆'𝘀 𝗰𝗹𝗶𝗲𝗻𝘁𝘀
𝗦𝗮𝗹𝗮𝗿𝘆 𝗿𝗮𝗻𝗴𝗲: 𝗥𝘀 𝟲𝟬𝟬𝟬𝟬𝟬 - 𝗥𝘀 𝟮𝟲𝟬𝟬𝟬𝟬𝟬 (𝗶𝗲 𝗜𝗡𝗥 𝟲-𝟮𝟲 𝗟𝗣𝗔)
Experience: 6+ yrs
Location: India
Job Type: Full-time
We are looking for an experienced GCP Data Engineer with strong expertise in Google Cloud, BigQuery, Dataflow, Apache Beam, Cloud Composer, Airflow, Python, SQL, data warehousing, and data modelling.
The role involves designing, developing, and maintaining scalable data platforms and pipelines that support analytics, reporting, business intelligence, and data-driven applications. The ideal candidate will have strong hands-on experience working with large datasets, cloud-native data engineering services, batch and streaming pipelines, and modern data warehouse architectures.
You will work closely with data architects, analysts, software engineers, and business stakeholders to transform complex data requirements into reliable, scalable, and high-performance data solutions.
Requirements
Key Responsibilities
- Design, develop, and maintain scalable data pipelines on Google Cloud Platform (GCP).
- Build batch and streaming data pipelines using Dataflow and Apache Beam.
- Develop and optimize analytical solutions using Google BigQuery.
- Design and implement ETL/ELT processes for structured and semi-structured data.
- Develop reusable data processing frameworks using Python.
- Write complex and optimized SQL queries for data transformation, analysis, and validation.
- Build and manage workflow orchestration using Cloud Composer and Apache Airflow.
- Design DAGs, scheduling mechanisms, dependencies, retries, alerts, and error-handling workflows.
- Develop scalable data warehouse and data modelling solutions to support reporting and analytics.
- Design fact tables, dimension tables, data marts, schemas, and analytical data structures.
- Integrate data from databases, APIs, files, applications, and other enterprise data sources.
- Implement data validation, quality checks, reconciliation, and monitoring across pipelines.
- Optimize BigQuery queries, Dataflow jobs, data models, and pipeline performance.
- Identify opportunities to improve scalability, reliability, maintainability, and cloud cost efficiency.
- Troubleshoot pipeline failures, data quality issues, performance bottlenecks, and production incidents.
- Implement appropriate data security, access controls, governance, and privacy practices.
- Collaborate with Data Architects, Data Scientists, BI Developers, Software Engineers, and business stakeholders.
- Support deployment and automation using Git, CI/CD, and modern cloud development practices.
- Maintain technical documentation covering pipelines, data models, workflows, dependencies, and operational procedures.
- Contribute to migration and modernization of legacy data platforms into GCP-based cloud data architectures.
What Makes You a Great Fit
- 6+ years of professional experience in data engineering, data platform development, or a related field.
- Strong hands-on expertise in GCP / Google Cloud data engineering services.
- Proven experience with BigQuery for large-scale analytical workloads.
- Strong experience with Dataflow and Apache Beam for batch and streaming data processing.
- Hands-on experience with Cloud Composer and Apache Airflow for workflow orchestration.
- Strong programming skills in Python and advanced SQL.
- Strong understanding of data warehousing, data modelling, ETL/ELT, and analytical data architectures.
- Experience designing dimensional models, fact and dimension tables, data marts, and scalable schemas.
- Good understanding of batch and real-time data processing architectures.
- Experience working with large datasets and optimizing data processing and query performance.
- Strong knowledge of data quality, validation, monitoring, governance, and security practices.
- Experience with Git, CI/CD, automation, and production deployment workflows.
- Familiarity with other GCP services such as Cloud Storage, Pub/Sub, Dataproc, Cloud Functions, or Cloud Run is an advantage.
- Experience with Terraform or Infrastructure-as-Code is beneficial.
- Strong troubleshooting, analytical, and problem-solving abilities.
- Excellent communication and collaboration skills across technical and business teams.
- Ability to independently own complex data engineering initiatives from design through production.
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related discipline is preferred.
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Analyze business and user requirements across financial services value streams, define technical requirements, develop cloud solution options, and manage business and IT stakeholders. The role requires understanding existing framework capabilities, complex data models, cross-functional solution design, clear documentation, team coordination across regions, and familiarity with the SDLC from design through testing and implementation.
Top Skills:
Google Cloud Platform
Information Technology
Design, build, and optimize scalable GCP data pipelines and BigQuery data warehouses. Develop ETL/ELT workflows, implement data modeling, ensure data quality, optimize query performance and costs, maintain orchestration with Cloud Composer/Dataflow/Pub‑Sub, and automate deployments with CI/CD while enforcing security and governance.
Top Skills:
BigQueryCi/Cd PipelinesCloud Composer (Airflow)Cloud FunctionsCloud RunCloud Storage (Gcs)DataflowGitMySQLPostgresPub/SubPythonShell ScriptingSQLSQL Server
Artificial Intelligence • Cloud • Sales • Security • Software • Cybersecurity • Data Privacy
Manage and grow SailPoints India partner ecosystem by developing regional partnership strategy, driving partner business plans and KPIs, enabling partners, coordinating with field sales, and generating pipeline through joint GTM activities and partner enablement.
Top Skills:
Sailpoint
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
