Lead design and implementation of scalable ETL/ELT data platforms on GCP. Develop high-performance SQL models and Python data applications, architect real-time event-driven solutions, ensure governance, security, CI/CD, and mentor engineering teams while collaborating with stakeholders to deliver reliable, cost-efficient data solutions.
Job Description:
Job TitleLead Developer / Engineer
Years of Experience6 to 10 years
Job DescriptionWe are seeking an experienced GCP Cloud Tech Lead – Data Engineering to lead the design, development, and modernization of scalable, secure, and high-performance data platforms on Google Cloud.
In this role, you will:
- Provide technical guidance
- Define cloud data architecture
- Mentor engineers
- Ensure best practices across data engineering, analytics, and AI initiatives
You will work closely with architects and business stakeholders to translate complex requirements into robust cloud-native data solutions.
Must-Have SkillsSQL & Python (Mandatory)
- Strong expertise in SQL and Python
- Advanced SQL for:
- Data modeling
- Transformations
- Performance tuning
- Analytics in BigQuery
- Strong Python proficiency with:
- OOP principles (encapsulation, inheritance, polymorphism, abstraction)
- SOLID principles
- Design patterns
Data Engineering & Pipelines
- Proven experience designing and building modular, scalable ETL/ELT pipelines
- Strong system design, troubleshooting, and performance tuning skills
- Experience working with distributed data systems
GCP Cloud Expertise
- Hands-on experience with Google Cloud Platform (GCP) services:
- BigQuery
- Cloud Storage (GCS)
- Cloud Functions
- Cloud Run
- Cloud Composer
- Pub/Sub
- Strong understanding of event-driven and real-time architectures using Pub/Sub and Eventarc
Integration & APIs
- Experience integrating and consuming REST APIs for data ingestion and platform integration
Data Governance & Security
- Solid understanding of:
- Data governance
- Data quality
- Privacy
- Security best practices in cloud environments
DevOps & CI/CD
- Experience building and automating CI/CD pipelines
- Familiarity with Cloud Build (preferred)
Collaboration & Communication
- Excellent verbal and written communication skills
- Ability to explain complex technical concepts to non-technical stakeholders
- Strong collaboration with product, engineering, QA, DevOps, and compliance teams
- Experience with Dataform for SQL-based transformations
- Exposure to Dataflow (Apache Beam) for large-scale batch and streaming processing
- Knowledge of cloud cost optimization and performance tuning on GCP
- Experience with API testing tools (e.g., Postman)
- Understanding of:
- Clean architecture
- Microservices
- Domain-driven design
- Experience building reusable frameworks, libraries, or accelerators
- Exposure to AI/ML pipeline orchestration
Architecture & Development
- Lead design and implementation of scalable ETL/ELT pipelines on GCP
- Develop high-performance SQL models and Python-based data applications
- Architect and maintain cloud-native solutions using:
- BigQuery, GCS, Cloud Functions, Cloud Run
- Cloud Composer, Pub/Sub, Eventarc, Dataform
Real-Time & Integration
- Define and implement event-driven, real-time data architectures
- Lead integration with external APIs and enterprise systems
Governance & Best Practices
- Establish standards for:
- Data quality
- Governance
- Lineage
- Security
- Compliance
DevOps & Performance
- Build and maintain CI/CD pipelines (Cloud Build)
- Ensure:
- Performance
- Scalability
- Reliability
- Cost efficiency
Leadership & Collaboration
- Provide technical leadership and mentorship
- Partner with cross-functional teams to deliver scalable solutions
- Maintain high-quality:
- Technical documentation
- Architecture diagrams
- Engineering standards
- Bachelor’s or Master’s degree in:
- Computer Science
- Information Systems
- Data Engineering
- Or a related field
- GCP Professional Data Engineer Certification (preferred)
- Snowflake Associate / Core certification
- Additional certifications (any one preferred):
- Databricks
- Snowflake
- Other cloud data platform certifications
- 12:00 PM – 9:00 PM IST
- OR 2:00 PM – 11:00 PM IST
Location:
DGS India - Bengaluru - Manyata N1 BlockBrand:
MerkleTime Type:
Full timeContract Type:
PermanentSimilar Jobs
Cloud • Information Technology • Security • Software • Cybersecurity
The Lead Technical Enablement Engineer designs and delivers training programs for technical roles, focusing on onboarding, collaboration, and driving engagement for Solutions Consulting teams.
Top Skills:
Cloud SecurityEnterprise Security TechnologiesNetworking
AdTech • Marketing Tech
Lead design and development of scalable cloud data platforms and ETL/ELT pipelines across AWS/Azure/GCP. Build and optimize data warehouses/lakehouses, implement MDM and governance, support analytics/BI, mentor junior engineers, and contribute to presales PoCs and solution designs.
Top Skills:
Agentic AiAWSAzureBigQueryDatabricksGCPInformaticaMdmProfiseePythonRedshiftReltioSnowflakeSQLSynapse
AdTech • Marketing Tech
Design and maintain scalable data architectures and ETL/ELT pipelines; write optimized SQL and Python code; deploy solutions on AWS (S3, Lambda) and Snowflake; implement data quality checks and monitoring; support GenAI initiatives; collaborate with stakeholders; document work and perform mid-to-complex tasks with minimal supervision.
Top Skills:
AWSEltETLGenaiLambdaPythonS3SnowflakeSQL
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
