Design, build, and optimize large-scale cloud data platforms and ETL/ELT pipelines on Azure. Implement data lakes, Delta Lake, Synapse, and Databricks solutions for batch and real-time processing, ensure data quality/governance, optimize SQL and analytics workloads, support CI/CD and deployments, and collaborate with stakeholders to deliver scalable data architectures and reporting.
This role is for one of the Weekday's clients
Salary range: Rs 500000 - Rs 2000000 (ie INR 5-20 LPA)
Experience: 4+ yrs
Location: Bengaluru, Pune, Hyderabad, NCR, Chennai
Job Type: full-time
We are seeking a Data Engineer / Senior Data Engineer to design, build, and optimize large-scale cloud-based data platforms that support business intelligence, analytics, and data-driven decision-making. This role focuses on developing modern data pipelines, data lakes, and warehouse solutions while working with real-time and batch processing systems in a cloud environment. You will collaborate with cross-functional teams to deliver scalable, high-quality data architectures that enable actionable insights across the organization.
Requirements
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for processing large volumes of structured and unstructured data.
- Build and manage cloud-based data platforms, including data lakes, data warehouses, and analytics solutions.
- Develop and optimize data pipelines using Spark, PySpark, and cloud-native data services.
- Work with real-time and batch data processing frameworks to ensure reliable and efficient data delivery.
- Design robust data models and implement best practices for data quality, governance, and performance.
- Optimize SQL queries, database performance, and large-scale analytics workloads.
- Develop and maintain reporting and visualization solutions to support business stakeholders.
- Collaborate with business, engineering, analytics, and product teams to define data requirements and solutions.
- Support CI/CD processes, automation, monitoring, and deployment of data engineering solutions.
- Create technical documentation and ensure adherence to modern engineering and data management standards.
What Makes You a Great Fit
- 7+ years of experience in Data Engineering, ETL development, and cloud-based analytics solutions.
- Strong expertise in Azure Data Factory, Databricks, Azure Synapse Analytics, Spark, and PySpark.
- Hands-on experience with data lakes, Delta Lake architecture, and modern big data technologies.
- Strong SQL development, query optimization, and performance tuning skills.
- Experience designing scalable cloud-hosted data platforms and enterprise-grade data architectures.
- Knowledge of reporting and analytics tools, including data modeling and dashboard development.
- Familiarity with CI/CD pipelines, DevOps practices, and cloud security concepts such as RBAC.
- Strong problem-solving, communication, and stakeholder management skills.
- Ability to translate business requirements into scalable technical data solutions while maintaining high standards of quality and reliability.
Similar Jobs
Fintech • Consulting
The Azure Data Engineer will design and maintain data pipelines, optimize data workflows, and ensure system performance, leveraging Azure technologies.
Top Skills:
Azure Data FactoryAzure Data LakeAzure DatabricksPysparkPythonSQL
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Lead business analysis and delivery for loans transformations: gather end-to-end requirements, map current/target-state loan processes, define user stories and acceptance criteria, support solution design, drive UAT, and manage delivery governance, risks, and milestone reporting across stakeholders.
Top Skills:
Advanced AnalyticsAgileAi-Enabled SolutionsAutomationConfluenceJIRA
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Senior SAS developer responsible for designing, developing, and optimizing SAS programs and ETL workflows. Work includes Base SAS, macros, PROC SQL, SAS DI Studio and EG, scheduling (LSF), Unix/shell scripting, performance tuning of SAS servers, and transforming banking data for enterprise data warehouses. Collaborate with clients, QA, DBAs and support teams.
Top Skills:
Base SasETLHadoopLsf Platform SuiteProc SqlSASSas Data Integration StudioSas EgSas MacroSas ServersShell ScriptingSQLUnix
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
