Design and implement scalable ETL pipelines using AWS and Python, optimize data solutions, and ensure data quality and compliance.
- Design and build scalable ETL pipelines and data integration workflows using AWS services and Python.
- Develop and optimize data lake and data warehouse solutions for structured and unstructured data.
- Leverage Apache Spark for large-scale data processing and transformation tasks.
- Collaborate with cross-functional teams to gather requirements and deliver clean, usable datasets for analytics and reporting.
- Ensure high data quality, security, and compliance in all stages of the data lifecycle.
- 5+ years of hands-on experience in data engineering roles.
- Proficiency in Python for building and automating data pipelines.
- Strong experience with AWS services (S3, Glue, Redshift, Lambda, etc.).
- Solid understanding of ETL processes and modern data warehousing concepts.
- Experience with big data tools, especially Apache Spark (PySpark preferred).
- Familiarity with DevOps and CI/CD practices for data pipeline deployment.
- Knowledge of data governance and cataloging tools.
- Strong problem-solving, communication, and collaboration skills.
Top Skills
Spark
AWS
Aws Glue
Aws Lambda
Aws Redshift
Aws S3
ETL
Python
Similar Jobs
Information Technology • Software
As a DevOps Engineer, you will automate deployment processes, support development teams, improve infrastructure reliability, and mentor teammates.
Top Skills:
.NetAzure Cloud ServicesAzure DevopsBashC#DockerGitKubernetesNpmNugetPowershell
Information Technology • Software
Execute standardized SaaS services, manage incidents, collaborate on service delivery, and contribute to improvements under supervision.
Top Skills:
Elastic SearchItsm ToolsJira Service DeskKubernetesLinuxAzureMongoDBOpsgeniePagerdutyRedisService Now
Information Technology • Software
The Senior Deal Desk Analyst collaborates with sales teams to build and approve quotes, ensuring compliance and managing commercial negotiations while driving operational efficiency.
Top Skills:
ExcelPowerPointSalesforceSalesforce Cpq
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
