The Senior Data Engineer will design and develop scalable data pipelines, manage data workflows, and ensure data quality while collaborating with BI teams and enforcing governance policies.
Job Title: Senior Data Engineer
Location: Remote
Experience: 7 - 10 Years
Only CV's which are as per JD will be shortlisted
We are seeking a highly skilled Senior Data Engineer to design, develop, and maintain scalable data pipelines and modern data platforms leveraging the Microsoft Fabric and Azure ecosystem. This role is critical in enabling data-driven decision-making by building efficient, reliable, and high-performance data solutions that support analytics, reporting, and advanced data use cases across the organization.
Key ResponsibilitiesData Engineering & Development- Design, build, and maintain robust ETL/ELT pipelines using Azure Data Factory (ADF) and Microsoft Fabric
- Develop and optimize data workflows using PySpark and SQL
- Implement and manage Lakehouse architecture using the Medallion (Bronze, Silver, Gold) layers
- Work with Azure Data Lake Storage (ADLS) and Delta Lake for scalable data storage and processing
- Integrate data from multiple sources while ensuring data quality, consistency, and reliability
- Partner with BI developers and analysts to enable data models and reporting solutions
- Collaborate with cross-functional stakeholders to translate business requirements into technical solutions
- Monitor, troubleshoot, and optimize data pipelines and system performance
- Ensure high availability and scalability of data platforms
- Implement and enforce data governance, security, and compliance best practices
- Ensure proper data lineage, cataloging, and documentation
- Lead the design and architecture of scalable data platforms and solutions
- Define and enforce data engineering standards, frameworks, and best practices
- Mentor junior data engineers and conduct code reviews to ensure quality and performance
- Optimize large-scale data processing and improve system efficiency and cost optimization
- Drive adoption of Microsoft Fabric and modern data engineering practices across teams
- 5+ years of experience in Data Engineering
- Strong hands-on experience with:
- Azure Data Factory (ADF)
- Microsoft Fabric
- PySpark and SQL
- Azure Data Lake Storage (ADLS)
- Delta Lake
- Solid understanding of data modeling, ETL/ELT concepts, and Lakehouse architecture
- Experience with Medallion architecture (Bronze, Silver, Gold layers)
- Strong problem-solving and performance optimization skills
- Experience working in cloud-based data platforms (Azure preferred)
- Experience with Power BI or other BI tools
- Familiarity with data governance and security frameworks
- Exposure to CI/CD pipelines for data engineering workflows
- Experience working in Agile/Scrum environments
Similar Jobs
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
As a Senior Data Engineer, you will design and build cloud-based data solutions, develop data pipelines, ensure data quality, and collaborate with stakeholders to optimize data storage and retrieval.
Top Skills:
Ab-InitioAirflowAzure Data FactoryAzure DatabricksAzure Datalake Gen2Azure Stream AnalyticsAzure Synapse AnalyticsBig QueryCloud FunctionCloudwatchDatabricksDataflowDataplexDataprocDbtEmrFivetranGcsGlueGoInformaticaJavaKafkaLambdaNeo4JPl/SqlPubsubPysparkPythonRedshiftS3SQLTalend
Artificial Intelligence • Automotive • Computer Vision • Information Technology • Internet of Things • Logistics • Software
The Sr. Quality Engineer applies spatial data engineering techniques to validate data quality, develop automated QA pipelines, and support geospatial machine learning for quality evaluation in accordance with SD/HD Map specifications.
Top Skills:
ArcgisAWSFmeJSONPythonQgisSQLTableauXML
Information Technology • Consulting
The Senior Data Engineer role involves designing and operating cloud-native data solutions and building reliable pipelines for analytics and AI, requiring extensive knowledge in data engineering and collaboration with various teams.
Top Skills:
AuroraAWSAzureDatabricksJavaPostgresPythonSparkSQL
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
