The Azure Data Engineer will implement big data pipelines using Azure technologies, collaborate with stakeholders, and utilize tools like Data Factory and Databricks.
This is a remote position.
Description :
- At least 2 years hands on experience working with an Agile data engineering team working on big data pipelines using Azure in a commercial environment.
- Dealing with senior stakeholders/leadership
- Understanding of Azure data security and encryption best practices. [ADFS/ACLs]
- Data Bricks - experience writing in and using data bricks Using Python to transform, manipulate data.
- Data Factory - experience using data factory in an enterprise solution to build data pipelines. Experience calling rest APIs.
- Synapse/data warehouse - experience using synapse/data warehouse to present data securely and to build & manage data models.
- Microsoft SQL server - We- d expect the candidate to have come from a SQL/Data background and progressed into Azure
- PowerBI - Experience with this is preferred
Additionally :
- Experience using GIT as a source control system
- Understanding of DevOps concepts and application
- Understanding of Azure Cloud costs/management and running platforms efficiently
Requirements
- 3 to 6 years of experience.
- Must Have: Azure services like Azure Data Factory, Databricks, Azure Data lakes, Synapse/DW. and SQL
- Good to have: Python, PowerBI, Git
Similar Jobs
Artificial Intelligence • Big Data • Information Technology • Consulting
The Azure Data Engineer will build data pipelines and dataflows, implement security modules, and work with Azure services while supporting business requirements and data analysis.
Top Skills:
Azure Data FactoryAzure Data LakeC#C++ .NetMicrosoft Sql Server 2012Microsoft Sql Server 2016PowershellSql DatabaseSsisVisual Basic .Net
Fintech • Financial Services
The Data Engineer will design, develop, and maintain data pipelines and solutions, ensuring data quality and collaborating with cross-functional teams to meet analytics needs.
Top Skills:
Apache AirflowSparkAWSAzureDockerDynamoDBGCPGitHadoopJavaJenkinsKafkaKubernetesLuigiMongoDBMySQLNode.jsPostgresPythonTerraform
Fintech • Financial Services
Lead the design and development of scalable data pipelines using Ab-initio and Python, migrating workflows, ensuring data quality, and mentoring team members.
Top Skills:
Ab-InitioAWSAzureDeltaDockerGCPGitPandasParquetPysparkPythonSparkSQL
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
