Job Description Summary
Under moderate supervision, designs, develops, tests, debugs and implements moderately complex operating systems components, software tools, and utilities required for the operation, maintenance, and control of computer systems. Makes moderately complex modifications to existing software to fit specialized needs and configurations, and maintains program libraries and technical documentation. Assists in planning, scheduling and implementation. Initiates corrective action to stay on schedule.
ESSENTIAL DUTIES AND RESPONSIBILITIES
· Participate and collaborate with team members to review the use cases for data pipelines.
· Design and develop highly optimized and reliable enterprise level data pipelines to move data to data warehouse in real-time.
· Review and analyze existing business process, identify any gaps, and recommend improvements.
· Design and develop modern real-time solutions with Scala, Spark, Azure Databricks.
· Partner with business users and data scientists to build and deploy machine learning models.
· Improve data governance and operational excellence.
· Develop data expertise and take ownership of data quality for the newly developed pipelines.
· Adhere to company coding standards and best practices.
· Conduct code reviews and create technical solution document.
MINIMUM QUALIFICATIONS
· 8+ overall experience
· 5+ years of building data pipelines for structured and un-structured data.
· 2+ years of experience of azure data pipeline development.
· 3 + years of experience with Hadoop, Azure Databricks, Stream Analytics, Eventhub, Kafka.
· 3+ years of experience of relational and dimensional data modeling with SQL Server and data warehouse.
· 3+ years of experience with SQL Server, T-SQL, stored procedure development and database optimization.
· 3+ years of experience with MS BI stack (SSIS/SSRS/SSAS, Power BI).
· 2 + years of experience with Azure data platform (Azure data factory, Azure data lake, Azure SQL DB, Azure DW/Synapse).
· 2 + years of experience with Azure ML, Python, Scala
· 3+ years of experience with big data technologies.
· 2 + years of experience with Power BI, Azure Analysis Services
· Experience with working with business stakeholders, requirements & use case analysis.
· Strong communication and collaboration skills with creative problem-solving skills.
PREFERRED QUALIFICATIONS
· Bachelor's degree in Computer Science or equivalent work experience.
· Experience with Agile/Scrum methodology.
· Experience with tax and accounting domain a plus.
· Azure Data Engineer certification a plus.
