Data Enigneer-III

Role: SDE III Data Engineer
Location: Remote
Employment Type: Contractor (6 months)

Application Stack 

·       Python/ PySpark

·       Apache AirFlow

·       Snowflake

·       SQL Database

·       MLOps

·       Snowpark

·       Azure/AWS Functions

·       Blob Storage

·       XML, JSON

·       GitHub

Essentials for Being Successful in This Role

·       Minimum 5 years of experience in software development and system design 

·       Minimum 2 years of experience as a Data Engineer focused on data modelling, governance, and platform administration.

·       Expertise in Snowflake or similar MPP databases.

·       Strong SQL skills.

·       Experience with multiple database technologies (MSSQL, PostgreSQL, OLAP, OLTP).

·       Familiarity with ETL/ELT tools and methodologies.

·       Experience with database health monitoring, stored procedures, and schema design.

·       Practical understanding of data governance principles.

·       Experience with cloud environments (SaaS, PaaS, IaaS).

·       Strong communication skills, both written and verbal.

·       Experience in technical design and documentation.

·       Proficiency with source control systems (Git, Gitflow).

·       Experience with Agentic AI (Claude, GPT, etc.).  

Desirables (Nice to Have) 

·       Familiarity with Generative AI and Spec-Driven Development.

·       Experience with test automation.

·       Experience with Kafka, Azure Event Hubs, Azure Service Bus.

·       Knowledge of CI/CD practices.

·       Ability to work independently and within Agile teams.

·       Experience with Azure cloud services.

·       Exposure to Master Data Management (MDM).

·       Python skills for data transformation and pipeline automation.

·       Understanding data mesh principles.

·       Strong prioritization and multitasking abilities.

Roles & Responsibilities 

·       Designing, building, testing, and maintaining scalable data models using Apache Airflow and Python.

·       Administering Snowflake environments, monitoring system health and performance.

·       Managing and optimizing Snowflake virtual warehouses for efficiency and cost-effectiveness.

·       Managing the MLOps workflow through Snowflake registrations and Snowpark.

·       Promoting self-service and decentralized data ownership across teams.

·       Enforcing data governance policies to ensure standardization, quality, and compliance.

·       Providing technical assistance and cross-training to team members.

·       Participating in design sessions and creating technical design documents and diagrams.

·       Conducting code reviews and offering constructive feedback.

·       Mentoring engineers to support team growth and development.

·       Collaborating with senior engineers in the Architecture Review Board.

·       Ensuring delivery quality and supporting production services.