Director Data Engineering

Requisition Number: 2349096
Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.
Optum Global Advantage is part of a global health care technology and business solutions ecosystem focused on enabling better experiences, solider operational outcomes, and innovation at scale. Through our delivery network, Optum has access to scalable, high-quality talent and infrastructure across geographies, enabling modern platform engineering, cloud transformation, and enterprise data innovation.
What makes us stand apart

• Impact on millions of people through constant innovation
• Ability to produce your life's best work through passion, engineering excellence, and relentless commitment
• A culture focused on creating better outcomes and healthier lives for those we serve

A Better Way to Care | A Better Way to Think | A Better Way to Succeed
Positions in this function design, engineer, modernize, and operate enterprise data platforms, data products, AI solutions, and reusable frameworks-including AI Builder and Agentic AI capabilities-that power analytics, intelligent automation, interoperability, and operational decisioning. They support short and long term business priorities by building secure, observable, scalable, and cost efficient data and AI ecosystems; developing platform and AI accelerators; and enabling trusted data and intelligence across ingestion, transformation, model development, agent orchestration, consumption, and governance layers.
Primary Responsibilities:

• Leadership and Management
  • Build, lead, and retain high-performing, diverse data engineering teams focused on enterprise-scale, mission-critical data platforms and products
  • Create a culture of technical excellence, accountability, ownership, experimentation, and continuous learning
  • Invest in the career development of engineers and technical leads through mentoring, coaching, and structured growth plans
  • Lead with a customer-first mindset, ensuring platform reliability, scalability, security, and performance for internal and external stakeholders
  • Define and govern enterprise scale architecture for data, AI, and Agentic AI solutions, enabling intelligent, autonomous, and secure decisioning across platforms and business workflows
• Platform Reimagination and Data Product Strategy
  • Lead the reimagination of legacy and on-prem data ecosystems into cloud-native, AI-enabled, framework-driven platforms
  • Define and evolve modern architecture patterns including Medallion Architecture, Lakehouse Architecture, and Data Mesh / Data Fabric and Agentic AI reference patterns for autonomous and intelligent workflows
  • Drive a data-and-AI-as-a-product strategy with clear ownership, SLAs, discoverability, reusable APIs, and well-defined data contracts
  • Establish enterprise reference architectures, standards, blueprints, and platform guardrails for data and Agentic AI across Azure, AWS, and GCP
• Data Frameworks, Orchestration, CI/CD, Security and Observability
  • Design and implement reusable Data Acquisition (DA) and Data Ingestion (DI) frameworks for batch, streaming, CDC, API-based, and database-driven ingestion and AI/GenAI-ready data pipelines
  • Build Kafka-driven acquisition patterns using event-driven architectures, pub/sub models, schema registry, exactly-once semantics, and idempotent processing
  • Create metadata-driven and configuration-based frameworks to accelerate onboarding, reduce development effort, and standardize engineering practices
  • Implement enterprise orchestration using Apache Airflow, Azure Data Factory, and GCP Composer for DAG-based, event-driven, and hybrid batch/streaming workflows
  • Establish end-to-end CI/CD, DevOps, DataOps, and DevSecOps practices, extending to MLOps and GenAI pipeline with automated testing, environment promotion, deployment automation, rollback, and monitoring
  • Build enterprise data and AI security frameworks covering RBAC/ABAC, encryption, masking, tokenization, row- and column-level security, and secure data sharing
  • Enable end-to-end data observability and lineage across data and AI pipelines from-source, ingestion, transformation, and consumption with quality checks, freshness checks, SLA monitoring, alerting, anomaly detection, and root cause analysis
• Platform Infrastructure, Reliability and Performance Engineering
  • Provide deep technical direction across multi-cloud platform infrastructure spanning Azure, AWS, GCP, Databricks, Snowflake, Spark, Hadoop, and distributed data systems, supporting data and AI driven workloads at scale
  • Design and implement High Availability and Disaster Recovery (HADR) strategies for Databricks and Snowflake, including replication, failover, and business continuity patterns
  • Drive performance tuning across Snowflake and Databricks, including warehouse sizing and right-sizing, dynamic clustering, clustering keys, query tuning, concurrency scaling, partitioning, caching, join optimization, and job parallelism
  • Optimize end-to-end workload performance, latency, throughput, resiliency, and cost-efficiency for both real-time and batch data platforms
• AI-Driven Engineering, Migration and Collaboration
  • Embed AI across the SDLC to accelerate engineering delivery through code generation, test automation, documentation support, migration accelerators, and engineering productivity tools
  • Lead large-scale on-prem to cloud migration programs using rehost, replatform, and refactor strategies while ensuring minimal downtime, data integrity, and performance optimization
  • Enable AI/ML-ready data platforms that support feature engineering, real-time pipelines, model data preparation, and modern MLOps patterns
  • Partner with Product, Architecture, Cybersecurity, Platform, Analytics, and AI/ML teams to define strategy, prioritize execution, and communicate complex technical decisions to senior leadership
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

We, at Core Data and Data Platforms (CDDs), are constantly growing and innovating to build solider enterprise data engineering capabilities and deploy them to solve complex healthcare and business problems. With the right mindset, platforms, and tools, we forge meaningful partnerships across the enterprise and help data work better for everyone.

• Being a specialized platform engineering team, we are always on the lookout for talent that can share our goals and deliver together

Data Engineering Function

• Define and evolve the enterprise data engineering strategy and roadmap, aligning platform capabilities with business priorities, AI adoption, and long term modernization goals
• Positions in this function build innovative and scalable solutions using structured and unstructured data across cloud-native and hybrid ecosystems
• Apply knowledge of distributed computing, data modeling, data warehousing, streaming, observability, governance, security, and advanced performance optimization to deliver enterprise-grade platforms
• Use technologies such as SQL, Python, PySpark, Scala, Databricks, Snowflake, Hadoop, Kafka, and cloud-native services to engineer reliable data products and platform services
• Work closely with Product, Architecture, Security, DevOps, Analytics, and AI/ML teams to align technical strategy to business outcomes
• Enable AI, GenAI, and advanced analytics use cases by ensuring data platforms are trusted, governed, scalable, and optimized for feature engineering, real time processing, and model consumption

Role - Director Data Engineering

• Lead the design, development, modernization, and operations of enterprise data platforms aligned to data products, analytics, AI, and interoperability capabilities
• Own platform cost governance and optimization, balancing performance, scalability, and reliability with financial stewardship across cloud and data platforms
• Manage strategic vendor and partner relationships, driving platform leverage, innovation, and delivery outcomes across cloud and data ecosystem providers
• Define and track engineering and platform success metrics, including reliability, adoption, performance, cost efficiency, and business impact
• Ensure platform compliance with enterprise security, privacy, regulatory, and data governance standards, embedding guardrails into engineering and AI workflows
• Build reusable platform services, modules, APIs, and framework accelerators for ingestion, acquisition, transformation, orchestration, security, lineage, observability, and performance engineering
• Create technology roadmaps focused on reusability, scalability, resilience, performance, security, cloud-native engineering, and AI-first delivery
• Collaborate with enterprise architects and platform leaders to evolve architecture standards, best practices, and engineering governance
• Present platform vision, modernization strategy, and technical trade-offs to senior leadership, delivery teams, and business stakeholders
  Implement CI/CD, DevOps, DataOps, MLOps, and reliability practices to deliver production-grade platform capabilities at scale
  Drive backlog prioritization, technical feasibility assessments, and framework adoption to accelerate development and migration timelines across teams
  Bring hands-on depth in Databricks, Snowflake, Azure, AWS, GCP, Spark, Kafka, data warehousing, and distributed systems, while leading engineers through architectural deep dives and code reviews

Required Qualifications:

• Bachelor's or master's degree in computer science, Engineering, Information Systems, or a related field
• 15+ years of progressive experience in data engineering, platform engineering, cloud modernization, and enterprise technology delivery including exposure to AI-enabled platform
• 15+ years of combined experience in building software, platforms, and enterprise data engineering solutions, including significant experience in leadership roles
• Hands on experience developing and scaling enterprise data and AI ready platforms, reusable frameworks, and data products in complex, regulated business environments
• Experience integrating AI, GenAI and Agentic AI into software engineering and migration workflows to improve speed, quality, and consistency
• Demonstrated experience leading global teams and driving large-scale transformation across data platforms and cloud ecosystems and AI ready architecture
• Proven expertise in platform modernization, on-prem to cloud migration, multi-cloud architecture, and large-scale distributed systems and AI-enabled platform transformation
• Proven solid command of performance engineering, data warehousing, real-time and batch processing, lineage, observability, governance and operational readiness for AI and GenAI workloads
• Proven excellent analytical, problem-solving, stakeholder management, and executive communication skills

Preferred Qualifications:

• Relevant certifications in Azure, AWS, GCP, Databricks, and Snowflake and AI /data platform certifications
• Experience in healthcare or other regulated industries

In addition to this, we highly value these traits!

• Being Accountable
• Believing in taking initiative and calculated risks
• Spending time to understand goals, priorities, and plans
• Committed to delivering results
• Dedicated to serving customers
• Innovative and creative, with a logical and methodical approach to problem solving
• Ability to relay technical and analytical insight to internal and external stakeholders through solid technical and functional depth

Technical Skills

• Core Data Platforms
  • Databricks, Delta Lake, Snowflake, Snowpark, Medallion Architecture, Lakehouse Architecture, Data Mesh / Data Fabric, Databricks HADR, Snowflake HADR, dynamic clustering, warehouse and query tuning
• Cloud and Big Data
  • Azure (ADF, ADLS, Synapse, Azure Databricks), AWS (S3, EMR, Glue, Lambda, Redshift), GCP (BigQuery, Dataflow, Pub/Sub, Composer), Spark, Hadoop ecosystem, distributed storage and compute
• Data Acquisition and Ingestion
  • Batch and streaming ingestion, CDC, metadata-driven ingestion, API-based ingestion, reusable connectors, schema enforcement, data contracts, data onboarding automation
• Streaming and Event Patterns
  • Kafka, Event Hubs, Pub/Sub, event-driven architecture, pub/sub, schema registry, exactly-once processing, idempotent design, real-time acquisition patterns
• Orchestration and CI/CD
  • Apache Airflow, Azure Data Factory, GCP Composer, DAG orchestration, automated testing, Jenkins, Azure DevOps, GitHub Actions, Git, deployment automation, environment promotion, rollback
• Security and Governance
  • RBAC / ABAC, encryption, masking, tokenization, row- and column-level security, secure data sharing, governance controls, DevSecOps integration
• Lineage and Observability
  • Data lineage, data quality rules, freshness checks, SLA / SLO monitoring, alerting, monitoring dashboards, anomaly detection, root cause analysis, observability platforms
• Performance Engineering
  • Snowflake warehouse sizing, query optimization, clustering keys, concurrency scaling, Databricks / Spark optimization, partitioning, caching, joins, job tuning, cost optimization
• Programming and APIs
  • Python, PySpark, Scala, SQL, Java, REST APIs, microservices, reusable services and platform APIs
• AI and Modern Engineering
  • AI-first mindset, AI in SDLC, GenAI tools, development and migration accelerators, MLOps, feature store readiness, automation-led engineering

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.