JPMorganChase Logo

JPMorganChase

Senior Lead Software Engineer - Big Data

Posted 2 Days Ago
Be an Early Applicant
Hybrid
Bengaluru, Bengaluru Urban, Karnataka
Senior level
Hybrid
Bengaluru, Bengaluru Urban, Karnataka
Senior level
Leads engineering teams responsible for enterprise data platforms, warehouses, lakes, and production data solutions. Designs Databricks-based architectures, reusable ETL frameworks, batch and streaming pipelines, and secure AWS solutions. Oversees data modeling, governance, reporting accuracy, performance, cost, reliability, and incident resolution. Mentors engineers, sets technical direction, enforces software engineering practices, and drives responsible AI-assisted development, automation, testing, and operational improvements across the SDLC.
The summary above was generated by AI

Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products.

As a Senior Lead Software Engineer at JPMorganChase within the Commercial and Investment Bank and C360 Technology team,  you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. You will set the technical direction, build reusable frameworks and guide a team of engineers. You  also have a solid understanding of the broader software and application landscape that our data solutions integrate with.

Job responsibilities

  • Take full responsibility for the data platforms, including architecture, roadmap, reliability, cost, operational health and production support.
  • Mentor and guide engineers. Set technical direction, clear blockers quickly, delegate effectively and answer for the team's deliverables and code quality.
  • Architect and build data warehouse and data lake solutions on Databricks. Apply sound data modeling, history tracking and data organization practices so the data is well structured and ready for analytics.
  • Create reusable, configurable ETL and processing frameworks covering ingestion, transformation, data quality, reconciliation and auditability, and drive their adoption across teams.
  • Define and govern standard approaches for bringing data in from databases, files, APIs and event sources, covering both full and incremental loads and ship high-volume batch and streaming pipelines with Spark/PySpark and Kafka/Kinesis. Make sure they're reliable, resilient to failure and meet agreed service levels.
  • Tune performance and cost across Spark, Databricks, SQL engines and NoSQL stores. Design secure, resilient solutions on AWS using services such as S3, Glue, EMR, Lambda, Redshift and Athena.
  • Govern the models, aggregates and data marts behind reporting so that business and regulatory reports are accurate, reconciled, timely and fast.
  • Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test acceleration, release readiness, incident/root-cause analysis), while establishing measurable validation standards (secure coding, peer review, automated testing) and promoting reuse of proven patterns and automation within the SDLC/TLM toolchain.
  • Applies knowledge of tools within the Software Development Life Cycle toolchain, including approved AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
  • Enforce modular design, code reviews, CI/CD, automated testing, monitoring and Git-based workflows. Work with application, API and UI teams, using your knowledge of the full software stack to shape integration decisions.
  • Promote reuse-first practices that use enterprise-authorized AI capabilities for design analysis, test generation, control validation and documentation. Validate all outputs, respect data sensitivity, and keep work traceable and auditable.


    Required qualifications, capabilities, and skills

  • Formal training or certification on software engineering concepts and 5+ years applied experience
  • Experience in data engineering with a track record of leading teams, mentoring engineers, removing obstacles and being accountable for delivery, platform stability and incident resolution.
  • Expertise in designing and developing data warehouses and data lakes on Databricks, with strong data modeling skills. Familiarity with Snowflake and comparable platforms such as Redshift, BigQuery or Azure Synapse.
  • Exceptional SQL skills, including complex queries, optimization and analytical problem solving on large datasets with solid grounding in reporting: business-friendly data layers, BI tools (e.g., Tableau, Power BI, QuickSight), dashboard performance, and accuracy controls.
  • Outstanding production-grade Python and PySpark, with clean, efficient, testable and maintainable code.
  • Mastery of data ingestion and processing designs, with a history of building enterprise ETL frameworks and running batch and streaming workloads.
  • Broad knowledge of relational databases (e.g., PostgreSQL, Oracle), NoSQL stores (e.g., DynamoDB, MongoDB, Cassandra) and AWS data services. You can tune performance across these technologies and choose the right one for each use case.
  • Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security
  • Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching senior engineers/leads on compliant usage patterns and controls.
  • A well-rounded understanding of software engineering and application development, including the SDLC, design principles, microservices, APIs (e.g., Spring Boot) and front-end concepts. You also troubleshoot well and know CI/CD and GitHub/Bitbucket workflows.
  • Experience using enterprise-authorized AI capabilities in engineering work. You review and validate AI-assisted outputs before using them, escalate when unsure, and follow data handling requirements.


    Preferred qualifications, capabilities, and skills

  • Infrastructure provisioning in AWS using Infrastructure as Code (e.g., Terraform, AWS CloudFormation).
  • Databricks and/or AWS certifications.
  • Hands-on experience with Snowflake, or with migrating data between platforms.
  • Experience with orchestration tools (e.g., Airflow) and with data governance and lineage tools.
  • Experience with containerization (Docker/Kubernetes) and in financial services or other regulated environments.

Similar Jobs

One Month Ago
Hybrid
Senior level
Senior level
Financial Services
Designs, develops, operates, and scales secure data platforms, backend systems, pipelines, and distributed services. Provides technical leadership, reviews code, influences architecture, and serves as a subject matter expert in data processing, storage, and reliability. Leads enterprise adoption of AI-assisted engineering and SDLC automation by establishing governance, security controls, quality gates, and measurable outcomes. Collaborates across global teams and promotes scalable engineering practices and technologies.
Top Skills: AIAirflowAWSCi/CdClaudeData LakeDatabricksEcsEksEmrJavaKafkaLambdaMachine LearningMcpPythonSpark
14 Hours Ago
In-Office
Mid level
Mid level
Artificial Intelligence • Cloud • Information Technology • Consulting
Designs and develops cloud application features and modules, creates and automates test plans, deploys code, troubleshoots application and deployment issues, and addresses system administration or network concerns. The role requires programming expertise, cloud-native development experience, DevOps practices, Kubernetes orchestration, and a bachelor's degree in a related field.
Top Skills: Cloud Managed ServicesCloud-Native ApplicationsContinuous DeploymentContinuous IntegrationDevOpsGoJavaJavaScriptKubernetesNext-Generation DatabasesPython
14 Hours Ago
In-Office
Mid level
Mid level
Artificial Intelligence • Cloud • Information Technology • Consulting
Develop and validate private cloud infrastructure across compute, storage, networking, virtualization, and operating systems. Build Python, REST API, scripting, and test automation solutions; identify automation opportunities; and apply AI tools to testing, troubleshooting, log analysis, and documentation. Test cloud integrations, upgrades, resiliency, and recovery, investigate failures, support release readiness, and collaborate with engineering, QA, architecture, and program teams.
Top Skills: Ci/CdContainersDnsGenerative AiGitKubernetesLinuxLlmsPythonRest ApisTcp/IpTest Automation FrameworksVirtualizationVlansWindows

What you need to know about the Pune Tech Scene

Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account