As a Senior Data Engineer, you will design and manage databases and data pipelines in a cloud environment, ensuring data reliability and performance. You will collaborate with teams to develop solutions while maintaining focus on continuous improvement.
OrganizationAdvertising End Date: 29/06/2025
At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things.Job Title: Senior Data Engineer Location: BangaloreBusiness & Team:The Business Banking Technology Domain works in an Agile methodology with our business banking business to plan, prioritise and deliver on high value technology objectives with key results that meet our regulatory obligations and protect the community.You will work within the VRM Crew that is working on initiatives such as Gen AI based cash flow coach to provide relevant data to our regulatorsImpact & Contribution:As a Senior Data Engineer and database engineer you will be creating and managing the cloud databases and data pipelines that underpin our decoupled cloud architecture and API first approach. You have proven expertise in database design, data ingestion, transformation, data writing, scheduling and query management within a cloud environment.You will have proven experience and expertise in working with AWS Cloud Infrastructure Engineers, Software/API Developers to design, develop, deploy and operate data services and solutions that underpin a cloud ecosystem. You will take ownership and accountability of functional and non-functional design and work within a team of Engineers to create innovative solutions that unlock value and modernise technology designs.You will role model continuous improvement mindset in the team, and in your project interactions, by taking technical ownership of key assets, including roadmaps and technical direction of data services running on our AWS environments. Roles & Responsibilities:
- Can design and implement databases for data integration in the enterprise
- Can performance tune applications from a database code and design perspective
- Can automate data ingestion and transformation processes using scheduling tools.
Monitor and troubleshoot data pipelines to ensure reliability and performance. - Can design application logical database requirements and implement physical solutions
- Can collaborate with business and technical teams in order to design and build critical databases and data pipelines
- Minimum 7 to 10 years of experience
- AWS Data products such as AWS Glue and AWS EMR
- Oracle and AWS Aurora RDS such as PostgreSQL
- AWS S3 ingestion, transformation and writing to databases
- Proficiency in programming languages like Python, Scala or Java for developing data ingestion and transformation scripts.
- Strong knowledge of SQL for writing, optimizing, and debugging queries.
- Familiarity with database design, indexing, and normalization principles.
Understanding of data formats (JSON, CSV, XML) and techniques for converting between them. Ability to handle data validation, cleaning, and transformation. - Proficiency in automation tools and scripting (e.g., bash scripting, cron jobs) for scheduling and monitoring data processes.
- Experience with version control systems (e.g., Git) for managing code and collaboration.
- Bachelor’s degree in engineering in Computer Science/Information Technology.
If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application. We’re keen to support you with the next step in your career.
We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.
Top Skills
Aws Aurora Rds
Aws Emr
Aws Glue
Aws S3
Bash
Java
Oracle
Postgres
Python
Scala
SQL
Similar Jobs
eCommerce • Fintech • Payments
Design and implement scalable data pipelines, collaborate with data teams, manage data architectures, and ensure data quality for AI/ML initiatives.
Top Skills:
AirflowAws BedrockAws GlueAzure OpenaiCassandraDatabricksDockerFlinkGcp BigqueryGoogle VertexaiJavaKafkaKubernetesMongoDBPythonPyTorchScalaScikit-LearnSnowflakeSnowflake CortexSparkSQLTensorFlow
eCommerce • Fintech • Payments
Design, implement, and maintain data pipelines and architectures for AI/ML use cases, collaborating with data scientists and engineers to optimize data quality and access.
Top Skills:
AirflowAws GlueCassandraDatabricksDockerFlinkGcp BigqueryJavaKafkaKubernetesMongoDBPythonPyTorchScalaScikit-LearnSnowflakeSparkSQLTensorFlow
Fintech • Financial Services
The role involves integrating AI models into business processes, developing APIs, managing data access patterns, and deploying scalable AI solutions on cloud platforms while ensuring collaboration and compliance.
Top Skills:
Apache KafkaAWSDockerEmrKubernetesPythonRRedshiftSage MakerSnowflakeSpark EtlTeradata
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
