The Advanced Data Engineer designs and manages data architecture and processes, develops ETL systems, collaborates with analysts, enforces data governance, and ensures data quality and security.
Advanced Data Engineer
Location: Bangalore (Hybrid)
Join a team that is elevating our strategy to drive advanced analytics and visualization tools across the Value Engineering COE. In this role, Advanced Data Engineer , you will design, implement, and manage the data architecture, systems, and processes to effectively collect, store, process and analyze high volume, high dimensional data to provide strategic insight into complex business problems. This will involve creating and maintaining scalable, efficient, and secure data pipelines, data warehouses, and data lakes. You need to ensure consistency in data quality and availability for analysis and reporting including compliance with data governance and security standards.
ResponsibilitiesDuties and Responsibilities
- Work in complex data science and analytics projects in support of the Value Engineering organization.
- Understand and translate business requirements into technical solutions, leveraging available and new technologies as required.
- Design and implement data models and schemas to support analytical and reporting requirements.
- Collaborate with data scientists and data analysts to define and structure data for effective analysis and reporting.
- Develop and maintain ETL (Extract, Transform, Load) processes.
- Administer, optimize, and manage databases, data warehouses, and data lakes to ensure performance, reliability, and scalability.
- Enforce data governance policies, standards, and best practices to maintain data quality, privacy, and security.
- Create and maintain comprehensive documentation for data architecture, processes, and systems.
- Troubleshoot and resolve data-related problems and optimize system performance.
- Partner with IT support team on production processes, continuous improvement, and production deployments.
YOU MUST HAVE
- 7 plus years of relevant experience in Data Engineering, ETL Development, Database Administration.
- Experience in Azure Data Factory, Azure DW, Databricks, and APIs
- Expertise in querying and programming languages, such as Python, SQL, PySpark
- Experience with Structured data. Knowledge of unstructured data is not needed but valued.
- Experience in Snowflake
- Knowledge of Agile development methodology
WE VALUE
- Knowledge of databases, data warehouse platforms (Snowflake) and Cloud based tools.
- Understanding of Data Mining concepts & applications such as Classification or Predictive algorithms.
- Experience in using data integration tools for ETL processes.
- Knowledge of Data Modelling techniques including schema design for relational databases
- Experience with Python packages such as: polars, scikit – especially those packages linked to data processing & transformation.
- Ability to develop and communicate technical vision for projects and initiatives that can be understood by customers and management.
Similar Jobs
Aerospace • Security • Energy • Industrial
The Advanced Data Engineer is responsible for building and maintaining data pipelines, generating reports, and automating workflows to support analytics and operational efficiency in engineering and finance contexts.
Top Skills:
DatabricksExcelPower BIPythonSQLVBA
Aerospace • Security • Energy • Industrial
Design and develop Hyperion and OneStream applications, implement data rules, collaborate with users for technical specifications, ensuring data quality and advanced analytics.
Top Skills:
Business RulesCalc ScriptEssbaseExcelFdmeeHyperion PlanningMdxOnestream
Edtech • Information Technology • Other
The Senior Data Engineer will optimize the data platform, build scalable data architectures, and lead engineering projects while collaborating across teams.
Top Skills:
AirflowDbtKubernetesMeltanoSnowflakeSQLTerraform
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
