IBS Software Logo

IBS Software

Data Engineer

Posted One Month Ago
In-Office or Remote
Hiring Remotely in India
Expert/Leader
In-Office or Remote
Hiring Remotely in India
Expert/Leader
Leads the design, development, and maintenance of scalable ETL/ELT pipelines and data products across distributed, multi-cloud environments. Defines data engineering architectures, resolves complex production issues, and designs efficient data solutions. Provides technical leadership and mentorship while collaborating with architects, engineers, and business stakeholders. Requires expertise in big data, AWS services, data warehousing, databases, data modeling, orchestration, streaming, security, observability, version control, containers, and modern development practices.
The summary above was generated by AI
Job Title Lead Data Engineer Location (s) Cochin Years of Experience 5-10yrs Job Description

Lead Data Engineer

What You’ll Do

  • Lead the design, development, and maintenance of scalable ETL/ELT pipelines and data products in multi-cloud, multi-region, distributed environments.
  • Define and apply appropriate data engineering patterns and architectures based on business and technical requirements.
  • Drive technical investigations and resolve complex data, operational, and production issues.
  • Design scalable, flexible, efficient, and supportable solutions using appropriate technologies and disciplined development practices.
  • Provide technical leadership, guidance, and mentorship to data engineering teams.
  • Collaborate with architects, engineering teams, and business stakeholders to deliver robust data solutions.

Key Skills

  • Data Engineering & Architecture: Strong understanding of data engineering patterns and modern architectures such as Data Mesh, Data Fabric, Data Lake, and Data Warehouse.
  • Big Data & Cloud: Proficiency in Amazon EMR, AWS Glue, Data Lake, and technologies for processing large datasets.
  • Programming: Strong proficiency in Python, Java, or Scala, with solid OOP expertise.
  • Data Warehousing: Strong knowledge of data warehousing solutions, preferably Snowflake.
  • ETL/ELT & Orchestration: Hands-on experience with ETL/ELT pipelines and data orchestration tools.
  • Databases: Strong expertise in relational and NoSQL databases.
  • Data Modelling: Experience designing efficient OLAP and OLTP data models.
  • Streaming: Knowledge of streaming data technologies and architectures.
  • Version Control: Experience with Git and modern development practices.
  • Containers & Orchestration: Understanding of Docker and Kubernetes.
  • Data Security: Knowledge of encryption, access control, data privacy, and compliance.
  • Monitoring & Logging: Experience implementing monitoring, logging, alerting, and observability for data pipelines.

Role & Responsibilities
  • Lead Data Engineer
    • What You’ll Do
    • Lead the design, development, and maintenance of scalable ETL/ELT pipelines and data products in multi-cloud, multi-region, distributed environments.
    • Define and apply appropriate data engineering patterns and architectures based on business and technical requirements.
    • Drive technical investigations and resolve complex data, operational, and production issues.
    • Design scalable, flexible, efficient, and supportable solutions using appropriate technologies and disciplined development practices.
    • Provide technical leadership, guidance, and mentorship to data engineering teams.
    • Collaborate with architects, engineering teams, and business stakeholders to deliver robust data solutions.
    • Key Skills
    • Data Engineering & Architecture: Strong understanding of data engineering patterns and modern architectures such as Data Mesh, Data Fabric, Data Lake, and Data Warehouse.
    • Big Data & Cloud: Proficiency in Amazon EMR, AWS Glue, Data Lake, and technologies for processing large datasets.
    • Programming: Strong proficiency in Python, Java, or Scala, with solid OOP expertise.
    • Data Warehousing: Strong knowledge of data warehousing solutions, preferably Snowflake.
    • ETL/ELT & Orchestration: Hands-on experience with ETL/ELT pipelines and data orchestration tools.
    • Databases: Strong expertise in relational and NoSQL databases.
    • Data Modelling: Experience designing efficient OLAP and OLTP data models.
    • Streaming: Knowledge of streaming data technologies and architectures.
    • Version Control: Experience with Git and modern development practices.
    • Containers & Orchestration: Understanding of Docker and Kubernetes.
    • Data Security: Knowledge of encryption, access control, data privacy, and compliance.
    • Monitoring & Logging: Experience implementing monitoring, logging, alerting, and observability for data pipelines.
    •  
Technical Qualifications Python + AWS EMR + AWS Glue + Snowflake + ETL/ELT + Data Engineering + Data Architecture + Big Data + SQL

Similar Jobs

12 Minutes Ago
In-Office or Remote
Karnataka, IND
Senior level
Senior level
Retail • Manufacturing
Own Azure Data Factory pipelines, Salesforce and SAP integrations, KPI logic, dashboards, semantic definitions, validation rules, and reconciliations for trade promotion and sales data. Investigate discrepancies, monitor feeds, support platform changes, review AI-generated financial figures, and improve data quality. Serve as the India-hours escalation point while collaborating globally, guiding junior colleagues, and partnering with commercial, finance, IT, and vendor teams.
Top Skills: Azure Data FactoryCrm AnalyticsPbcsS/4HanaSalesforceSalesforce Consumer Goods CloudSalesforce Trade Promotion ManagementSAPSQL
2 Hours Ago
In-Office or Remote
India
Senior level
Senior level
Aerospace • Security • Energy • Industrial
Designs and maintains scalable batch and streaming data pipelines in Databricks using Spark, PySpark, SQL, Delta Lake, and medallion architecture. Builds data models, automates workflows, develops Power BI reports, and provides statistical analysis, KPI tracking, financial and operational reporting, and business insights. The role coordinates with engineering, product operations, finance, and program leaders, supports automation and AI agent initiatives, mentors junior colleagues, and helps improve program performance and business efficiency.
Top Skills: Ai AgentsSparkAuto LoaderDatabricksDatabricks JobsDatabricks WorkflowsDelta LakeLakeflow Declarative PipelinesMedallion ArchitectureExcelMicrosoft OutlookMicrosoft PowerpointMicrosoft ProjectPower BIPysparkPythonSpark SqlSQL
Yesterday
Remote
India
Senior level
Senior level
Aerospace
Designs and maintains scalable batch and streaming data pipelines in Databricks using Spark, PySpark, SQL, Delta Lake, and medallion architecture. Builds data models, automates workflows, develops Power BI reports, and delivers operational, financial, cost, quality, and schedule analytics. Partners with engineering, product operations, project management, and business leaders to support planning and performance decisions. The role also mentors junior colleagues, oversees work, performs statistical analysis, and drives process automation and efficiency improvements.
Top Skills: Ai AgentsSparkAPIsAuto LoaderAutomated TestingDatabricksDatabricks JobsDatabricks WorkflowsDelta LakeLakeflow Declarative PipelinesMedallion ArchitectureExcelMicrosoft OutlookMicrosoft PowerpointMicrosoft ProjectPower BIPysparkPythonSpark SqlSQL

What you need to know about the Pune Tech Scene

Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account