Designs and implements data ingestion and data lake solutions using Big Data and open-source technologies. Develops cost-efficient, performant cloud data pipelines; integrates multiple data sources; supports analytics, reporting, and data science teams; optimizes infrastructure and query performance; and applies data mining techniques and ingestion APIs to populate data lakes.
Role Big Data Lead Responsibilities Design & Implement Data ingestion and Data lakes-based solutions using Big Data Technologies. The Tech Lead should be highly proficient in the use of Big Data / Open-Source Technologies and standard techniques of Data Integration, Data Manipulation. Should be able to design and develop cost efficient and performant data pipelines in the cloud platform Create data environment to support our data analytics, reporting and data science teams Experience with integration of data from multiple data sources Knowledge of various Data Pipeline techniques and frameworks Performance optimization - need to monitor the complete process and apply necessary infrastructure changes to speed up the query execution. Efficient data ingestion - Discovering patterns in data sets with data mining techniques and using different data ingestion APIs and inject data into the data lake as per need
Hexaware Technologies Pune, Mahārāshtra, IND Office
North Block, Plot No. 19, Rajiv Gandhi InfoTech Park, MIDC - SEZ, Phase 3, Hinjawadi, Pune, Maharastra, India, 411057
Similar Jobs
Information Technology • Consulting
Build and maintain data pipelines using dbt, Python, and Airflow; develop reliable data models and analytics datasets; create Power BI dashboards, semantic models, and DAX measures; investigate and validate data; troubleshoot issues across the data lifecycle; translate business needs into reporting solutions; and collaborate with business, analytics, and engineering teams on production-ready solutions.
Top Skills:
Apache AirflowDaxDbtPower BIPythonSnowflakeSQL
Information Technology • Consulting
Lead the design and development of scalable Microsoft Fabric data solutions, including end-to-end pipelines, Spark notebooks, ETL/ELT processes, Salesforce ingestion, data modeling, semantic models, and BI reporting. Establish config-driven frameworks, ensure CI/CD and documentation standards, and own deliverables with minimal supervision.
Top Skills:
SparkAzure Data FactoryAzure Synapse AnalyticsCi/CdDatabricksDelta LakeMicrosoft FabricPythonSalesforceSQL
Information Technology • Consulting
Build scalable data pipelines for large datasets using Microsoft Fabric, Azure data services, advanced SQL, and Python/Spark. Develop data transformations and apply data modeling principles such as star schemas. Experience working with financial services data is desirable.
Top Skills:
SparkAzure Data FactoryAzure Data LakeAzure SynapseDataflowsLakehouseMicrosoft FabricPythonSQLWarehouse
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
