We are looking for a Data Engineer with skills in Python, SQL, and Azure technologies to develop Big Data applications. Responsibilities include working with Spark and Docker to handle large scale data processing and analytics.
This is a remote position.
- Must have - Python, SQL, Spark, Databricks, Azure Data Factory, Azure Synapse, Docker
- Good to have - Knowledge of Teradata, Airflow, AWS, GC
- experience in developing Big Data applications
- Experience in large-scale Big Data & Analytics applications
- Hands on experience in using technologies like Spark, Scala,
- Experience in Azure & Docker are added advantage.
- Strong problem-solving skills, data structures and algorithms
- Experience with distributed systems handling large amounts of data.
Requirements
- Immediate Joiner
- Remote location
- Data Engineer with 2 to 6 Yrs of experience
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design, build, and maintain cloud-native data pipelines and ETL/ELT workflows for high-volume trade lifecycle data. Integrate DTCC sources (RTTM, Settlement Web) into a Unified Trade Record, process ISO 15022 messages, manage trade exception databases, optimize SQL Server performance, and ensure data quality, lineage, governance, and observability while collaborating with business stakeholders.
Top Skills:
AWSDockerDtcc RttmDtcc Settlement WebEksEltETLFixFxmlIbm MqIndexingIso 15022KafkaKubernetesRdsRest ApiS3SQLSQL ServerStored ProceduresUnified Trade Record (Utr)
Edtech • Software
Lead a two-month project to assess the current data environment and design, build, and optimize a scalable cloud data warehouse. Deliver automated ETL/ELT pipelines, dimensional models, analytics-ready datasets, documentation, and handover while partnering with analysts and stakeholders for BI and reporting.
Top Skills:
Amazon RedshiftApache AirflowAPIsAWSAws GlueAzureAzure Data FactoryAzure SynapseDbtGitGoogle BigqueryGCPLookerPower BIPythonQuicksightRelational DatabasesSnowflakeSQLTableau
Other
Maintain and monitor Python-based web scraping configurations, detect and resolve scraping issues, coordinate with stakeholders, produce reports, and build ETL pipelines to ingest and transform data into the data lake.
Top Skills:
AirflowAzureData WarehousingDatalakeDbtETLGitNoSQLPythonSQLTerraformWeb Scraping
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
