The Data Engineer role involves analyzing raw data, building data systems, conducting complex analyses, and enhancing data quality by collaborating with data scientists and architects.
This is a remote position.
Responsibilities:
Requirements
Requirements and skills:
Similar Jobs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead data engineering architecture and execution for large-scale, low-latency analytics. Own technical direction, ensure operational data quality, design streaming and batch pipelines, mentor engineers, coordinate cross-functional teams, and deliver scalable solutions using Spark, Airflow, and AWS or equivalent data platforms.
Top Skills:
AirflowAthenaColumn StoresDatabricksDatabricks ApisEmrFlinkHiveKafkaKappa ArchitectureLambda ArchitectureMaster Data Management (Mdm)Microservices ArchitectureRedshiftSparkSQLStreaming Pipelines
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Own and enhance scalable batch and incremental data pipelines supporting reliability metrics, incident reporting, PIRs, and SLOs. Design data models and marts using Databricks, dbt, and modern table formats; implement automated data quality, anomaly detection, and alerting; optimize SQL, Spark, and dbt performance and costs; enable Tableau and executive reporting; maintain engineering standards, CI/CD, documentation, and mentorship.
Top Skills:
AirflowApache IcebergSparkAWSBitbucket PipelinesDatabricksDbtDelta LakeGCPJIRAJira Service ManagementOpsgeniePythonSQLTableau
%20(1).png)
Fintech • Financial Services
Leads data engineering initiatives involving ETL/ELT pipelines, database optimization, orchestration, big data technologies, data modeling, BI performance, and data quality monitoring. Requires strong Scala, Python, SQL, and shell scripting skills, plus payments-domain expertise covering transaction processing, settlements, disputes, reconciliation, gateways, acquiring, issuing, and PCI-DSS compliance. The role also involves cross-functional communication, planning, facilitation, negotiation, and collaboration with senior stakeholders and clients.
Top Skills:
AirflowBusiness IntelligenceData ModelingData Quality FrameworksEltETLHadoopJavaLuigiMonitoringNoSQLPci-DssPrefectPythonScalaShell ScriptingSparkSQLTest Automation
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
