We are looking for a Data Engineer with skills in Python, SQL, and Azure technologies to develop Big Data applications. Responsibilities include working with Spark and Docker to handle large scale data processing and analytics.
This is a remote position.
- Must have - Python, SQL, Spark, Databricks, Azure Data Factory, Azure Synapse, Docker
- Good to have - Knowledge of Teradata, Airflow, AWS, GC
- experience in developing Big Data applications
- Experience in large-scale Big Data & Analytics applications
- Hands on experience in using technologies like Spark, Scala,
- Experience in Azure & Docker are added advantage.
- Strong problem-solving skills, data structures and algorithms
- Experience with distributed systems handling large amounts of data.
Requirements
- Immediate Joiner
- Remote location
- Data Engineer with 2 to 6 Yrs of experience
Similar Jobs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead data engineering architecture and execution for large-scale, low-latency analytics. Own technical direction, ensure operational data quality, design streaming and batch pipelines, mentor engineers, coordinate cross-functional teams, and deliver scalable solutions using Spark, Airflow, and AWS or equivalent data platforms.
Top Skills:
AirflowAthenaColumn StoresDatabricksDatabricks ApisEmrFlinkHiveKafkaKappa ArchitectureLambda ArchitectureMaster Data Management (Mdm)Microservices ArchitectureRedshiftSparkSQLStreaming Pipelines
HR Tech • Information Technology • Professional Services
Design, implement, and optimize graph data solutions for a supplier chain project. Responsibilities include graph modeling with LPG and RDF, Neo4j database development and query optimization, GCP data processing using BigQuery, GCS, and Dataproc, and integrating graph data with visualization tools. The role also involves schema and data design, SQL-based relational database work, performance tuning, and potentially ETL/ELT, streaming, API, and distributed systems development.
Top Skills:
Api DevelopmentBigQueryCypherDataprocDistributed SystemsEtl/EltGcp Data FusionGoogle Cloud Platform (Gcp)Google Cloud Storage (Gcs)KafkaKinesisLabeled Property Graphs (Lpg)Neo DashNeo4JNeo4J AuraPower BIPysparkResource Description Framework (Rdf)SparksqlSQLTableau
Cloud • Information Technology • Software • Cybersecurity
Build and operate a production data pipeline ingesting Indian MCA registration data. Responsibilities include raw-first S3 storage, resilient scraping and parsing, snapshot and diff processing, granular change events, scheduled batch jobs, alert digests, monitoring, anomaly detection, reference-table maintenance, and historical backfills. The role requires strong Python, PostgreSQL, testing, operational reliability, and production experience with uncooperative data sources.
Top Skills:
Amazon CloudwatchAmazon S3Amazon SesAmazon SqsBeautifulsoupCeleryDkimLxmlPandasPdfplumberPostgresPython
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
