Design, build, and optimize large-scale ETL pipelines and distributed data applications (batch and streaming) using Spark and Scala. Ensure data quality and performance, deploy and manage workloads in cloud/container environments, troubleshoot production issues, and collaborate with product, engineering, and analytics teams to deliver scalable AdTech data solutions.
We are looking for Senior Data Engineers to join our AdTech team and build scalable, high-performance data platforms that power advertising insights and analytics. The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Spark and Scala.
You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
- Build and optimize distributed data applications using Spark and Scala.
- Develop reliable, high-performance data pipelines for batch and streaming workloads.
- Work with large datasets to ensure data quality, consistency, and performance.
- Collaborate with engineering, product, and analytics teams to deliver robust data solutions.
- Optimize data workflows for scalability, reliability, and cost efficiency.
- Deploy and manage data workloads in cloud and containerized environments.
- Troubleshoot production issues and continuously improve platform performance.
Requirements
Candidates who demonstrate:
- 5+ years of experience in Data Engineering or Big Data Engineering.
- Strong hands-on experience with Apache Spark and Scala.
- Experience building and maintaining ETL pipelines.
- Familiarity with Google Cloud Storage (GCS).
- Experience with Kubernetes (K8s).
- Strong SQL skills and understanding of distributed data processing.
- Excellent debugging, problem-solving, and performance optimization skills.
- Strong communication and collaboration skills.
Good to Have
- Experience with AWS and cloud-native data services.
- Familiarity with streaming technologies such as Kafka.
- Experience working on large-scale data platforms or AdTech systems.
- Exposure to orchestration tools such as Airflow.
Benefits
- Best-in-class salary: We hire strong talent and compensate accordingly.
- Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners.
- Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development.
- High-impact work: Build AI-first systems and products used at scale by global clients.
About Us
Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.
Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.
Similar Jobs
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Leads sales performance, pipeline, account strategy, complex deal execution, forecasting, negotiations, and contract delivery across defense and aerospace markets. Manages and coaches sales teams, builds executive customer relationships with government and defense stakeholders, coordinates global cross-functional teams, and drives regional growth across India, the Middle East, and Turkey. Requires defense or aerospace sales expertise, strong commercial judgment, customer networks, and experience managing multinational portfolios.
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Provides technical leadership for display graphics application software and platform services. Develops software architecture, designs, and embedded C++ implementations; evaluates new technologies; optimizes GPU, CPU, and memory performance; supports integration and field investigations; participates in design reviews; creates software tests; and mentors junior engineers.
Top Skills:
C++CpuDevice DriversDo-178CEmbedded C++FirmwareGpuLean/AgileMulticore ProcessingMultithreading
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Provides administrative, documentation, coordination, and office operations support for an engineering department. Responsibilities include managing schedules, records, reports, meetings, engineering change documentation, procurement coordination, NPI logistics, KPI tracking, equipment records, onboarding, training, and cross-functional communication. The role also supports AI- and data-driven manufacturing enablement using MES, equipment, yield, analytics, and machine learning data.
Top Skills:
AsanaData AnalyticsDocument Management SystemsIndustry 4.0JIRAMachine LearningMesExcelMicrosoft OutlookMicrosoft PowerpointMicrosoft TeamsMicrosoft WordMs ProjectPythonSharepointSQL
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
