Aligned Automation Jobs

Pyspark developer

Aligned Automation

Pyspark developer

Posted 3 Months Ago

Be an Early Applicant

In-Office

Pune, Maharashtra, IND

Senior level

In-Office

Pune, Maharashtra, IND

Senior level

Design, develop, and optimize large-scale PySpark ETL/ELT pipelines on Databricks. Implement Delta Lake lakehouse architectures, tune Spark jobs, write complex SQL, monitor production, enable CI/CD, mentor junior engineers, and collaborate with stakeholders on scalable data solutions.

The summary above was generated by AI

About the Job

About Aligned Automation

At Aligned Automation, we live by our "Better Together" philosophy to build a better world. As a strategic service provider to Fortune 500 companies, we help digitize enterprise operations and drive impactful business strategies. Our purpose goes beyond projects—we strive to deliver meaningful, sustainable change that shapes a more optimistic and equitable future.

Our culture is deeply rooted in our 4Cs—Care, Courage, Curiosity, and Collaboration—ensuring that each employee is empowered to grow, innovate, and thrive in an inclusive workplace.

Senior Data Engineer – PySpark

Experience: 7–8 Years
Location: Pune (Work from office)
Job Summary

We are seeking a highly skilled Senior PySpark Data Engineer with 7–8 years of experience in designing, developing, and optimizing large-scale data engineering solutions. The ideal candidate should have extensive experience with PySpark, Python, SQL, Databricks, cloud platforms, and modern data architectures. The role requires hands-on development, solution design, client interaction, and mentoring junior team members.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.
Build high-performance data pipelines to process large volumes of structured and semi-structured data.
Develop reusable frameworks for data ingestion, transformation, validation, and monitoring.
Optimize Spark jobs by tuning partitions, joins, caching, and memory configurations.
Design and implement Data Lake/Lakehouse architectures using Delta Lake.
Write complex SQL queries, stored procedures, CTEs, and window functions.
Collaborate with business stakeholders, architects, and data analysts to understand business requirements.
Participate in architecture discussions and recommend scalable data solutions.
Perform code reviews and enforce coding standards and best practices.
Monitor production pipelines, troubleshoot issues, and implement performance improvements.
Work with DevOps teams to implement CI/CD for data pipelines.
Mentor junior engineers and provide technical leadership.
Estimate effort, prepare technical documentation, and participate in Agile ceremonies.

Required Technical Skills
Programming

Python (Advanced)
PySpark (Advanced)
SQL (Advanced)

Big Data Technologies

Apache Spark
Spark SQL
Delta Lake
Parquet
iceberg
IOMETE

Databases

SQL Server
PostgreSQL
Azure SQL

Cloud Platforms (Any One)

Azure
AWS

Data Engineering Tools

Databricks
Apache Airflow

Version Control & DevOps

Git
Azure DevOps / GitHub
CI/CD Pipelines

Similar Jobs

Takeda

E&C Country Head, Takeda India & Sri Lanka

Yesterday

Hybrid

Expert/Leader

Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing

Leads Takeda India’s ethics and compliance program, translating global and regional priorities into country execution. Advises senior leadership on ethical decision-making, governance, risk management, and regulatory compliance. Oversees risk assessments, controls, training, communications, monitoring, investigations support, remediation, escalation, and local compliance committees. Leads and develops local E&C resources while promoting integrity, accountability, speak-up culture, and continuous improvement across the India organization.

Tufin

Assistant Controller

Yesterday

Remote or Hybrid

India

Mid level

Security • Cybersecurity

Own global revenue recognition under ASC 606 and accounting processes including commissions, COGS, and leases. Review tax provisions, returns, transfer pricing, and subsidiary filings; support external audits and cross-functional reporting. Lead accounting projects, process improvements, and AI-driven workflow automation. Requires a CA, Big 4 high-tech experience, strong US GAAP and revenue recognition knowledge, ERP experience, advanced Excel skills, and excellent communication.

Top Skills: Ai Automation ToolsAsc 340-40Asc 606ExcelMS OfficeNetSuiteUs Gaap

Ericsson

Head ASP Management

Yesterday

In-Office

Expert/Leader

Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)

The Head ASP Management is responsible for procurement of ASP services, vendor relationships, contractual compliance, quality delivery, and supplier performance evaluation while driving cost efficiency and improvements in service delivery processes.

Top Skills: AspMicrowaveOhsRan

What you need to know about the Pune Tech Scene

Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.