Aligned Automation Jobs

Pyspark developer

Aligned Automation

Pyspark developer

Posted 6 Days Ago

Be an Early Applicant

In-Office

Pune, Maharashtra, IND

Senior level

In-Office

Pune, Maharashtra, IND

Senior level

Design, develop, and optimize large-scale PySpark ETL/ELT pipelines on Databricks. Implement Delta Lake lakehouse architectures, tune Spark jobs, write complex SQL, monitor production, enable CI/CD, mentor junior engineers, and collaborate with stakeholders on scalable data solutions.

The summary above was generated by AI

About the Job

About Aligned Automation

At Aligned Automation, we live by our "Better Together" philosophy to build a better world. As a strategic service provider to Fortune 500 companies, we help digitize enterprise operations and drive impactful business strategies. Our purpose goes beyond projects—we strive to deliver meaningful, sustainable change that shapes a more optimistic and equitable future.

Our culture is deeply rooted in our 4Cs—Care, Courage, Curiosity, and Collaboration—ensuring that each employee is empowered to grow, innovate, and thrive in an inclusive workplace.

Senior Data Engineer – PySpark

Experience: 7–8 Years
Location: Pune (Work from office)
Job Summary

We are seeking a highly skilled Senior PySpark Data Engineer with 7–8 years of experience in designing, developing, and optimizing large-scale data engineering solutions. The ideal candidate should have extensive experience with PySpark, Python, SQL, Databricks, cloud platforms, and modern data architectures. The role requires hands-on development, solution design, client interaction, and mentoring junior team members.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.
Build high-performance data pipelines to process large volumes of structured and semi-structured data.
Develop reusable frameworks for data ingestion, transformation, validation, and monitoring.
Optimize Spark jobs by tuning partitions, joins, caching, and memory configurations.
Design and implement Data Lake/Lakehouse architectures using Delta Lake.
Write complex SQL queries, stored procedures, CTEs, and window functions.
Collaborate with business stakeholders, architects, and data analysts to understand business requirements.
Participate in architecture discussions and recommend scalable data solutions.
Perform code reviews and enforce coding standards and best practices.
Monitor production pipelines, troubleshoot issues, and implement performance improvements.
Work with DevOps teams to implement CI/CD for data pipelines.
Mentor junior engineers and provide technical leadership.
Estimate effort, prepare technical documentation, and participate in Agile ceremonies.

Required Technical Skills
Programming

Python (Advanced)
PySpark (Advanced)
SQL (Advanced)

Big Data Technologies

Apache Spark
Spark SQL
Delta Lake
Parquet
iceberg
IOMETE

Databases

SQL Server
PostgreSQL
Azure SQL

Cloud Platforms (Any One)

Azure
AWS

Data Engineering Tools

Databricks
Apache Airflow

Version Control & DevOps

Git
Azure DevOps / GitHub
CI/CD Pipelines

Similar Jobs

Barclays

Engineer - PySpark/SQL

7 Days Ago

In-Office

Pune, Maharashtra, IND

Senior level

Fintech • Financial Services

Design, build and maintain data pipelines, data lakes and warehouses using PySpark, Snowflake and AWS analytics stack. Implement scalable ETL/ELT, ingestion, transformation, and reusable components; collaborate with data scientists and stakeholders; ensure data security, governance and operational excellence.

Top Skills: Ab InitioAlationAmazon AthenaAmazon S3Apache AirflowAWSAws GlueAws Lake FormationAws LambdaCsvDataframesDbtIcebergImmutaJSONNoSQLParquetPl/SqlPysparkPythonRddSnowflakeSnowflake TasksSparksqlSQL

Barclays

Engineer - PySpark/Snowflake

3 Days Ago

In-Office

Pune, Maharashtra, IND

Mid level

Fintech • Financial Services

Design, build and maintain data pipelines, data warehouses and data lakes using PySpark, Snowflake and AWS. Implement processing/analysis algorithms, collaborate with data scientists to deploy ML models, ensure security and data quality, and support team delivery and governance.

Top Skills: AirflowAWSData WarehouseDevops ToolsIcebergPysparkSnowflake

Mastercard

Software Engineer

An Hour Ago

Hybrid

Pune, Maharashtra, IND

Mid level

Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing

Develop, test, deploy, and maintain payment-focused software components. Troubleshoot and refactor code, adopt new languages and patterns, write tests and automation, follow secure coding standards, participate in agile delivery, document APIs, and mentor junior team members.

Top Skills: APIsBashBatch ServicesBig DataCheckmarxCi/CdCloudContainerizationCweDevOpsEvent-Driven ServicesGitflowGroovyInfrastructure As CodeJavaJavaScriptOwaspPerlPythonRubySei CertService-Oriented ArchitectureSonarSQLVirtualizationWeb ApplicationsZally

What you need to know about the Pune Tech Scene

Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.