ScatterPie Analytics Private Limited
Sr. Data Engineer (Python | PySpark | FastAPI)
We are looking for a skilled Data Engineer with strong hands-on experience in Python, PySpark, and FastAPI. The ideal candidate should be proficient in building scalable data pipelines, working with cloud/data warehousing environments, and collaborating with cross-functional teams to deliver high-quality data solutions.
Design, develop, and maintain scalable data pipelines using Python and PySpark.
Build and manage REST APIs using FastAPI for data integration and consumption.
Work with SQL extensively for querying, data transformation, and performance optimization.
Implement and follow best practices in data warehousing, data modeling, and ETL/ELT processes.
Collaborate using GitHub or similar version control and collaboration tools.
Work with CI/CD pipelines for deploying data workflows and services.
Conduct data profiling and quality checks to ensure accuracy and reliability.
Gather business requirements, perform analysis, and create clear documentation.
Participate in design reviews and contribute to data process improvements and architecture decisions.
RequirementsRequired Skills & Experience
Strong proficiency in Python, PySpark, and SQL.
Hands-on experience building APIs with FastAPI.
Experience with GitHub or similar code collaboration/version control tools.
Good understanding of data warehousing concepts, data modeling, and best practices.
Exposure to CI/CD pipelines and modern DevOps practices.
Experience in data profiling, requirements documentation, and data process design.
Strong analytical and problem-solving abilities.
Excellent communication and documentation skills.
.png)
