We're looking for a dynamic, people oriented, programming language agnostic, and highly analytical person to join our development team in our Pune office. You would be working with a team of 4-6 highly skilled engineers to scale Pattern’s Data Platform.
About Team: Our team focus is on building and maintaining an open source data platform Caterpillar, leveraging a mono-repo strategy and a modular "lego block" approach to pipeline construction.
What Would You Need?
6-10 years of experience in complex data engineering roles, designing and operating reliable ingestion systems.
Exposure in building data ingestion and transformation pipelines using modern ETL frameworks.
Strong capabilities in at least one programming language used for data pipelines (e.g., Go preferred; Ruby on Rails/Python/Scala a plus).
Working knowledge of relational (Postgres/MySQL) and NoSQL databases. Advanced SQL query skills for performance and scale.
Strong experience with AWS Services: including S3, EMR/Glue, Athena, Lambda, EC2, RDS, IAM, and cost monitoring.
Demonstrated experience optimizing large-scale data pipelines for performance, including techniques such as parallel processing, multithreading, and/or massive parallel processing (MPP).
Understanding of bottlenecks in data pipelines and ability to optimize for high throughput and low latency.
Familiarity with DevOps and SRE principles; experience with infrastructure as code (Terraform/CloudFormation).
Experience writing and maintaining automated tests.
Clear, concise technical documentation skills and championing clean code.
What Could Set You Apart?
Deep hands-on expertise in building workflows with Apache Airflow, orchestrating complex DAGs, integrations, and monitoring.
Extensive experience with data streaming systems like Apache Kafka for high-throughput, low-latency ingestion and event-driven architectures.
Solid understanding of monitoring and observability stacks, including Prometheus for metrics and Grafana for dashboarding and incident response.
Experience building dashboards and visualizations in Apache Superset (or similar tools).
Bachelor’s or Master’s degree in Computer Science, Information Systems, or relevant field.
Familiarity with Snowflake, Trino query, Spark and similar distributed query/execution services.
Prior experience implementing end-to-end monitoring strategies and incident response processes.
Strong experience working with remote/global teams.
Exceptional written, verbal, and visual communication skills, including presenting complex data topics to non-technical stakeholders.
Experience operating in a highly regulated or security-conscious environment.
Demonstrated thought leadership in optimizing and automating cost and resource use in cloud data solutions.
Good to have Hands-on experience leveraging GPU acceleration (e.g., CUDA, RAPIDS, or similar) for distributed data processing or analytics workloads.
Hands-on experience applying AI applications and tools in software engineering tasks such as code generation, code review, automated testing, or research ("surfing") for solutions.
What We're About?
Game Changers- A game changer is someone who looks at problems with an open mind and shares new ideas with team members, regularly reassesses existing plans and attaches a realistic timeline to goals, makes profitable, productive, and innovative contributions, and actively pursues improvements to Pattern’s processes and outcomes.
Data Fanatics- A data fanatic is someone who recognizes problems and seeks to understand them through data, draws unbiased conclusions based on data that lead to actionable solutions, and continues to track the effects of the solutions using data.
Partner Obsessed- An individual who is partner obsessed clearly explains the status of projects to partners and relies on constructive feedback, actively listens to partner’s expectations, and delivers results that exceed them, prioritizes the needs of your partners, and takes the time to create a personable experience for those interacting with Pattern.
Team of Doers- Someone who is a part of team of doers uplifts team members and recognizes their specific contributions, takes initiative to help in any circumstance, actively contributes to supporting improvements, and holds themselves accountable to the team as well as to partners.
We are looking for individuals who are:
Pattern provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability, status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.
