Matillion Logo

Matillion

Senior QA Engineer - AI & LLM

Posted Yesterday
Be an Early Applicant
Hybrid
Hyderabad, Telangana
Senior level
Hybrid
Hyderabad, Telangana
Senior level
The Senior QA Engineer will oversee quality assurance in SaaS development, implementing automation, mentoring teams, and using GenAI technologies to enhance testing practices.
The summary above was generated by AI

Matillion is The Data Productivity Cloud.


We are on a mission to power the data productivity of our customers and the world, by helping teams get data business ready, faster. Our technology allows customers to load, transform, sync and orchestrate their data. 


We are looking for passionate, high-integrity individuals to help us scale up our growing business. Together, we can make a dent in the universe bigger than ourselves.


With offices in the UK, US and Spain, we are now thrilled to announce the opening of our new office in Hyderabad, India. This marks an exciting milestone in our global expansion, and we are now looking for talented professionals to join us as part of our founding team.


About the Role


We’re on the hunt for a Senior Quality Engineer to help us push the boundaries of SaaS quality assurance while integrating cutting-edge Generative AI (GenAI) technologies. At the heart of our engineering culture is a commitment to delivering high-quality, scalable, and secure software – and we believe quality should be a shared responsibility across the team.


As a Senior QA Engineer, you'll play a pivotal role in shaping and maintaining quality throughout the SDLC – combining deep QA expertise with a passion for emerging AI technologies. You’ll work hands-on with cross-functional teams, take ownership of our testing strategies, and guide the adoption of best practices across automation, GenAI evaluations, and testing frameworks.

What You'll Be Doing

  • Champion a shift-left testing culture by embedding quality across every phase of our software development lifecycle.
  • Design, build, and maintain robust test automation frameworks for functional, integration, performance, security, and accessibility testing.
  • Collaborate closely with product managers, developers, and AI engineers to test and evaluate LLM-integrated features using both traditional and GenAI-specific testing methods.
  • Implement tools and techniques to test LLM-based functionality, including prompt evaluation, RAG-enhanced responses, and API behavior.
  • Drive the use of modern CI/CD pipelines, ensuring testing is efficient, scalable, and automated wherever possible.
  • Analyze, interpret, and share meaningful insights from test results – especially when comparing LLM model responses, prompt outcomes, and system behavior.
  • Mentor team members in test strategies, automation practices, and GenAI testing methodologies.
  • Continuously assess and improve the quality landscape, identifying opportunities to increase speed, accuracy, and innovation in how we test.

What We're Looking For

  • Core QA Expertise
  • Strong experience with end-to-end testing of distributed systems and SaaS applications.
  • Deep understanding of testing methodologies including exploratory, regression, risk-based, performance, and security testing.
  • Skilled in writing automated tests using tools like Cypress, Pact, or similar frameworks, with coding experience in Java/JavaScript.
  • Solid understanding of the Test Pyramid, CI/CD pipelines, and agile methodologies (Scrum, Kanban).
  • Experience working with cloud platforms like AWS, GCP, or Azure, plus comfort with SQL and database testing.
  • A team player with a QA-first mindset and a drive to elevate engineering standards across the board.

  • GenAI/LLM Knowledge (Essential)
  • Hands-on experience working with LLM APIs like OpenAI, Anthropic, or HuggingFace.
  • Familiarity with prompt engineering, fine-tuning prompts for desired outcomes.
  • Practical experience using Retrieval-Augmented Generation (RAG) to enrich LLM responses.
  • Understanding of key GenAI concepts like system/user prompts, tokens, embeddings, temperature, top-p, context windows, and stop sequences.

  • Nice-to-Have
  • Experience with fine-tuning LLMs or training smaller models for specific tasks.
  • Familiarity with prompt compression techniques and advanced GenAI patterns like prompt chaining, agentic workflows, or LLM routing.
  • Exposure to abstraction libraries such as Spring AI or similar.
  • Experience with LLM eval frameworks to benchmark model performance and prompt effectiveness.

Matillion has fostered a culture that is collaborative, fast-paced, ambitious, and transparent, and an environment where people genuinely care about their colleagues and communities. 


Our 6 core values guide how we work together and with our customers and partners. We operate a truly flexible and hybrid working culture that promotes work-life balance, and are proud to be able to offer the following benefits:


- Company Equity

- 27 days paid time off

- 12 days of Company Holiday

- 5 days paid volunteering leave

- Group Mediclaim (GMC)

- Enhanced parental leave policies

- MacBook Pro

- Access to various tools to aid your career development


More about Matillion

Thousands of enterprises including Cisco, DocuSign, Slack, and TUI trust Matillion technology to load, transform, sync, and orchestrate their data for a wide range of use cases from insights and operational analytics, to data science, machine learning, and AI. 


With over $300M raised from top Silicon Valley investors, we are on a mission to power the data productivity of our customers and the world.


We are passionate about doing things in a smart, considerate way. We’re honoured to be named a great place to work for several years running by multiple industry research firms. 


We are dual headquartered in Manchester, UK and Denver, Colorado.


We are keen to hear from prospective Matillioners, so even if you don’t feel you match all the criteria please apply and a member of our Talent Acquisition team will be in touch. Alternatively, if you are interested in Matillion but don't see a suitable role, please email [email protected].


Matillion is an equal opportunity employer. We celebrate diversity and we are committed to creating an inclusive environment for all of our team. Matillion prohibits discrimination and harassment of any type. Matillion does not discriminate on the basis of race, colour, religion, age, sex, national origin, disability status, genetics, sexual orientation, gender identity or expression, or any other characteristic protected by law.

Top Skills

Anthropic
AWS
Azure
Cypress
GCP
Huggingface
Java
JavaScript
Llm Apis
Openai
Pact
Spring Ai
SQL

Similar Jobs

8 Hours Ago
Hybrid
Hyderabad, Telangana, IND
Senior level
Senior level
Financial Services
As Principal Architect at JPMorgan Chase, you will develop cloud-based architectural platforms, advise on technology selection, and lead cross-functional teams in delivering secure code and innovative solutions.
Top Skills: Java
Yesterday
Hybrid
Hyderabad, Telangana, IND
Junior
Junior
Financial Services
As a Software Engineer II, you'll design, develop, and troubleshoot software solutions, collaborate within an agile team, and contribute to secure application development.
Top Skills: AWSJavaPythonSpring
Yesterday
Hybrid
Hyderabad, Telangana, IND
Mid level
Mid level
Financial Services
As a Software Engineer III, you'll design and deliver technology solutions, develop secure production code, and engage in continuous improvement leveraging data insights. You will contribute to team culture and software engineering communities while applying your technical skills in a collaborative agile environment.
Top Skills: AWSContainersJavaKafkaKubernetesMicroservicesPivotal Cloud FoundrySQLTerraform

What you need to know about the Pune Tech Scene

Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account