Evaluate generative AI systems, RAG pipelines, and agents using Python-based frameworks and evaluation platforms. Responsibilities include model and agent quality assessment, Responsible AI validation, RAI metrics reporting, golden and synthetic dataset creation, and establishing AI observability and tracing with OpenTelemetry.
• Strong understanding of GenAI system architecture (RAG / Agents) • Implement python-based AI Evaluation frameworks like Ragas, DeepEval, etc • Perform Model & Agent quality evaluation using tools like Arize, LangFuse, ConfidentAI, etc • Perform Responsible AI validation & report RAI metrics • Golden dataset curation & Synthetic dataset generation • Ability to instrument & establish AI Observability / Tracing using OTEL libraries Expert in Python
Hexaware Technologies Pune, Mahārāshtra, IND Office
North Block, Plot No. 19, Rajiv Gandhi InfoTech Park, MIDC - SEZ, Phase 3, Hinjawadi, Pune, Maharastra, India, 411057
Similar Jobs
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Maintains and troubleshoots semiconductor test equipment, performs preventive maintenance and calibration, monitors equipment and process data, investigates performance deviations, documents activities, communicates issues across engineering and production teams, and supports yield improvement and continuous improvement projects using data and Generative AI insights.
Top Skills:
Generative Ai
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Supports 0–13-week assembly and test execution planning by aligning supply, demand, inventory, and manufacturing capacity. Monitors production plans, inventory, fill rates, delivery performance, and at-risk products; analyzes gaps and root causes; coordinates recovery actions across supply chain, manufacturing, and business teams; and contributes to process improvements, system enhancements, and cross-functional planning initiatives.
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Manage pharmaceutical quality systems and compliance for quality control and stability laboratories. Responsibilities include approving method transfer and validation documentation, ensuring cGMP compliance, leading change control, tracking CAPA and metrics, managing LIMS builds, reviewing laboratory data, supporting investigations, participating in audits and safety inspections, recommending corrective actions, and mentoring junior colleagues.
Top Skills:
CapaChange ControlCurrent Good Manufacturing Practices (Cgmp)Laboratory Information Management Systems (Lims)Quality Management Systems (Qms)Risk Management
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
