Evaluate generative AI systems, RAG pipelines, and agents using Python-based frameworks and evaluation platforms. Responsibilities include model and agent quality assessment, Responsible AI validation, RAI metrics reporting, golden and synthetic dataset creation, and establishing AI observability and tracing with OpenTelemetry.
• Strong understanding of GenAI system architecture (RAG / Agents) • Implement python-based AI Evaluation frameworks like Ragas, DeepEval, etc • Perform Model & Agent quality evaluation using tools like Arize, LangFuse, ConfidentAI, etc • Perform Responsible AI validation & report RAI metrics • Golden dataset curation & Synthetic dataset generation • Ability to instrument & establish AI Observability / Tracing using OTEL libraries Expert in Python
Hexaware Technologies Pune, Mahārāshtra, IND Office
North Block, Plot No. 19, Rajiv Gandhi InfoTech Park, MIDC - SEZ, Phase 3, Hinjawadi, Pune, Maharastra, India, 411057
Similar Jobs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead and grow engineering teams building large-scale AI/ML platform infrastructure (model serving, training, feature stores). Drive technical strategy, deliver complex backend systems to production, partner with product and stakeholders, uphold high engineering and operational standards, and coach engineers for career growth.
Top Skills:
Backend PlatformsDistributed SystemsFeature StoresGpu-Accelerated Model ServingLlmsTraining And Data Access Infrastructure
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Develop and maintain secure, high-performance Android applications used by sales and partner teams. Responsibilities include Kotlin development, Android architecture, testing, performance optimization, backend API integration, troubleshooting, code reviews, mentoring, and technical planning. The role requires collaboration with business and engineering stakeholders and familiarity with Jetpack technologies, Salesforce integrations, and AI-enabled workflow improvements.
Top Skills:
Ai TechnologiesAndroid SdkAndroid StudioClean ArchitectureCoupaEspressoFlowGoogle Play StoreGradleHiltJetpack ComposeJIRAKotlinKotlin CoroutinesLightning Web ComponentsLivedataMaterial DesignMulesoftMviMvvmNavigationRestful ApisRoomSalesforceSalesforce Experience CloudViewmodel
Cloud • Information Technology • Security • Software
Define and drive next-generation platform architecture, embed AI capabilities into identity and device management systems, guide executive technical strategy, design globally scalable cloud systems, mentor engineering teams, evaluate competitors, and adapt architecture to emerging AI, security, and privacy requirements.
Top Skills:
AIAWSAzureCloud-Native ArchitectureData PipelinesDistributed SystemsDockerGCPGoKubernetesPythonRust
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
