Lead the design of prompts and development of custom AI agents, optimizing LLM performance and integrating APIs across internal platforms.
Overview
We are looking for a hybrid AI Platform Engineer to lead the design of high-performance prompts and the development of custom AI agents and integrations across our internal platforms. Collaborating closely with our AI Platform Manager and functional business units, this role bridges the creative precision of prompt engineering with the technical fluency required to build and deploy powerful LLM-based tools using APIs, system integrations, and emerging AI infrastructure.
Key Responsibilities
Requirements
We are looking for a hybrid AI Platform Engineer to lead the design of high-performance prompts and the development of custom AI agents and integrations across our internal platforms. Collaborating closely with our AI Platform Manager and functional business units, this role bridges the creative precision of prompt engineering with the technical fluency required to build and deploy powerful LLM-based tools using APIs, system integrations, and emerging AI infrastructure.
Key Responsibilities
- Prompt Engineering & Optimization
- Design, test, and refine prompts for general and use-case-specific applications.
- Evaluate and improve LLM output quality based on accuracy, latency, hallucination, and user satisfaction.
- Maintain a scalable prompt library with robust documentation and experimentation tracking.
- Custom Agent & GPT Development
- Build and maintain custom GPTs (via OpenAI) or Google Agents (Gemini, Vertex AI).
- Configure tools, memory, system prompts, and API-calling capabilities for agent workflows.
- Ensure agents follow internal data governance and compliance policies.
- API & Platform Integration
- Integrate LLM functionality into internal systems (e.g., dashboards, Slack, CRMs, internal portals).
- Work with APIs from OpenAI, Google Cloud, Azure, and others to create functional pipelines.
- Support rapid prototyping and deployment of AI features in close coordination with product and automation teams.
- Evaluation & Instrumentation
- Define key performance metrics (quality, reliability, usage patterns).
- Collaborate with data engineering to log, analyze, and visualize LLM performance.
- Implement feedback loops to continuously optimize prompt and agent behavior.
- Enablement & Documentation
- Provide tooling and guidance to business users and product teams for prompt-based workflows.
- Contribute to internal AI playbooks and best practices around prompt usage and API integration.
Requirements
- 5+ years of experience in AI/ML, NLP applications, or full-stack product development
- Hands-on experience with OpenAI APIs, Google Cloud Vertex/Gemini, or similar platforms
- Strong experience with prompt engineering and LLM evaluation methods
- Proficiency in Python and working with APIs, JSON, and vector databases
- Familiarity with LangChain, LlamaIndex, or custom agent frameworks
- Excellent collaboration and communication skills
Top Skills
APIs
GCP
JSON
Langchain
Llamaindex
Openai Apis
Python
Vector Databases
Vertex Ai
Rapid7 Pune, Mahārāshtra, IND Office
Pune, India
Similar Jobs at Rapid7
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
As a Lead Salesforce Engineer, you will design and implement Salesforce solutions, manage environments, write code, ensure system health, and support deployment. You'll collaborate with global teams while adhering to compliance and change management protocols.
Top Skills:
ApexApplication EngineeringGearsetJIRALightning Web ComponentsSalesforceSalesforce CpqSOQLSoslVisualforce
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
The Senior Software Engineer will enhance vulnerability coverage, improve automation frameworks, mentor engineers, and collaborate on roadmap planning.
Top Skills:
AnsibleAWSAzureGCPGitJavaJenkinsPythonRubyTerraform
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
The Data Engineer will manage Snowflake infrastructure, develop Tableau reports, implement data modeling, and ensure compliance, enhancing data accessibility and analytics capabilities.
Top Skills:
AirflowDbtEltETLFivetranMatillionSnowflakeSQLTableau
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
