Design and validate realistic broken Linux and containerized infrastructure environments for AI systems. Build multi-service failure scenarios using Docker, Kubernetes, databases, and message queues; troubleshoot complex production-like incidents; create reference solutions, setup scripts, automated verifiers, and testing guidance. Investigate networking, storage, permissions, replication, and recovery issues while ensuring scenarios are reproducible, technically accurate, and challenging.
This role is for one of Weekday’s clients
Min Experience: 2+ years
Engagement Type: Ongoing, Hourly Contract
Work Mode: Fully Remote
Working Hours: 5–6 hours per day
Hourly Compensation: USD 20–25 per hour
We are looking for an experienced DevOps / SRE Test Bench Engineer to design, develop, and validate challenging infrastructure troubleshooting tasks for AI systems. In this role, you will create realistic, intentionally broken Linux and containerized environments that require deep technical investigation, systematic debugging, and recovery.
You will also develop reference solutions and automated verification scripts to ensure that each issue is genuinely resolved. The ideal candidate has strong hands-on infrastructure experience, a methodical troubleshooting approach, and the ability to build complex, realistic scenarios that test practical engineering skills.
RequirementsKey Responsibilities
- Design realistic, multi-service failure scenarios that require extensive investigation, root-cause analysis, and recovery.
- Build and configure test environments using Linux, Docker, Kubernetes, databases, message queues, and service dependencies.
- Develop broken infrastructure setups that simulate real-world production incidents and operational challenges.
- Write detailed reference solutions, troubleshooting guidance, and reliable automated verifiers with clearly defined success criteria.
- Validate task quality to ensure scenarios are fair, technically accurate, reproducible, and not solvable through a fixed runbook.
- Troubleshoot complex issues involving networking, storage, permissions, service configurations, replication, and system recovery.
- Develop setup scripts, automation utilities, and verification tests using Python or Bash.
- Accurately log working hours, communicate progress, and actively participate in the project’s WhatsApp community.
- Maintain the confidentiality of all project-related tasks, environments, and technical materials.
- Hands-on experience in DevOps, Site Reliability Engineering (SRE), platform engineering, infrastructure engineering, or a related technical role.
- Strong Linux administration and command-line skills, including SSH, shell scripting, process management, log analysis, file permissions, networking, and system services.
- Strong hands-on expertise in Docker and Kubernetes, including diagnosing container failures, broken deployments, service connectivity issues, and orchestration problems.
- Experience with databases and infrastructure components such as PostgreSQL, Redis, message queues, storage systems, replication, and recovery mechanisms.
- Proven ability to troubleshoot production-like environments involving multiple interdependent services.
- Familiarity with CI/CD pipelines, Infrastructure as Code (IaC), and cloud infrastructure.
- Ability to investigate ambiguous failures independently, identify root causes, and implement effective solutions without relying on predefined runbooks.
- A strong testing mindset, including experience designing automated checks, validation scripts, and deterministic verifiers.
- Proficiency in Python or Bash for environment setup, troubleshooting automation, and test development.
- Experience with cloud platforms such as AWS, Google Cloud Platform (GCP), or Microsoft Azure.
- Familiarity with Nginx, systemd, Logical Volume Manager (LVM), and Linux security hardening.
- Understanding of distributed systems, service dependencies, and fault recovery.
- Experience working with AI coding agents or AI-assisted software engineering tools.
- Complete the sign-up form to express your interest.
- Shortlisted candidates may be invited to complete a brief technical screening.
- Selected candidates will receive onboarding instructions and access to the project’s WhatsApp community.
- Compensation is hourly, based on approved and accurately logged working hours.
- All project tasks, technical materials, and related information must remain confidential and must not be shared outside the project.
Similar Jobs
Digital Media • Information Technology • News + Entertainment
Designs and customizes software applications, manages releases, gathers stakeholder requirements, mentors junior engineers, maintains technical documentation, evaluates performance metrics, and partners with Quality Assurance to ensure reliable, compliant software. The role also uses market trends to guide product improvements and participates in peer programming, design sprints, and prototyping. Requires independent judgment, consistent attendance, and flexibility to work nights, weekends, and variable schedules.
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Monitor quality deviations, investigate manufacturing defects, perform root cause analysis, manage lot containment, and coordinate corrective actions. The role also monitors equipment and process performance, supports qualification and troubleshooting, drives yield and quality improvements, and uses Excel, Power BI, AI tools, dashboards, and statistical methods to identify trends and risks in semiconductor manufacturing.
Top Skills:
Automated Vision InspectionChatgptFmeaGenerative AiMachine LearningMicrosoft CopilotExcelMicrosoft Power BiStatistical Analysis
Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Lead the AI Quality engineering team while remaining hands-on building evaluation infrastructure, observability tooling, diagnostic systems, datasets, annotation workflows, and CI/CD pipelines for production AI agents. Run experiments across prompts, models, routing, caching, and agent harnesses to improve quality, latency, and cost. Partner with AI Core engineering and manage technical direction and team priorities.
Top Skills:
Ai AgentsBraintrustCi/CdLangsmithLlmsPythonRuby On Rails
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
