Project/Opportunity Details
A SWEAP task is a real-world GitHub problem packaged with a testing environment and solution (i.e., the golden patch). This environment and solution are used to train an agent to solve the problem and verify the solution in the testing environment. A contributor’s job is to verify the testing environment based on testing logs, categorize the problem by specificity and knowledge areas, and write additional notes to help an agent understand what a good solution looks like based on the code edited in “the golden patch” and notes on GitHub.
Accepted Locations
We accept applicants from the US, Canada, and most countries in LATAM and Europe. We are also accepting candidates from some countries in Africa and Asia. For the complete list of accepted locations, click here. This work is 100% remote.
Loom Video
Our Founder/CEO, Gabe Greenberg, created an in-depth Loom video that we highly recommend you watch! Check it out here: Loom Video
Overview
Join our expert annotation team to create training data for the world's most advanced AI models. No previous AI experience is necessary. You'll get your foot in the door with one of the most prominent players in the AI/LLM space today. We're seeking contributors with professional software engineering experience on production repositories and experience building and maintaining large-scale coding repositories. Projects typically include discrete, highly variable problems that involve engaging with these models as they learn to code. We currently have 200+ roles open!
What Will I Be Doing?
-
Verifying the testing environment based on testing logs.
-
Categorizing problems by specificity and knowledge areas.
-
Writing additional notes to help an agent understand what a good solution looks like based on the code edited in “the golden patch” and notes on GitHub.
-
Evaluating the quality of AI-generated code, including human-readable summaries of your rationale.
-
Solving coding problems and writing functional and efficient code in various programming languages.
-
Writing robust test cases to confirm code works efficiently and effectively.
-
Creating instructions to help others and reviewing code before it goes into the model.
-
Engaging in a variety of projects, from evaluating code snippets to developing full applications using chatbots.
Pay Rates
Compensation rates average at $30/hr and can go up to $50/hr. Expectations are 15+ hours per week; however, there is no upper limit. You can work as much as you want and will be paid weekly per hour of work done on the platform.
Contract Length
This is a long-term contract with no end date. We expect to have work for the next 2 years. You can end the contract at any time, but we hope you will commit to 12 months of work.
Flexible Schedules
Developers can set their own hours—ideal candidates will be interested in spending 40 hours a week. You will be with teams, so strong performers will adapt to the urgency of projects and stay engaged, but we are incredibly flexible on working hours. You can take a 3-hour lunch with no problem. Instead of tracking your hours, you are paid according to time spent on the platform, calculated in the coding exercises.
Interview Process
-
Apply using this Ashby form.
-
If you seem like a good fit, we'll send an async RLHF code review that will take 35 minutes and must be finished within 72 hours of us sending it.
-
You'll receive credentials to the RLHF platform. We'll then set up a group call to answer any further questions about onboarding with the company.
-
You'll perform a simulated production-level task (RLHF task) on the platform. This will be your final interview, which will ultimately determine your employment and leveling. Successful completion of this process provides you with an opportunity to work on projects as they become available.
Tech Stack Priorities
The current priority for this team is contributors who are well versed in Go, Git, Java, JavaScript, Python, and TypeScript.
Required Qualifications
-
Professional software engineering experience on production repositories.
-
Experience building and maintaining large-scale coding repositories.
-
Complete fluency in the English language.
-
Ability to articulate complex technical concepts clearly and engagingly.
-
Excellent attention to detail and ability to maintain consistency in writing. Solid understanding of grammar, punctuation, and style guidelines.
Nice To Haves:
-
Bachelor's or Master's degree in Computer Science.
-
Experience with modern JavaScript frameworks and libraries (Next.js, Vue, Angular).
-
Familiarity with frontend testing frameworks (Jest, React Testing Library, Cypress).
-
Knowledge of state management solutions (Redux, Context API, MobX).
-
Experience with TypeScript and modern frontend tooling.
-
Recognized accomplishments or contributions to the coding community or in projects.
-
Proven analytical skills with an ability to approach problems creatively.
-
Adept communication skills, especially when understanding and discussing project requirements.
-
A commitment to continuous learning and staying updated with the latest coding advancements and best practices.
-
Enthusiasm for teaching AI models and experience with technical writing!
If you're passionate about software engineering and the future of AI technology, this is an excellent opportunity to contribute to cutting-edge AI technology while leveraging your expertise!
.png)
