Software EngineerStraight from Mercor’s careers page. Apply on the company site — no recruiter, no middleman.
Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI labs models. Youll assess repository-level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric-based written feedback.
Basic Qualifications • 3+ years professional software engineering • Real open-source contribution or maintainer experience (merged PRs, committer / maintainer roles) • Strong ability to audit reference patches, test runners, and Docker isolation, and to detect answer leakage / reward hacking • Fluency across common ecosystems (Python and at least one of Java / Go / TypeScript / C++)
Preferred Qualifications • Familiarity with SWE-Bench (Verified) or similar repository benchmarks • Maintainer history on major Python OSS (Django, Flask, scikit-learn, sympy, pytest, etc.) • Prior code-review or task-grading experience
Similar remote jobs
All Software Engineer jobs →
Discover More than 100,000 Hidden Remote Jobs Before Everyone Else
Unlock All Remote Jobs Today
Simple pricing. Big savings on Quarterly and Yearly.
Monthly Access
- Instant access to fresh remote jobs from 500+ companies
- New opportunities added hourly, often 3-7 days before anywhere else
- Advanced filtering by role type, stack, pay, and location
- Priority customer support
Yearly Access
- Everything in Monthly
- Save $169 (~74%) vs paying monthly
- Average job search takes ~6 months - get covered for the whole journey
- Less than the cost of one lunch per month for competitive advantage
- Equivalent to just ~$4.92/month