<div><p>HackerRank helps thousands of companies like OpenAI, NVIDIA, Amazon hire developers based on their skills vs pedigree, and also nurtures a community of millions of developers to upskill themselves to become the next-gen developers. <br /><br />The people at HackerRank care deeply about their work and have an extremely intense work ethic. In many companies, speed &amp; quality is a tradeoff. At HackerRank, it’s not -- we expect you to ship in about half the time that most competent people think it’s possible while maintaining a standard of quality you’d proudly sign your name on. The only way to make this happen is if you truly love your craft and are deeply committed to growth.</p></div><h4><strong>About the Role</strong></h4> <p>We are seeking a Staff AI Scientist with deep expertise in machine learning fundamentals and a proven track record of building and evaluating advanced AI systems. This role is for someone who thrives at the intersection of data science, statistics, and ML engineering, with an emphasis on dataset rigor, metric design, and evaluation methodology.</p> <p>At the staff level, you will set the direction for how we evaluate AI systems, work across teams to ensure scientific rigor is balanced with product and engineering needs, and deliver both datasets and pipelines that stand up to scrutiny.</p> <h4><strong>What you’ll do</strong></h4> <ul> <li><strong>Dataset Preparation &amp; Evaluation</strong></li> <ul> <li>Design, prepare, and curate high-quality evaluation datasets with defensible methodology.</li> <li>Define criteria for dataset construction, ensuring statistical rigor, reproducibility, and fairness.</li> <li>Develop new metrics and evaluation frameworks to measure model performance in nuanced ways.<br /><br /></li> </ul> <li><strong>Model Evaluation &amp; ML Engineering</strong></li> <ul> <li>Evaluate LLMs and other pre-trained models using carefully chosen datasets and metrics.</li> <li>Build scalable pipelines for training, fine-tuning, and benchmarking models.</li> <li>Contribute to projects involving fine-tuning, retrieval-augmented generation (RAG), and other adaptation methods.<br /><br /></li> </ul> <li><strong>Cross-Functional Leadership</strong></li> <ul> <li>Partner with product and engineering to align scientific rigor with business outcomes.</li> <li>Define evaluation standards and ML lifecycle practices that raise the bar across the company.</li> <li>Mentor scientists and engineers, guiding best practices in experimentation, statistics, and ML development.</li> </ul> </ul> <h4><strong>Who you are</strong></h4> <ul> <li>Education: Master’s degree (PhD preferred) in Computer Science, Statistics, Machine Learning, or a related quantitative field.</li> <li>Strong background in mathematical and statistical foundations of machine learning (probability, linear algebra, optimization, experimental design).</li> <li>Demonstrated experience in end-to-end ML lifecycle: dataset preparation, model training, evaluation, deployment, and monitoring.</li> <li>Proven expertise in evaluation dataset design and metric creation, not just using existing benchmarks but knowing when and how to improve them.</li> <li>Experience with LLM evaluation, fine-tuning, and RAG, with the engineering skills to build production-ready pipelines.</li> <li>Track record of strategic impact at a staff or principal level setting evaluation and research standards across teams.</li> </ul> <h4><strong>Even better if you have</strong></h4> <ul> <li>Publications, open-source contributions, or widely adopted benchmarks in ML/AI.</li> <li>Prior experience shaping AI evaluation frameworks at an organizational or industry level.</li> <li>Comfort working at the interface of science, engineering, and product.</li> </ul> <h4><strong>You will thrive in role if</strong></h4> <ul> <li>You define gold-standard evaluation datasets and metrics that internal teams adopt as a standard.</li> <li>You ship datasets and pipelines that are statistically rigorous, reproducible, and scalable.</li> <li>You elevate the organization’s ML practice by embedding sound statistical and scientific methodology into product and engineering work.</li> <li>You are recognized as a thought leader who sets strategy while still contributing technically at depth.</li> </ul><div><p><strong>Want to learn more about HackerRank?</strong> Check out <a href="http://hackerrank.com/">HackerRank.com</a> to explore our products, solutions and resources, and dive into our story and mission <a href="https://www.hackerrank.com/about-us/">here</a>.</p> <p>HackerRank is a proud equal employment opportunity and affirmative action employer. We provide equal opportunity to everyone for employment based on individual performance and qualification. We never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines. </p> <p><a href="https://www.linkedin.com/company/hackerrank/">Linkedin</a> |<a href="https://twitter.com/hackerrank">X</a> |<a href="https://www.linkedin.com/company/hackerrank/"> </a><a href="https://blog.hackerrank.com/?utm_medium=email&amp;utm_content=signature">Blog</a> | <a href="https://www.instagram.com/hackerrank?igsh=MWNid2d6aWlxYnl0OQ==">Instagram</a> | <a href="https://www.instagram.com/lifeathackerrank?igsh=eDFta3owaTJsbW5x">Life@HackerRank</a>|</p> <p><em>Notice to prospective HackerRank job applicants:</em></p> <ul> <li><em>Our Recruiters use @hackerrank.com email addresses.</em></li> <li><em>We never ask for payment or credit check information to apply, interview, or work here.</em></li> </ul></div>

HackerRank

Similar Remote Jobs