<p>Description</p><p><strong>About CloudBees</strong></p><p><br /></p><p>CloudBees is the leading software delivery platform for enterprise DevOps teams. As a high-growth startup, we empower developers to build, deploy, and manage software more efficiently. Now, we’re bringing <strong>agentic intelligence</strong> into our platform to supercharge developer workflows—and we need a data scientist who can both drive insights and tell the story behind the metrics.</p><p><br /></p><h3><strong>The Role</strong></h3><h3><br /></h3><p>CloudBees is seeking a startup-savvy <strong>Data Scientist</strong> to help define, measure, and evangelize the impact of <strong>Agentic Applications</strong> across our platform. You’ll work closely with engineers and product teams to <strong>prototype and measure AI and Agentic experiences, using evals, telemetry, and AI benchmarks</strong> to help the company drive the conversation in the market and with customers. Translating performance into clear, compelling narratives to our customers and internal teams.</p><p><br /></p><p>As a founding member of the team, you will lead the charge as equal parts <strong>builder, evaluator, and communicator</strong>—with the technical depth to prototype in Python notebooks, Claude Code, and other tools to drive clarity to write about what matters.</p><h3><br /></h3><h3><strong>Key Responsibilities</strong></h3><ul><li>Partner with our platform team to develop and prototype <strong>telemetry, eval frameworks, and benchmarks</strong> for emerging agentic systems.</li><li>Partner with product and engineering teams to <strong>measure AI outcomes and usage across customers and teams</strong>.</li><li>Help define KPIs and success metrics for AI and LLM-driven features and workflows.</li><li>Use Python notebooks to <strong>explore data, visualize insights</strong>, and test hypotheses rapidly and share insights.</li><li>Tell the story behind the numbers: Write internal documentation, performance summaries, and thought leadership around outcomes.</li><li>Enable engineering teams to <strong>instrument, log, and evaluate</strong> agent performance effectively.</li><li>Stay up to date with evolving metrics and evaluation techniques in the LLM and agentic AI ecosystem.</li></ul><h3><strong>Required Qualifications</strong></h3><ul><li>3+ years of experience in data science or ML analytics roles, ideally in startup or high-growth environments.</li><li>Proficiency in <strong>Python</strong>, including building and sharing analysis via <strong>Jupyter notebooks</strong>.</li><li>Experience working with evals, telemetry, A/B testing, and evaluating user-facing ML systems.</li><li>Experience with AI/ML tools such as MLFlow, Hugging Face, or other Model / LLM tools.</li><li>Ability to partner with technical teams to define meaningful metrics and benchmarks.</li><li>Clear communication skills—capable of <strong>writing about outcomes</strong>, sharing learnings, and influencing stakeholders.</li><li>Comfort working in fast-paced, ambiguous environments where speed and clarity matter.</li></ul><h3><strong>Preferred Qualifications</strong></h3><ul><li>Experience with <strong>agentic or LLM-based applications</strong> (e.g., evaluating AI copilots, autonomous workflows).</li><li>Familiarity with tools like LangSmith, OpenInference, or custom eval stacks.</li><li>Background in developer tools, DevOps, or platform engineering environments.</li></ul><h3><strong>Why Join CloudBees</strong></h3><ul><li>Shape the future of AI-driven DevOps with real user impact.</li><li>Join a nimble, passionate team at the forefront of agentic system development.</li><li>Work in a flexible, remote-first culture built on trust and innovation.</li><li>Competitive salary, startup equity, and excellent benefits.</li></ul><p><strong>CloudBees is proud to be an Equal Opportunity Employer.</strong> We value diverse voices, ideas, and experiences as essential to building great products.</p><p><br /></p>