Machine Learning & Data Operations Engineer

Location: US, Indianapolis IN

Time Type: Full time

Job Description

At Lilly, the work is demanding because patients are waiting. We unite caring with discovery to help make life better for people around the world, knowing that every decision, every detail, and every day matters. Headquartered in Indianapolis, Indiana, our over 50,000 employees around the globe take on complex challenges to discover and deliver life-changing medicines, strengthen how health is understood and managed, and support the communities we serve. This is hard, urgent, selfless work—but it’s work worth doing. If you’re driven by purpose and ready to bring your best to work that truly matters for patients, we invite you to join us. 

Position Summary

As a Machine Learning & Data Operations Engineer on TuneLab, you will build cutting-edge ML and AI tools alongside a team of engineers and scientists to accelerate and enhance Lilly’s drug discovery process. You will take a hands-on role across the full lifecycle of models and the data that feeds them: moving trained models from research into reliable production environments, running inference at scale, and building the data pipelines and readiness checks that keep the data substrate underpinning those models trustworthy. You will stand up the validation, monitoring, and model-card review that keep both models and data production-ready—catching anomalies, schema drift, and performance regressions before they reach researchers. You will collaborate closely with partners across Lilly Research Labs, AI, Software Engineering, Data Science, and IT Operations, along with industry-leading external collaborators, to put the power of ML and computational tooling directly into researchers’ day-to-day work.

Core Responsibilities

Model Deployment, Serving & Inference

• Move trained models from research and experimentation into production, packaging, versioning, and promoting them across development, staging, and production environments and across cloud targets (AWS, Azure, GCP) and on-prem or hybrid infrastructure
• Build and operate scalable inference services and APIs—batch, real-time, and streaming—delivering low-latency, high-throughput serving that meets researcher and downstream-system needs
• Design and maintain model-serving infrastructure using containers and Kubernetes, with autoscaling, versioned rollouts (e.g., blue-green or canary), and rollback so updates ship without disrupting users
• Integrate models into researcher-facing tools and enterprise systems, ensuring seamless interoperability and data flow across platforms

Data Pipelines & Readiness

• Design, build, and maintain scalable, secure data pipelines—batch, change-data-capture (CDC), and streaming—that move and transform data across the platform, including the embedding, vectorization, and feature pipelines that feed downstream ML and LLM applications
• Implement scalable storage and retrieval for large-scale structured and unstructured scientific data across cloud and on-prem or hybrid infrastructure
• Build and operate automated data-readiness and quality-monitoring workflows for high-dimensional scientific and enterprise datasets, including multi-method anomaly and outlier detection across numerical and categorical data
• Validate files for missing values, illegal characters, and structural issues, and build schema-drift detection with historical tracking and automated reporting—catching data-contract changes before they reach models and significantly reducing manual data QA

Model & Data Validation, Monitoring & Governance

• Author, review, and validate model cards—verifying documented performance, intended use, limitations, data lineage, and evaluation results before models are promoted
• Run and automate model validation and evaluation—reproducing metrics, checking calibration and performance against acceptance criteria, and gating promotion on the results
• Implement production monitoring for model, data, and service health—latency, throughput, data and prediction drift, and quality—with alerting and proactive remediation
• Define acceptance criteria, audit trails, and reproducible checks; adjudicate flagged data and model issues with data owners and scientists; and track and report operational metrics

Software & Platform Engineering

• Design and develop robust, scalable, and secure software solutions with a hands-on approach, from architecture through implementation
• Build and maintain microservices architectures and APIs (REST and GraphQL) that support model serving, data access, and tool-calling workflows
• Implement infrastructure-as-code and CI/CD pipelines to automatically test and deploy model, data, and service updates, applying test-driven development to catch regressions early
• Apply systems-engineering practices to distributed systems with high throughput and availability requirements, and troubleshoot complex issues across the model, data, and serving stack

Cross-functional Partnership

• Collaborate within a team of engineers using best practices such as design reviews, code reviews, testing, and continuous integration and deployment
• Partner with Lilly Research Labs, Data Science, AI/ML, and IT Operations to translate research and business requirements into technical solutions
• Work with external, industry-leading collaborators to integrate models, data, and tooling into shared and federated workflows within Lilly’s controlled cloud environment
• Contribute to platform adoption through clear documentation, data dictionaries, runbooks, and support for internal end users

Required Qualifications

• Ph.D. in Computer Science or a related computational field (e.g., Computational Science, Computational Biology, Bioinformatics, or a related quantitative computational discipline)
• Hands-on experience in software engineering and architecture, with a proven track record of delivering complex, cross-functional solutions
• Proficiency in a systems or object-oriented language (Go, Rust, Java, or C++) and a scripting language (Python and/or JavaScript)
• Hands-on experience deploying to containers, serverless, Kubernetes, and other hosting targets
• Experience deploying and serving machine learning models in production, including packaging, versioning, and promotion across environments
• Experience building data pipelines and working with relational and non-relational data stores (e.g., PostgreSQL, MySQL, MongoDB)
• Solid understanding of HTTP and RESTful APIs
• Experience using CI tools to automatically test and CD tools to automatically deploy updates, and applying test-driven development to prevent feature regression
• Experience applying systems-engineering concepts to distributed systems with high throughput and availability requirements

Preferred Qualifications

• Experience integrating AI/ML models into production with a focus on scalability, performance, and reliability (MLOps)
• Familiarity with MLOps and model-serving tooling (e.g., MLflow, Kubeflow, and model or artifact registries such as JFrog Artifactory)
• Experience with model validation, evaluation, and model-card and documentation practices for model governance
• Experience implementing data-quality, anomaly-detection, or schema-drift monitoring for production datasets
• Familiarity with streaming and CDC tooling (e.g., Kafka, Kafka Streams, Spark Streaming) and big-data processing (Spark)
• Familiarity with LLM application patterns—retrieval-augmented generation, tool-calling, and multi-agent orchestration—and with inference optimization
• Experience with infrastructure-as-code (Terraform), service mesh, and cloud-native monitoring and observability
• Exposure to drug discovery, life sciences, or healthcare data and workflows, including high-dimensional or biological datasets
• Experience contributing to federated or collaborative ML and data initiatives across organizations

Lilly is dedicated to helping individuals with disabilities to actively engage in the workforce, ensuring equal opportunities when vying for positions. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form (https://careers.lilly.com/us/en/workplace-accommodation) for further assistance. Please note this is for individuals to request an accommodation as part of the application process and any other correspondence will not receive a response.

Lilly is proud to be an EEO Employer and does not discriminate on the basis of age, race, color, religion, gender identity, sex, gender expression, sexual orientation, genetic information, ancestry, national origin, protected veteran status, disability, or any other legally protected status.

Our employee resource groups (ERGs) offer strong support networks for their members and are open to all employees. Our current groups include: Africa, Middle East, Central Asia (AMECA), Black Employees at Lilly (BE@Lilly), Chinese Culture Network (CCN), EnAble, Evolve, Lilly Indian Network (LIN), Organization of Latinx at Lilly (OLA), Pride (LGBTQ+ Allies), Veterans Leadership Network (VLN) and Women’s Initiative for Leading at Lilly (WILL).

Actual compensation will depend on a candidate’s education, experience, skills, and geographic location. The anticipated wage for this position is

$151,500 - $244,200

Full-time equivalent employees also will be eligible for a company bonus (depending, in part, on company and individual performance). In addition, Lilly offers a comprehensive benefit program to eligible employees, including eligibility to participate in a company-sponsored 401(k); pension; vacation benefits; eligibility for medical, dental, vision and prescription drug benefits; flexible benefits (e.g., healthcare and/or dependent day care flexible spending accounts); life insurance and death benefits; certain time off and leave of absence benefits; and well-being benefits (e.g., employee assistance program, fitness benefits, and employee clubs and activities).Lilly reserves the right to amend, modify, or terminate its compensation and benefit programs in its sole discretion and Lilly’s compensation practices and guidelines will apply regarding the details of any promotion or transfer of Lilly employees.

## WeAreLilly