Staff Machine Learning EngineerStraight from Robots & Pencils’s careers page. Apply on the company site — no recruiter, no middleman.
Staff ML Engineer – AWS Trainium & SageMaker
Location: Remote - Canada
Department: Engineering
Robots & Pencils is an AWS Partner building production AI systems for enterprise clients who need real engineering, not proofs of concept that never ship. We work forward-deployed, embedded directly with client teams, solving the problems that are too new or too specialized for a typical vendor relationship to handle.
The Role
Were looking for an engineer who can operate and train models on Amazon SageMaker running on AWS Trainium, AWSs custom silicon built specifically for large-scale model training. This isnt a role where you call an API and wait. Youll be walking up the stack: understanding what a training request actually looks like at the Trainium hardware and compiler level, then carrying that understanding all the way up through PyTorch training code and into a production SageMaker pipeline.
PyTorch is the backbone of this work. If you know the framework deeply and youre comfortable reasoning about how your code actually behaves on custom accelerator hardware rather than treating it as a black box, this role is built around that skill set specifically.
What Youll Do
- Train and operate models on Amazon SageMaker with AWS Trainium as the underlying compute
- Write and optimize PyTorch training code with a real understanding of how it compiles and executes on Trainium (NeuronCore architecture, compiler behavior, memory and throughput tradeoffs)
- Diagnose training run issues that show up specifically because of the hardware, not just the model, distinguishing a data or code problem from a compiler or device-level one
- Translate a request for a Trainium job into an actual working, cost-aware training pipeline, end to end
- Tune distributed training runs for throughput and cost on SageMakers training infrastructure
- Work directly with client and internal engineering teams to scope and deliver real production training workloads, not experiments that stay in a notebook
What Youll Bring
- Strong, hands-on PyTorch experience, ideally including distributed or multi-device training
- Production experience with Amazon SageMaker for training and/or inference
- Comfort working close to the hardware layer: you understand device-specific compilation and can debug issues that are actually about the accelerator, not just the model
- AWS Trainium or Inferentia (Neuron SDK) experience is a strong plus; if you dont have it yet but have deep PyTorch and a track record of picking up new hardware targets fast, we want to talk to you
- Solid Python fundamentals and comfort operating in a client-facing, production engineering environment
Similar remote jobs
More like this →
Murmuration
Manager, Software Engineering
Remote
New York, NY$196k–$196k✓ From careers page· about 1 hour ago
Gemba Advantage
DevOps Engineer, Platform Engineering
Remote
London, UK✓ From careers page· about 2 hours ago
Discover More than 100,000 Hidden Remote Jobs Before Everyone Else
Unlock All Remote Jobs Today
Simple pricing. Big savings on Quarterly and Yearly.
Monthly Access
$19/month
- Instant access to fresh remote jobs from 500+ companies
- New opportunities added hourly, often 3-7 days before anywhere else
- Advanced filtering by role type, stack, pay, and location
- Priority customer support
Most Popular
Yearly Access
$59/year
- Everything in Monthly
- Save $169 (~74%) vs paying monthly
- Average job search takes ~6 months - get covered for the whole journey
- Less than the cost of one lunch per month for competitive advantage
- Equivalent to just ~$4.92/month