Staff Engineer - Distributed Systems

Team: CRM & Automation

Location: India

Commitment: Employee India

Workplace Type: remote

What are we hiring for?

Responsibilities:

Own the architecture health of a billion-scale distributed system, its failure modes, capacity limits, consistency guarantees, and the interactions between 50+ deployments that no single team can see

Approve critical-path designs. Changes that touch the systems core go through you, not as bureaucracy, but as the person accountable for the whole staying sound. When theres a disagreement, you make your case on merit

Hunt gaps proactively, single points of failure, unbounded queues, missing idempotency, thundering herds, quiet data-loss windows, and drive the fixes before they become incidents

Build the parts nobody else can. No sprint tickets. You prototype the risky architectural bets yourself, ship the remediation after serious incidents, and pair into the gnarliest cross-team bugs, roughly a quarter to a third of your time in code, all of it on the hardest problems

Make resilience a property of the system, not a heroic act, degradation strategies, backpressure, isolation boundaries, capacity models that survive 10%+ month-over-month growth

Raise the teams around you. Design reviews that teach, post-mortems that change architecture (not just add alerts), and patterns that 80+ engineers build on

Set the standard for how AI-assisted engineering works safely on systems this critical, where a bad merge doesnt cost a demo, it costs real businesses their revenue

The terrain:

Runtime: Node.js (TypeScript), Go, thousands of pods on GKE

Messaging & async: GCP Pub/Sub, Cloud Tasks, Redis

Storage: MongoDB, Firestore, ClickHouse, ElasticSearch

Scale: 21.5B automation actions/month, 2.6B messages/month, 28.5K req/s peaks, ~226B async events across the org

Requirements:

• 10+ years of engineering experience with deep, hands-on ownership of large-scale distributed systems, comparable scale strongly preferred: hundreds of services or thousands of instances, billions of daily events

• Youve carried sole accountability for a production system through real failures, not adjacent to it, not advising on it. Owned it

• Deep command of queueing and async architectures, delivery semantics, ordering, backpressure, idempotency, exactly-once myths and at-least-once realities

• Strong with multiple storage engines (SQL and NoSQL), you reason about consistency models, indexing at scale, and when each engine is the wrong choice

• Expert-level depth in Redis or comparable in-memory systems, including their failure modes under memory pressure and network partition

• Production experience on Kubernetes at scale, resource limits, autoscaling behavior, what actually happens when a node pool dies

• Exceptional design communication, docs, diagrams, and RCAs that drive decisions across multiple teams

• Fluent in Node.js and/or Go, enough to prototype your own proposals and ship fixes on the critical path

What Success Looks Like

• The critical paths of Workflows have named owners, capacity models, and tested failure modes, because you made it so

• Incident count trends down while traffic grows double-digit percent month over month

• Your design reviews are the ones engineers want their proposals to survive

• Your scope has expanded on results, more systems, more surface, more trust

 

What youre built for

• Youve been injured in production. A lot. Youve owned distributed systems at serious scale, through the outages, the migrations, the 3 AM discoveries, and every scar changed how you design

• Youve operated at staff scope, whatever your title said, the engineer everyone routed the hardest systems questions to

• You think in failure modes by default: when you see a design, you instinctively ask what happens at the tail, under partition, at 10x load

• You write design docs and RCAs that people reference years later, clear trade-offs, honest risks, real recommendations

• You can disagree with a team and still make them better, influence through rigor and respect, not title

• Youd rather prevent ten incidents quietly than be the hero of one loudly

Bonus Points

• Youve made AI agents genuinely productive on complex systems, and know how to keep AI-generated code from becoming AI-generated incidents

• GCP-native experience: Pub/Sub, Cloud Tasks, GKE, Firestore

• Youve done this job before under another name, the systems person, principal engineer, architect-who-still-codes

• Experience taking a 0→1 system to production alongside hardening mature ones