Staff Site Reliability Engineer - Dev Platform

As a Staff Site Reliability Engineer on the Dev Platform team at Kraken, you own the reliability, availability and performance of the AI Gateway, a production-critical service used across the company. The AI Gateway is Kraken's company-wide LLM gateway: a single, governed entry point that gives teams self-service access to leading models from providers such as Anthropic, OpenAI and Google. You set technical direction for the gateway's architecture, capacity planning and scaling as LLM traffic grows.

What you'll do

• Own the reliability, availability and performance of the AI Gateway as a production-critical service, including SLOs, error budgets and on-call practices
• Design the gateway's resilience: multi-provider routing, failover, retries, rate limiting and graceful degradation when an upstream model provider is slow or unavailable
• Set the technical direction for gateway architecture, capacity planning and scaling as LLM traffic grows and more workloads move onto the gateway
• Build observability for LLM traffic, covering latency, errors, token usage and cost, attributed by team and use case
• Partner with Security, Finance and AI governance stakeholders to put access controls, budgets, audit logging and data-handling requirements into the platform
• Lead incident response and blameless post-incident reviews, and turn what you learn into lasting reliability improvements
• Automate operations through infrastructure as code, safe deployments and progressive rollouts
• Mentor engineers and raise the operating standard across Dev Platform

What you bring

• Extensive experience in site reliability, infrastructure or platform engineering, with a track record of owning high-traffic, business-critical services in production
• Deep hands-on experience with Kubernetes, a major cloud provider and infrastructure as code such as Terraform
• Strong command of SLOs, observability and incident management, and the judgment to know which reliability work matters most
• Proficiency in a language such as Rust or Python, and comfort reading and contributing to a production service's codebase
• Experience with API gateways, reverse proxies or traffic management: load balancing, rate limiting, failover and caching
• Experience with Consul and Vault for service discovery, configuration and secrets management
• Working knowledge of authentication and authorization mechanisms for internal platforms and APIs, such as OAuth 2.0/OIDC, JWT, mTLS and API keys, and how to apply them to a multi-tenant service
• Experience with model evaluations: building or operating eval pipelines to compare models, prompts and agents, and to catch quality regressions before and after rollout
• Familiarity with coding agents and agent harnesses such as OpenHands and Claude Code, including running them safely in sandboxed environments for engineering teams
• Experience with agent memory systems and the databases behind them, including vector stores, search, relational and key-value stores, and operating them reliably
• Staff-level influence: you set direction across teams, write clear design documents and bring others with you
• Good judgment balancing reliability, cost and developer velocity for internal customers

Nice to have

• Experience operating LLM gateways or integrating LLM provider APIs at scale, including streaming, token-based rate limiting, model routing and caching
• Experience hosting and serving models yourself, including vLLM, and running distributed inference or training workloads with Ray, including GPU scheduling, batching, autoscaling and cost per token
• Experience designing, developing and maintaining Kubernetes controllers and operators with controller-runtime or kubebuilder, including CRDs, reconciliation loops and safe upgrades
• Experience with cost attribution, budgeting or FinOps for usage-based services
• Background in security-sensitive or regulated environments such as financial services or crypto
• Contributions to open-source gateway, proxy or observability projects

About Kraken

Kraken, founded in 2011, is one of the world's longest-standing crypto platforms, trusted by over 10 million individuals and institutions globally. It offers spot trading, margin, futures, staking and OTC services. Kraken is part of Payward, which operates a range of financial infrastructure platforms including NinjaTrader, Breakout, xStocks, Payward Services and CF Benchmarks.