Log inPost your job
← All jobs
BY

Senior Recommendation System Engineer

BybitKuala Lumpur, Malaysia
EngineeringSeniorOn-site

As a Senior Recommendation System Engineer at Bybit, you own the full pipeline of a high-concurrency, low-latency recommendation engine serving over 80 million users across more than 200 countries and regions. You drive the architecture, optimization, and real-time processing that power multi-channel recall, ranking, and personalization across Bybit's trading, payments, and Web3 ecosystem.

What you'll do

  • Own the development and refactoring of high-concurrency, low-latency recommendation serving engines across the full pipeline: multi-channel recall (two-tower, collaborative filtering, ANN vector retrieval), coarse ranking, fine ranking, and re-ranking.
  • Implement dynamic compute trimming and degradation mechanisms for dynamic UI personalization and global strategy dispatch, ensuring core engine stability under extreme traffic spikes.
  • Build high-throughput, low-latency real-time feature streams on Kafka and Flink, enabling minute-level and second-level user behavioral feature updates with dynamic sliding-window aggregations.
  • Contribute to the development of unified online and offline feature storage with stream-batch convergence architecture, solving industrial-grade consistency issues such as online-offline feature inconsistency and feature time-travel leakage.
  • Own the construction and optimization of large-scale vector retrieval systems (Faiss, Milvus, NSW) supporting candidate pools from tens of thousands to millions, tuning index structure parameters to achieve P99 retrieval latency under 100ms.
  • Build unified Embedding Pipelines and high-performance inverted index services covering trading products, news, KOL content, on-chain signals, and other heterogeneous data sources.
  • Own high-performance online deployment and operator optimization of complex deep ranking models such as DIN and SIM sequential models, and MMoE and PLE multi-objective deep models.
  • Solve compute explosion in multi-task and multi-objective online inference through optimization techniques (quantization, graph optimization, batching strategies), targeting fine-ranking P99 latency under 200ms.
  • Ensure global recommendation service P99 latency under 200ms and system availability above 99.9%, writing comprehensive overload protection, thread isolation, and disaster recovery degradation code.
  • Build and maintain end-to-end distributed tracing and monitoring systems (e.g., Jaeger, Prometheus) across multi-region, multi-site, multi-language cross-border environments, establishing minute-level root cause attribution and fault localization mechanisms.
  • Contribute to A/B experiment traffic-splitting platform upgrades, implementing CUPED variance reduction and sequential testing mechanisms to reduce sample size requirements and accelerate algorithm iteration pipelines.

What you bring

  • 5+ years of recommendation system engineering at consumer-scale internet companies, with deep participation in or leadership of architecture refactoring or launches of real-time recommendation systems serving tens of millions of users at scale. Proven experience building recommendation systems from 0→1 is strongly preferred.
  • Exceptionally solid low-level computer science fundamentals; proficiency in at least one of Go, Java, or C++ (Go preferred given current stack, C++ experience a plus for future engine optimization).
  • Familiarity with PyTorch and TensorFlow model online inference and deployment optimization.
  • Hands-on experience with the Spark, Flink, and Kafka stack, with real experience solving stream computing latency and data backlog issues.
  • Proficiency in Milvus and Faiss cluster deployment and tuning.
  • Deep understanding of computational complexity and online bottlenecks of core algorithms (collaborative filtering, two-tower recall, multi-objective optimization with MMoE and PLE).
  • Ability to interface smoothly with algorithm teams for high-quality, efficient engineering translation.
  • Proven experience in actual development and core module design of recommendation platforms, feature platforms, experimentation platforms, or high-performance RPC frameworks at top-tier companies.

What we offer

  • Study Growth Fund to support your professional development and continuous learning.
  • Internal events including team-building activities, workshops, and innovation sessions.
  • Global collaboration within a diverse, international team.
  • Career advancement opportunities within a rapidly expanding global company.
  • Internal mobility and long-term development pathways to help build your career with us.

About Bybit

Bybit is one of the world's leading cryptocurrency exchanges and digital financial platforms, established in 2018 and serving over 80 million users across more than 200 countries and regions. The platform delivers a seamless ecosystem spanning trading, payments, wealth management, custody, institutional services, and Web3, recognized as one of the most trusted and transparent platforms in the digital asset industry.

What engineering roles in crypto pay

383 salaries · our own data
median $205k$157k$250k

Most engineering roles in crypto pay between $157k and $250k, with a median of $205k.

Senior Recommendation System Engineer | CryptoJobsHQ