AI Research Engineer (Kernel & Inference Optimization) - 100% Remote Worldwide

As an AI Research Engineer on Tether's AI model team, you will drive innovation in model serving and inference architectures for advanced AI systems. Your work focuses on optimizing model deployment and inference strategies to deliver highly responsive, efficient, and scalable performance across real-world applications. You will work on a wide spectrum of systems, ranging from resource-efficient models designed for limited hardware environments to complex, multi-modal architectures that integrate text, images, and audio.

What you'll do

• Design and deploy state-of-the-art model serving architectures that deliver high throughput and low latency while optimizing memory usage. Ensure these pipelines run efficiently across diverse environments, including resource-constrained devices and edge platforms, with clear performance targets such as reduced latency, improved token response, and minimized memory footprint.
• Build, run, and monitor controlled inference tests in both simulated and live production environments. Track key performance indicators such as response latency, throughput, memory consumption, and error rates, with special attention to metrics specific to resource-constrained devices. Document iterative results and compare outcomes against established benchmarks to validate performance across platforms.
• Identify and prepare high-quality test datasets and simulation scenarios tailored to real-world deployment challenges, specifically those encountered on low-resource devices. Set measurable criteria to ensure that these resources effectively evaluate model performance, latency, and memory utilization under various operational conditions.
• Analyze computational efficiency and diagnose bottlenecks in the serving pipeline by monitoring both processing and memory metrics. Address issues such as suboptimal batch processing, network delays, and high memory usage to optimize the serving infrastructure for scalability and reliability on resource-constrained systems.
• Work closely with cross-functional teams to integrate optimized serving and inference frameworks into production pipelines designed for edge and on-device applications. Define clear success metrics such as improved real-world performance, low error rates, robust scalability, and optimal memory usage, ensuring continuous monitoring and iterative refinements for sustained improvements.

What you bring

• A degree in Computer Science or a related field. Ideally a PhD in NLP, Machine Learning, or a related field, complemented by a solid track record in AI research and development with publications in top-tier conferences.
• Knowledge of Metal Shading Language (MSL) and the ability to write custom compute shaders from scratch.
• Proven experience in low-level kernel optimizations and inference optimization on mobile devices, with contributions that led to measurable improvements in inference latency, throughput, and memory footprint for domain-specific applications, particularly on resource-constrained devices and edge platforms.
• A deep understanding of modern model serving architectures and inference optimization techniques, including state-of-the-art methods for achieving low-latency, high-throughput performance, and efficient memory management in diverse, resource-constrained deployment scenarios.
• Strong expertise in writing GPU kernels for mobile devices (smartphones) and a deep understanding of model serving frameworks and engines. Practical experience developing and deploying end-to-end inference pipelines, from optimizing models for efficient serving to integrating solutions on resource-constrained devices.
• Demonstrated ability to apply empirical research to overcome challenges in model serving, such as latency optimization, computational bottlenecks, and memory constraints. Proficiency in designing robust evaluation frameworks and iterating on optimization strategies to continuously push the boundaries of inference performance and system efficiency.
• Knowledge of distributed inference systems, including techniques like Tensor Parallelism, Pipeline Parallelism, and Expert Parallelism to handle massive models on GPU clusters.
• Deep understanding of the mathematics and structure behind Diffusion Models and Vision Transformers.
• Understanding of Pruning, Quantization, Flash Attention, KV Cache, and Speculative Decoding (Eagle).

What we offer

• 100% remote work worldwide

About Tether

Tether is a global fintech leader pioneering digital finance through reserve-backed tokens across blockchains. The company operates multiple divisions including Tether Finance (home to USDT, the world's most trusted stablecoin), Tether Power (sustainable Bitcoin mining and energy solutions), Tether Data (AI infrastructure including KEET, a secure data-sharing platform), Tether Education (digital learning for the gig economy), and Tether Evolution (pushing technological innovation).