Embodied AI Toolchain
RLinf
Embodied AI Infra
A large-scale reinforcement learning framework
with integrated simulation, training, and deployment for embodied intelligence
Why Is Embodied Reinforcement Learning Today Harder, Slower, and More GPU-Intensive?
Severe compute waste
Even with multiple GPUs, provisioned, a large portion of resources remain idle, resulting in low overall utilization.
GPU memory contention
Simulators and rendering workloads compete with model training for GPU memory, leading to inflexible resource allocation and scheduling bottlenecks.
Sim-to-Real complexity
The physical world cannot be accelerated. Real-world systems and simulation environments remain fragmented, making transfer and adaptation costly and difficult.
Rigid resource scheduling
Traditional systems are either fully shared or fully isolated, making them poorly suited to the heterogeneous workloads required by embodied intelligence.
Active community and SOTA achievements
Released under the Apache-2.0 license with high-frequency daily updates driven by a global developer ecosystem.
1.5B/7B
Mathematical reasoning model support (SOTA)
10+
Supported algorithms (PPO/SAC/GRPO)
Daily
Code update frequency
Full
cl/test coverage
RLinf Framework Overview and Core Advantages
RLinf is jointly developed by Tsinghua University, Zhongguancun Academy, and INFINIGENCE, in collaboration with leading institutions including Peking University and UC Berkeley.
RLinf was officially open-sourced in September 2025. Within three months of release, it gained over 4,000 GitHub stars and quickly became one of the most active reinforcement learning frameworks. It has since been adopted by multiple well-known research teams in embodied intelligence across both academia and industry.
A Complete Stack from User to Hardware
User layer
Worker-based unified programming interface
Task layer
Reinforcement learning (PPO, GRPO...)
Simulation engine
Training engine
Inference engine
Low-intrusion component
encapsulation
Execution layer
Flexible execution modes
Shared mode
Separated mode
Quick installation
/uninstallation
Fine-grained pipeline
Hybrid mode
Scheduling layer
Automated scheduling
Dynamic scaling
mechanism
Automatic scheduling
Strategy
Communication layer
Adaptive communication library
Adaptive
CUDAIPC/
NCCL communication
Multi-channel Communication
Load balancing
enhanced queues
Fast communication
reconfiguration
Hardware layer
Cluster
(CPU,GPU)
Advantages
Hybrid execution mode
RLinf’s hybrid execution mode combines the strengths of both isolated and shared execution models. Together with a fine-grained pipeline design, this approach minimizes idle time and significantly improves overall system efficiency.
Low-intrusive multi-backend integration solution
RLinf integrates two backends:
Megatron + SGLang/vLLM: Designed for converged embodied "brain" models such as VLMs, enabling high-performance training on large-scale clusters.
FSDP + Hugging Face: Designed for non-converged embodied "cerebellar models" such as VLAs, providing out-of-the-box compatibility with Hugging Face models and lowering the development barrier.
Adaptive communication library and automated scheduling module
RLinf includes a reinforcement-learning–specific communication library and automated scheduling strategy. Communication optimizations include adaptive CUDA IPC and NCCL communication, load-balanced transmission queues, multi-channel concurrent communication, and fast communication reconfiguration. Combined with automated scheduling, RLinf selects the optimal execution mode based on the user’s training workflow and available compute resources.
RLinf delivers over 120% training speedup in integrated simulation–training–deployment workloads.
Tested using FSDP + HuggingFace backend
Embodied training performance:RLinf delivers over 120% training speedup in integrated simulation–training–deployment workloads.
When OpenVLA-OFT was trained using RLinf’s embodied GRPO algorithm, the average success rate reached 97.3%.
Achieved SOTA performance on all three test datasets
Tested using the Megatron + SGLang backend:
Inference performance：RLinf-math-1.5B and RLinf-math-7B achieved SOTA performance on all three test datasets.
(Note: All evaluated models are open-source models from Hugging Face and were tested using a unified evaluation script: https://github.com/RLinf/LLMEvalKit)
Advantages
Next-generation infrastructure purpose-built for integrated simulation–training–deployment
Core Dimension
Execution modes
GPU memory utilization
Embodied simulation support
Hardware/model adaptation
Scheduling flexibility
RLinf（This Solution）
Support dynamic switching between collocated, disaggregated, and hybrid modes
Very high (hybrid scheduling resolves memory contention)
Deeply optimized (IsaacLab, ManiSkill3, etc.)
Out-of-the-box (OpenVLA, π0, Franka)
Macro-to-micro flow switching
Traditional RL Frameworks (e.g., CleanRL)
Limited to single-node or simple distributed setups
Low (simulators and models compete for GPU memory)
Require manual adaptation with high integration effort
No built-in support
Static configuration
General-Purpose Cloud-Native Solutions
Typically fixed pipelines with limited flexibility
Moderate (often affected by resource fragmentation)
General support without embodied-specific optimization
Dependent on third-party ecosystems
Requires manual orchestration
System Efficiency in Embodied Scenarios
120%+
Embodied VLA Model Performance Gain
40~60%
3 Records in Math Reasoning
SOTA
Extensive Ecosystem Compatibility
More than a framework — it's the bridge between simulation, model, and real-world robot.
Compatible with Mainstream Models（VLA / LLM / World Model）
OpenVLA
OpenVLA-OFT
To/ To.s
GRO0T-N1.5
Qwen2.5-VL
OpenSora
Simulation & Real-Robot Support
Isaaclab
Maniskill3
LIBERO
RoboCasa
MetaWorld
Franka Emika (Real Robot)
RoboTwin (R2S2R)
Built on a rich Example Gallery, RLinf provides turnkey templates for the complete sim-to-real pipeline.
LIBERO + OpenVLA-OFT + GRPO
Achieve 99% Success Rate
Delivering a Stable and Reliable Embodied AI Toolchain
Enterprise-ready scalability with cost efficiency
