Prerequisites

• I have searched existing issues and reviewed documentation.

Problem Description

Hello, I set up an environment with two nodes, each with 1 GPU, following the Multi-Machine Setup Guide. The environment seems to initialize correctly, but when I try to deploy the model with the command:
sllm-cli deploy --model meta-llama/Llama-3.2-1B --num_gpus=2
I receive the following error:
Error: No available node types can fulfill resource request {'CPU': 1.0, 'worker_id_1': 0.1, 'worker_node': 0.1, 'GPU': 2.0}. Add suitable node types to this cluster to resolve this issue.
It seems that any resources request across nodes cannot be fullfilled. Is this error due to a configuration issue or currently sllm doesn't support it? Thank you for your assistance!

Proposed Solution

Multi-node inference.

Alternatives Considered

No response

Additional Context

No response

Importance

Nice to have

Usage Statistics (Optional)

No response