fix(train): support eval-only mode (--num-rollout 0)

[EazyReal]

Desired vs bug

Eval-only is intended. train.py already has:

# special case for eval-only
if args.num_rollout == 0 and args.eval_interval is not None:
    ray.get(rollout_manager.eval.remote(rollout_id=0))

The train loop is range(start_rollout_id, num_rollout), so num_rollout == 0 correctly does zero training steps. That part is working as designed.

The crash is a bug in setup order. create_training_models (and thus Megatron's LR scheduler) runs before that special case, because eval still needs the actor loaded and update_weights() into the rollout engines. Megatron asserts lr_decay_steps > 0. The scheduler is never stepped in eval-only; it just has to construct.

This is not divide-by-zero. global_batch_size is fine. The estimate is

args.train_iters = args.num_rollout * args.rollout_batch_size * args.n_samples_per_prompt // args.global_batch_size

which is 0 when num_rollout == 0. Then lr_decay_iters defaults to train_iters, so lr_decay_steps == 0, and optimizer init aborts. You never reach the eval special case.

Change

if args.num_rollout == 0:
    args.train_iters = 1
else:
    args.train_iters = (
        args.num_rollout * args.rollout_batch_size * args.n_samples_per_prompt // args.global_batch_size
    )

train_iters = 1 is only a dummy length so Megatron will construct the scheduler. It does not run a training step. Any num_rollout > 0 keeps the original formula.

Skipping optimizer/scheduler entirely for eval-only would also work, but that is a larger split of create_training_models. This keeps the existing bring-up path.

Test plan

• --num-rollout 0 --eval-interval 1 gets past optimizer init and hits rollout_manager.eval
• A normal training config still sees the same train_iters as before this PR

[EazyReal]

@zhuzilin could you review this one? Eval-only mode with --num-rollout 0 still constructs the Megatron optimizer scheduler, which rejects zero lr_decay_steps; this keeps the training loop at zero rollouts while giving the scheduler the smallest valid shape.

[EazyReal]

@zhuzilin refreshed on latest main and checks are green. This fixes eval-only runs with --num-rollout 0 so evaluation jobs can run without requiring rollout generation. Could you review when you have bandwidth?

[EazyReal]

Closing in favor of #2296.

train_iters = 1 was only a dummy so Megatron's scheduler constructor would accept eval-only. Eval-only does not train, so it should not build the optimizer or LR scheduler. #2296 skips that stack and still loads actor weights for rollout_manager.eval.