fix: treat temperature=0 as greedy selection, untempered logprobs - #2197
Closed
EazyReal wants to merge 3 commits into
Closed
fix: treat temperature=0 as greedy selection, untempered logprobs#2197EazyReal wants to merge 3 commits into
EazyReal wants to merge 3 commits into
Conversation
EazyReal
force-pushed
the
fix/rollout-temp-zero-guard
branch
from
August 18, 2026 23:37
f28ff8a to
9f9bb9f
Compare
Keep the existing != 1.0 guards; dividing by 0 produced infs. Value heads already pass apply_temperature=False.
EazyReal
force-pushed
the
fix/rollout-temp-zero-guard
branch
from
August 21, 2026 00:50
9f9bb9f to
8751e4d
Compare
SGLang greedy decoding still reports finite model log-probs, not a delta at the argmax. Dividing logits by T=0 produced infs; using literal log q=0 would also break on-policy PPO ratios. Share one helper for that convention, skip top-p nucleus replay under greedy, and test that recomputed log-probs match untempered log_softmax. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the helper and the logprob/top-p invariants; remove extra validation, comments, and the cloned GPU e2e. Co-authored-by: Cursor <cursoragent@cursor.com>
2 tasks
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
get_responsesandget_log_probs_and_entropydivide logits byrollout_temperaturewhenever it is not 1.0. Greedy rollout usestemperature=0, so that is a divide-by-zero.SGLang treats T=0 as greedy token selection, but still returns finite untempered model log-probs. slime stores those as
rollout_log_probsand builds PPO ratios fromexp(log π_θ − log π_old). Forcing greedy log-probs to 0 would make an on-policy sample start atr = π_θ(a).Value heads already pass
apply_temperature=False.Change
apply_rollout_temperatureis the shared rule at both logit sites:logits / TT=0 also skips top-p nucleus replay. SGLang remaps T=0 to
top_k=1and reports unmasked untempered log-probs.Test plan
tests/test_rollout_temperature.py— recomputed logprobs matchlog_softmaxat T=0/1 andlog_softmax(logits/T)at T=0.5;get_responsesdoes not scale at 0; greedy skips top-p replaytests/test_value_temperature.py— value head still ignores rollout temperature