Skip to content

Numerical instability in log‑based loss functions #36

Description

@github-actions

File: leanpass/nn.py

Functions Tensor.log, cross_entropy_loss and binary_cross_entropy_loss compute np.log of probabilities that can be exactly zero, producing -inf or nan gradients. This is a common source of crashes during training.

Fix: clamp inputs with a small epsilon (e.g., eps = 1e-12) before taking the log, and document the behavior.

Label: enhancement

Filed automatically by ai-issue-scan.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions