Unsafe screening with CDClassifier?

[tomMoral]

Setting shrinking=True in CDClassifier with loss log and penalty l1 seems to not converge toward the optimal solution. Increasing the number of iteration does not change this.

It looks like some coordinates are screened out and never put in again?

Here is a script showing that setting shrinking=True does not converge:

import numpy as np
from lightning.classification import CDClassifier

n_samples = 100
n_features = 5000

rng = np.random.RandomState(42)
X = rng.randn(n_samples, n_features)
y = 2*(rng.randn(n_samples) > 0) - 1

lmbd = 0.1 * abs(X.T.dot(y)).max()



def loss(X, y, lmbd, beta):
    y_X_beta = y * X.dot(beta.flatten())
    return np.log(1 + np.exp(-y_X_beta)).sum() + lmbd * abs(beta).sum()



for shrinking in [True, False]:
    clf = CDClassifier(
        loss='log', penalty='l1', C=1, alpha=lmbd,
        tol=0, permute=False, shrinking=shrinking,
        max_iter=2000)
    clf.fit(X, y)
    print(f"Shrinking: {shrinking}; Loss = {loss(X, y, lmbd, clf.coef_)}")

The output:

Shrinking: True; Loss = 43.29080271832241
Shrinking: False; Loss = 42.165458588261096

[mathurinm]

I have the same issue with CDRegressor in #186