Skip to content

Move wolfSSL_Init() call higher up to fix Segfault on Win - #301

Open
lealem47 wants to merge 1 commit into
wolfSSL:mainfrom
lealem47:wolfcrypt_init
Open

lealem47 wants to merge 1 commit into
wolfSSL:mainfrom
lealem47:wolfcrypt_init

Conversation

@lealem47

@lealem47 lealem47 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor
  • Fix for Segfault seen on running wolfCLU on Windows with FIPS because the call to wc_InitRng() early in main() would attempt to use a mutex that hadn't been created yet. Calling wolfSSL_Init() before using wc_InitRng properly initializes the mutex
  • Include stdint.h in clu_header_main.h so that uint8_t is available for wolfssl/test.h. Proper fix in wolfssl would be to use a byte there.
  • Moved around some fips hash logging so that it's no longer dead code.

@lealem47 lealem47 self-assigned this Oct 7, 2026
Copilot AI balanced review requested due to automatic review settings October 7, 2026 17:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Successful initialization overwrites the command success status, causing non-FIPS help commands to exit with failure.

1 open finding
What changed in this PR

Addresses the Windows FIPS startup crash by initializing wolfCrypt before RNG setup.

Changes:

  • Adds early wolfCrypt initialization with failure handling.
  • Includes <stdint.h> on Windows to provide uint8_t.
File Description
wolfclu/​clu_header_main.h Provides fixed-width integer types on Windows.
src/​clu_main.c Initializes wolfCrypt before FIPS RNG setup.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/clu_main.c Outdated
@lealem47 lealem47 changed the title Add wolfCrypt_Init() call to fix Segfault on Win Move wolfSSL_Init() call higher up to fix Segfault on Win Oct 7, 2026

@sebastian-carpenter sebastian-carpenter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  HIGH-1: FIPS error callback is now registered after wolfSSL_Init(), so integrity-hash and self-test failures are no longer reported [BLOCK] (bug)
  File:       src/clu_main.c:141-158
  Function:   main
  Confidence: High

  The diff moves `wolfSSL_Init()` ahead of `wolfCrypt_SetCb_fips(myFipsCb)`. wolfCLU always builds with OPENSSL_EXTRA, and in that configuration `wolfSSL_Init()` creates the global DRBG. The call chain is `wolfSSL_RAND_seed(NULL, 0)` → `wolfSSL_RAND_Init()` → `wc_InitRng(&globalRNG)`, at wolfSSL `src/ssl.c:2553-2557` and `src/ssl_crypto.c:3628`. In a FIPS build, `fips.h` maps `wc_InitRng` to `wc_InitRng_fips`. That function calls `FipsAllowed()` first, which invokes `errCb(0, posReturn, base16_hash)` only if a callback is already registered. On DRBG CAST failure, `AlgoAllowed(FIPS_CAST_DRBG)` → `EnterDegradedMode()` likewise only reports through `errCb` when one is set.

Before this change the order was: register `myFipsCb`, call `wc_InitRng(&rng)`, then the callback prints the hash with "copy above hash into verifyCore[] in fips_test.c and rebuild", then wolfCLU logs "Err %d, update the FIPS hash".

After this change, when the in-core integrity check fails, `wolfSSL_Init()` fails first while `errCb == NULL`. `main` then prints only "wolfSSL initialization failed!" and returns -1. `myFipsCb`, the "update the FIPS hash" message and the later `wolfCrypt_GetStatus_fips() == IN_CORE_FIPS_E` notice can no longer run in the case they were written for. Updating the FIPS hash is the usual first step when bringing up a FIPS build, including the Windows FIPS DLL that this commit targets. Users get no hash and no hint about the cause. wolfSSL's own `wolfcrypt/test/test.c:3919-3922` registers `wolfCrypt_SetCb_fips` before `wolfCrypt_Init()` for this reason.

  Code:
        if (wolfSSL_Init() != WOLFSSL_SUCCESS) {      /* seeds global DRBG via wc_InitRng_fips */
            wolfCLU_LogError("wolfSSL initialization failed!");
            return -1;                                 /* FIPS POS failure exits here, errCb still NULL */
        }
    
    #ifdef HAVE_FIPS
        wolfCrypt_SetCb_fips(myFipsCb);                /* registered too late */

  Recommendation: Leave `wolfSSL_Init()` early, as this fix intends, but move `wolfCrypt_SetCb_fips(myFipsCb)` and the `wc_SetSeed_Cb` call above it. Both are plain setters and safe to call before init. That way the first FIPS-wrapped call, which is now `wolfSSL_Init()`, reports POS and CAST failures through `myFipsCb` again. Keep the `wc_InitRng(&rng)` probe after init.
  LOW-2: FIPS RNG-failure early return now skips wolfSSL_Cleanup() [NIT] (bug)
  File:       src/clu_main.c:153-158
  Function:   main
  Confidence: High

  Before this diff, the `return ret;` after a failed `wc_InitRng(&rng)` ran before `wolfSSL_Init()`, so there was nothing to clean up. `wolfSSL_Init()` now runs first, so this early return leaves wolfSSL initialized and skips the `wolfSSL_Cleanup()` that normal exit runs at line 363. That means the init refcount, global mutexes and global DRBG are never released. A normal process exit hides this. On FREERTOS, however, `main` is renamed `clu_main` and called again for every UART command by `clu_entry()`. There the refcount and resources build up across commands.

  Code:
        ret = wc_InitRng(&rng);
    
        if (ret != 0) {
            wolfCLU_LogError("Err %d, update the FIPS hash\n", ret);
            return ret;          /* wolfSSL_Init() already succeeded; no wolfSSL_Cleanup() */
        }

  Recommendation: Call `wolfSSL_Cleanup()` before the early return so it mirrors the init that now comes first.

@sebastian-carpenter sebastian-carpenter removed their assignment Oct 7, 2026

@sebastian-carpenter sebastian-carpenter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  MEDIUM-1: wolfSSL_Debugging_ON() now runs after wolfSSL_Init(), so DEBUG_WOLFSSL builds lose the init trace [SUGGEST] (bug)
  File:       src/clu_main.c:146-171
  Function:   main
  Confidence: High

  Before this diff, non-FIPS builds called `wolfSSL_Debugging_ON()` first and `wolfSSL_Init()` after it. The diff moves `wolfSSL_Init()` up to line 146 but leaves the `#ifdef DEBUG_WOLFSSL` block at lines 169-171. In a DEBUG_WOLFSSL build, everything `wolfSSL_Init()` logs is now dropped. That includes `WOLFSSL_ENTER("wolfSSL_Init")`, "Bad wolfCrypt Init", "wolfSSL_RAND_seed failed", "Bad Init Mutex ..." and the wolfCrypt/FIPS DRBG messages from `wc_InitRng(&globalRNG)`. Those are exactly the messages you need when the new `wolfSSL initialization failed!` branch fires. `wolfSSL_Debugging_ON()` only sets `loggingEnabled = 1` (wolfcrypt/src/logging.c) and does not need the library to be initialized, so there was no reason to move it below Init. The FIPS path is affected too: the FIPS POST that the removed `wc_InitRng(&rng)` used to trigger now runs inside `wolfSSL_Init()`, which comes before debugging is switched on.

  Code:
        if (wolfSSL_Init() != WOLFSSL_SUCCESS) {
            wolfCLU_LogError("wolfSSL initialization failed!");
            ...
            return -1;
        }
    
        if (argc == 1) {
            ...
        }
    
    #ifdef DEBUG_WOLFSSL
        wolfSSL_Debugging_ON();
    #endif

  Recommendation: Move the `#ifdef DEBUG_WOLFSSL wolfSSL_Debugging_ON(); #endif` block above the `wolfSSL_Init()` call so the init and FIPS self-test trace is still printed in debug builds.
  LOW-2: Mismatched preprocessor indentation and a redundant newline in the moved FIPS error message [NIT] (style)
  File:       src/clu_main.c:148-160
  Function:   main
  Confidence: High

  The new FIPS diagnostic block opens with `#ifdef HAVE_FIPS` at column 0 (line 148) and closes with `    #endif` indented (line 160). The block just above (lines 138-144) uses the same nesting and keeps each pair aligned. Also, the moved `wolfCLU_LogError("Err %d, update the FIPS hash\n", ...)` keeps its trailing `\n`. `DefaultLoggingCb` already appends `\r\n`, so this prints an empty line in the middle of the diagnostic. None of the other WOLFCLU_LOG lines in the block end with `\n`.

  Code:
    #ifdef HAVE_FIPS
            if (wolfCrypt_GetStatus_fips() == IN_CORE_FIPS_E) {
                wolfCLU_LogError("Err %d, update the FIPS hash\n", IN_CORE_FIPS_E);
                ...
            }
        #endif

  Recommendation: Indent the `#endif` the same as its `#ifdef` and drop the trailing `\n` from the error string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants