nonnest2 사용하여 모델 간 비교 기능 보완 #48

Activity

1. seonghobae commented on Feb 14, 2026

   

   seonghobae

   on Feb 14, 2026

   CollaboratorAuthor

   More actions

   Execution checklist

   • Identify current model-selection insertion points in aefa() flow
   • Define nonnest2 as optional auxiliary diagnostics (not default selector)
   • Add optional dependency strategy (Suggests + runtime fallback)
   • Define compatibility checks and graceful fallback messages
   • Add regression tests for installed/not-installed nonnest2 paths

   Acceptance criteria

   • Default AIC/BIC/DIC selection behavior remains unchanged
   • nonnest2 path never hard-fails when package/model constraints are unmet
   • Results clearly separate primary selector vs auxiliary non-nested comparison
2. opencode-agent

   added

   priority: mediumNormal-priority or P2 workNormal-priority or P2 work

   status: triagedOpen issue has an organization taxonomy assignmentOpen issue has an organization taxonomy assignment

   type: featureNew or expanded product capabilityNew or expanded product capability

   on Aug 22, 2026
3. seonghobae commented on Sep 6, 2026

   

   seonghobae

   on Sep 6, 2026

   CollaboratorAuthor

   More actions

   ConceptWeave Research Intake consumer evidence, 2026-09-06

   This existing integration issue is the appropriate owner location for the following acceptance requirements; no additional numerical implementation or repository is being created by the consumer.

   Observed protected sources:

   • kaefa develop@5128d4867e24b5db73e6e3c8652a8dbeabd70aa0 describes its search as greedy/heuristic. Its current criterion helper rejects unavailable DIC instead of substituting AIC:

     kaefa/R/kaefa.R

     Lines 440 to 543 in 5128d48

     	#' Extract a scientifically valid information criterion from a mirt fit
     	#'
     	#' DIC is a posterior-deviance criterion (Spiegelhalter et al., 2002) and
     	#' therefore cannot be reconstructed from a maximum-likelihood AIC value.
     	#' AICc is reconstructed only from its original small-sample correction
     	#' (Hurvich and Tsai, 1989): AIC + 2 k (k + 1) / (n - k - 1).
     	#'
     	#' @param fit The model's named fit-statistic list.
     	#' @param criterion Requested information criterion.
     	#' @param sample_size Number of response patterns used to fit the model.
     	#' @return One finite information-criterion value.
     	#' @keywords internal
     	#' @noRd
     	.aefaFitCriterionValue <- function(fit, criterion, sample_size) {
     	criterion <- toupper(criterion)
     	fit_value <- function(name) {
     	value <- fit[[name]]
     	if (length(value) == 1L && is.numeric(value) && is.finite(value)) {
     	return(as.numeric(value))
     	}
     	NULL
     	}
     	if (criterion == "DIC") {
     	value <- fit_value("DIC")
     	if (is.null(value)) {
     	stop(
     	paste0(
     	"DIC is unavailable: DIC requires posterior deviance draws and an ",
     	"effective parameter count, but this mirt ML/MAP fit does not supply ",
     	"a DIC value. Use AIC, AICc, BIC, or saBIC; DIC is never replaced by AIC."
     	),
     	call. = FALSE
     	)
     	}
     	return(value)
     	}
     	if (criterion == "CAIC") {
     	stop(
     	"CAIC is ambiguous and is not an alias for corrected AIC; use 'AICc' explicitly.",
     	call. = FALSE
     	)
     	}
     	if (criterion == "AICC") {
     	value <- fit_value("AICc")
     	if (!is.null(value)) {
     	return(value)
     	}
     	aic <- fit_value("AIC")
     	log_likelihood <- fit_value("logLik")
     	if (is.null(aic) || is.null(log_likelihood)) {
     	stop(
     	"AICc is unavailable because the fit supplies neither AICc nor both AIC and logLik.",
     	call. = FALSE
     	)
     	}
     	if (length(sample_size) != 1L || !is.numeric(sample_size) ||
     	!is.finite(sample_size) || sample_size <= 0) {
     	stop("AICc requires one finite positive sample size.", call. = FALSE)
     	}
     	parameter_count <- (aic + 2 * log_likelihood) / 2
     	if (!is.finite(parameter_count) || parameter_count < 0) {
     	stop("AICc could not recover a valid parameter count from AIC and logLik.", call. = FALSE)
     	}
     	if (sample_size <= parameter_count + 1) {
     	stop(
     	paste0(
     	"AICc is undefined because n (", sample_size,
     	") must be greater than k + 1 (", parameter_count + 1, ")."
     	),
     	call. = FALSE
     	)
     	}
     	return(aic + (2 * parameter_count * (parameter_count + 1)) /
     	(sample_size - parameter_count - 1))
     	}
     	fit_name <- switch(
     	criterion,
     	AIC = "AIC",
     	BIC = "BIC",
     	SABIC = "SABIC",
     	NULL
     	)
     	if (is.null(fit_name)) {
     	stop(
     	"Unsupported model selection criterion. Use AIC, AICc, BIC, saBIC, or model-supplied DIC.",
     	call. = FALSE
     	)
     	}
     	value <- fit_value(fit_name)
     	if (is.null(value)) {
     	stop(
     	paste0(criterion, " is unavailable because the fitted model does not supply a finite value."),
     	call. = FALSE
     	)
     	}
     	value

   • CWL nonnest2 master@b62bf9ac928988a4b988fc3efb0adfb88549fef2 explicitly requires identical modeled variables and observation ordering, and documents that these are not checked. Its object checker does not establish those conditions: https://github.com/ContextualWisdomLab/nonnest2/blob/b62bf9ac928988a4b988fc3efb0adfb88549fef2/R/vuongtest.R#L15-L20 and https://github.com/ContextualWisdomLab/nonnest2/blob/b62bf9ac928988a4b988fc3efb0adfb88549fef2/R/vuongtest.R#L367-L415

   Before auxiliary comparison evidence is consumed, add owner regression cases for reordered or different observation identities, different outcome support, asymmetric missingness and unsupported sampling designs. Preserve distinguishability, relative fit, failed/unsupported comparisons and their denominators separately. The earlier graceful-fallback criterion should mean an explicit unavailable result, never a passing comparison, zero cost/error or an invented confidence score. A comparison p-value is not an ontology relevance weight or semantic approval.

   Current kaefa PR #79 (OPEN, head 1c5d9f0491fc178be3f7f307dac521fbcbba6978) owns a proposed RMSE protocol. nonnest2 PR #126 (OPEN Draft, head efca0b8534abc0ebc7211c8c2025bfaf36a67fec) concerns public option validation; it is not by itself evidence of paired-observation validation. Preserve those owner lanes and recheck their exact heads rather than copying their source or test claims.

   This is a source audit, not a reproduced numerical failure or an R test result. The three inspected statistical repositories have no GitHub release/tag entries; CRAN publication/provenance was not verified. ConceptWeave requires released contracts, compatible licensing and Rust-first production computation at the canonical measurement owner before adoption. No change to scientific equations, default selection, release publication or branch protection is authorized by this comment.
4. seonghobae

   added

   enhancement

   on Sep 7, 2026