From a100610d1455fe3d04982d406353abbcaccd3749 Mon Sep 17 00:00:00 2001 From: "robin.bygrave" Date: Fri, 18 Sep 2026 10:32:06 +1200 Subject: [PATCH] Add validateOnStaleSecs to validate idle connections on borrow Introduce a new `validateOnStaleSecs` option controlling the maximum idle age (in seconds) after which a free connection is validated when borrowed from the pool. This is separate from the background heartbeat and catches connections that died between heartbeat cycles before they are handed to the application. Behaviour of validateStaleMillis(): - Explicit validateOnStaleSecs is honoured; 0 disables borrow-time validation. - When unset and the heartbeat is enabled, stale-on-borrow validation remains disabled (unchanged default behaviour). - When unset and the heartbeat is disabled (e.g. AWS Lambda), it defaults to min(100, maxInactiveTimeSecs) seconds instead of the previous maxInactiveTimeSecs + trimPoolFreqSecs. Wires the setting through DataSourceBuilder, DataSourceConfig (copy, setDefaults merge, and properties load) and adds tests. Documents the option in the configuration reference, create-datasource-pool and connection-validation-best-practices guides, and the README Lambda section. --- README.md | 5 +- docs/guides/configuration-reference.md | 1 + .../connection-validation-best-practices.md | 41 ++++++++++ docs/guides/create-datasource-pool.md | 1 + .../ebean/datasource/DataSourceBuilder.java | 14 ++++ .../io/ebean/datasource/DataSourceConfig.java | 25 +++++- .../datasource/DataSourceConfigTest.java | 77 +++++++++++++++++++ 7 files changed, 162 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c3ac1ac..19e7f20 100644 --- a/README.md +++ b/README.md @@ -215,7 +215,10 @@ AWS Lambda runtime). When detected, `validateOnHeartbeat` is automatically set t in the background. This thread is skipped in Lambda to avoid unnecessary CPU costs. - **Connection validation still works:** Dead or stale connections are still detected eagerly when they are - returned to the pool or when you attempt to use them. This ensures the pool remains robust. + returned to the pool or when you attempt to use them. In addition, because the background heartbeat is + disabled, connections idle longer than `min(100, maxInactiveTimeSecs)` seconds are validated when borrowed + from the pool. Set `validateOnStaleSecs` to tune this threshold (or `0` to disable). This ensures the pool + remains robust. - **Why this matters:** Lambda functions are charged per millisecond of execution. Background threads consume CPU time even when the function is idle, directly increasing your Lambda costs. Serverless functions are diff --git a/docs/guides/configuration-reference.md b/docs/guides/configuration-reference.md index 45c3e15..141ce97 100644 --- a/docs/guides/configuration-reference.md +++ b/docs/guides/configuration-reference.md @@ -101,6 +101,7 @@ Property keys are matched case-insensitively. | Builder method | Property key | Default | Description | |----------------|--------------|---------|-------------| | `validateOnHeartbeat(boolean)` | `validateOnHeartbeat` | `true` (`false` in AWS Lambda) | Enable the background heartbeat that validates the pool. | +| `validateOnStaleSecs(int)` | `validateOnStaleSecs` | *(unset)* | Idle age in seconds after which a free connection is validated when borrowed. `0` disables. When unset: disabled while the heartbeat is on, otherwise `min(100, maxInactiveTimeSecs)` (e.g. in AWS Lambda). | | `heartbeatFreqSecs(int)` | *(builder only)* | `30` | How often the heartbeat runs. | | `heartbeatTimeoutSeconds(int)` | `heartbeatTimeoutSeconds` | `30` | Query timeout for the heartbeat validation. | | `heartbeatSql(String)` | `heartbeatSql` | `Connection.isValid()` / platform default | Explicit validation SQL. Rarely needed — see the validation guide. | diff --git a/docs/guides/connection-validation-best-practices.md b/docs/guides/connection-validation-best-practices.md index 5e901f0..72b889e 100644 --- a/docs/guides/connection-validation-best-practices.md +++ b/docs/guides/connection-validation-best-practices.md @@ -114,6 +114,40 @@ If a heartbeat validation takes longer than this timeout, the connection is mark --- +## Validate Stale Connections On Borrow + +Separate from the background heartbeat, the pool can validate a free connection *when it is borrowed* if it has been idle for longer than a threshold. This is controlled by `validateOnStaleSecs`: + +```java +DataSourcePool pool = DataSourcePool.builder() + .name("mypool") + .url("jdbc:postgresql://localhost:5432/myapp") + .username("user") + .password("pass") + .validateOnStaleSecs(60) // validate on borrow if idle > 60 seconds + .build(); +``` + +When a connection is taken from the free list, if its last-used time is older than `validateOnStaleSecs`, it is validated (via `Connection.isValid()` / `heartbeatSql`) and evicted if invalid. This catches connections that died between heartbeat cycles, before they are handed to your application. + +**Default behaviour (when `validateOnStaleSecs` is not set):** + +- **Heartbeat enabled** (standard applications) → stale-on-borrow validation is **disabled**; the background heartbeat keeps the pool healthy. +- **Heartbeat disabled** (e.g. AWS Lambda) → stale-on-borrow validation is **enabled** at `min(100, maxInactiveTimeSecs)` seconds, since there is no background thread to validate idle connections. + +**Values:** + +- `0` — explicitly disable stale-on-borrow validation. +- A positive number — validate a free connection on borrow once it has been idle longer than that many seconds. + +**When to set it explicitly:** + +- Lambda / short-lived runtimes where you want a specific stale threshold rather than the default. +- Applications running with `validateOnHeartbeat(false)` that still want borrow-time validation. +- Environments where connections may be silently dropped (firewalls, NAT idle timeouts) and you want a guaranteed check before use, in addition to the heartbeat. + +--- + ## When (Rarely) You Need Explicit heartbeatSql() In almost all modern scenarios, you should NOT set explicit `heartbeatSql()`. Only in these edge cases: @@ -248,6 +282,12 @@ Connection returned to pool after use │ └─ Dead connection removed from pool └─ If no error, connection stays in pool +Connection borrowed from pool (if validateOnStaleSecs applies) + ├─ If connection idle longer than validateOnStaleSecs + │ ├─ Validate connection before handing it out + │ └─ Evict and replace if invalid + └─ Otherwise, hand out connection as-is + Application shutdown ├─ Stop heartbeat thread └─ Close all connections @@ -333,6 +373,7 @@ This tells you the current state of connections validated by heartbeat. - Use `validateOnHeartbeat(true)` for all applications except Lambda - Use default `heartbeatFreqSecs(30)` unless you have specific reasons otherwise - Let ebean-datasource auto-disable heartbeat in Lambda +- Consider `validateOnStaleSecs` for borrow-time validation when the heartbeat is disabled or connections may be dropped while idle ❌ **DON'T:** - Set explicit `heartbeatSql("SELECT 1")` unless required for your database driver diff --git a/docs/guides/create-datasource-pool.md b/docs/guides/create-datasource-pool.md index a539bde..697f620 100644 --- a/docs/guides/create-datasource-pool.md +++ b/docs/guides/create-datasource-pool.md @@ -178,6 +178,7 @@ DataSourcePool pool = DataSourcePool.builder() | `readOnly` | false | Set to true for read-only workloads | | `autoCommit` | false | Set to true to skip transaction boundaries | | `validateOnHeartbeat` | true (false in Lambda) | Enable background connection validation | +| `validateOnStaleSecs` | unset | Validate a free connection on borrow once idle longer than this (seconds); 0 disables | | `heartbeatFreqSecs` | 30 | How often to validate connections (seconds) | ### Typical Sizing diff --git a/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceBuilder.java b/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceBuilder.java index d67c665..01d61cd 100644 --- a/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceBuilder.java +++ b/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceBuilder.java @@ -763,6 +763,14 @@ default DataSourceBuilder initDatabaseForPlatform(String platform) { */ DataSourceBuilder validateOnHeartbeat(boolean validateOnHeartbeat); + /** + * Set the maximum age in seconds of an idle connection before it is validated + * when borrowed from the pool. + * + * @param validateOnStaleSecs the stale validation threshold in seconds + */ + DataSourceBuilder validateOnStaleSecs(int validateOnStaleSecs); + /** * Load the settings from the properties with no prefix on the property names. * @@ -839,6 +847,12 @@ interface Settings extends DataSourceBuilder { */ boolean isValidateOnHeartbeat(); + /** + * Return the maximum age in seconds of an idle connection before it is + * validated when borrowed from the pool. + */ + int validateOnStaleSecs(); + /** * Return the connection properties including credentials and custom parameters. */ diff --git a/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceConfig.java b/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceConfig.java index c3e7792..2e5e947 100644 --- a/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceConfig.java +++ b/ebean-datasource-api/src/main/java/io/ebean/datasource/DataSourceConfig.java @@ -90,6 +90,7 @@ public class DataSourceConfig implements DataSourceBuilder.Settings { private String applicationName; private boolean shutdownOnJvmExit; private boolean validateOnHeartbeat = !System.getenv().containsKey("LAMBDA_TASK_ROOT"); + private int validateOnStaleSecs = UNSET; private boolean enforceCleanClose; @Override @@ -145,6 +146,7 @@ public DataSourceConfig copy() { copy.failOnStart = failOnStart; copy.shutdownOnJvmExit = shutdownOnJvmExit; copy.validateOnHeartbeat = validateOnHeartbeat; + copy.validateOnStaleSecs = validateOnStaleSecs; if (customProperties != null) { copy.customProperties = new LinkedHashMap<>(customProperties); } @@ -212,6 +214,9 @@ public DataSourceConfig setDefaults(DataSourceBuilder builder) { if (validateOnHeartbeat && !other.isValidateOnHeartbeat()) { validateOnHeartbeat = false; } + if (validateOnStaleSecs == UNSET) { + validateOnStaleSecs = other.validateOnStaleSecs(); + } if (customProperties == null) { var otherCustomProps = other.getCustomProperties(); if (otherCustomProps != null && !otherCustomProps.isEmpty()) { @@ -825,6 +830,17 @@ public DataSourceConfig validateOnHeartbeat(boolean validateOnHeartbeat) { return this; } + @Override + public int validateOnStaleSecs() { + return validateOnStaleSecs; + } + + @Override + public DataSourceConfig validateOnStaleSecs(int validateOnStaleSecs) { + this.validateOnStaleSecs = validateOnStaleSecs; + return this; + } + @Override public DataSourceConfig load(Properties properties) { return load(properties, null); @@ -882,6 +898,7 @@ private void loadSettings(ConfigPropertiesHelper properties) { offline = properties.getBoolean("offline", offline); shutdownOnJvmExit = properties.getBoolean("shutdownOnJvmExit", shutdownOnJvmExit); validateOnHeartbeat = properties.getBoolean("validateOnHeartbeat", validateOnHeartbeat); + validateOnStaleSecs = properties.getInt("validateOnStaleSecs", validateOnStaleSecs); enforceCleanClose = properties.getBoolean("enforceCleanClose", enforceCleanClose); @@ -1020,10 +1037,16 @@ public Properties connectionProperties() { } public long validateStaleMillis() { + if (validateOnStaleSecs > UNSET) { + // explicitly set, if 0 then disabled + return validateOnStaleSecs * 1_000L; + } if (validateOnHeartbeat) { + // TODO: consider a default like Math.min(300, maxInactiveTimeSecs) * 1_000L return 0L; } else { - return (maxInactiveTimeSecs + trimPoolFreqSecs) * 1_000L; + // typically lambda function, no background validation, defaults to 100 secs or maxInactiveTimeSecs + return Math.min(100, maxInactiveTimeSecs) * 1_000L; } } } diff --git a/ebean-datasource-api/src/test/java/io/ebean/datasource/DataSourceConfigTest.java b/ebean-datasource-api/src/test/java/io/ebean/datasource/DataSourceConfigTest.java index 77e715b..dcbe79d 100644 --- a/ebean-datasource-api/src/test/java/io/ebean/datasource/DataSourceConfigTest.java +++ b/ebean-datasource-api/src/test/java/io/ebean/datasource/DataSourceConfigTest.java @@ -286,6 +286,83 @@ public void defaults_someOverride2() { assertThat(readOnly.isValidateOnHeartbeat()).isFalse(); } + @Test + void validateOnStaleSecs_explicitValue() { + var config = new DataSourceConfig().validateOnStaleSecs(17); + + assertThat(config.validateOnStaleSecs()).isEqualTo(17); + assertThat(config.validateStaleMillis()).isEqualTo(17_000L); + } + + @Test + void validateOnStaleSecs_zeroDisablesStaleValidation() { + var config = new DataSourceConfig() + .validateOnHeartbeat(true) + .validateOnStaleSecs(0) + .setMaxInactiveTimeSecs(900); + + assertThat(config.validateStaleMillis()).isZero(); + } + + @Test + void validateStaleMillis_heartbeatDisablesStaleValidationByDefault() { + var config = new DataSourceConfig() + .validateOnHeartbeat(true) + .setMaxInactiveTimeSecs(900); + + assertThat(config.validateStaleMillis()).isZero(); + + config.setMaxInactiveTimeSecs(120); + + assertThat(config.validateStaleMillis()).isZero(); + } + + @Test + void validateStaleMillis_capsNonHeartbeatValidationAtOneHundredSeconds() { + var config = new DataSourceConfig() + .validateOnHeartbeat(false) + .setMaxInactiveTimeSecs(900); + + assertThat(config.validateStaleMillis()).isEqualTo(100_000L); + + config.setMaxInactiveTimeSecs(45); + + assertThat(config.validateStaleMillis()).isEqualTo(45_000L); + } + + @Test + void validateOnStaleSecs_inheritsFromDefaults() { + var defaults = create().validateOnStaleSecs(17); + var config = new DataSourceConfig(); + + config.setDefaults(defaults); + + assertThat(config.validateOnStaleSecs()).isEqualTo(17); + assertThat(config.validateStaleMillis()).isEqualTo(17_000L); + } + + @Test + void validateOnStaleSecs_preservesExplicitValueWhenApplyingDefaults() { + var defaults = create().validateOnStaleSecs(17); + var config = new DataSourceConfig().validateOnStaleSecs(23); + + config.setDefaults(defaults); + + assertThat(config.validateOnStaleSecs()).isEqualTo(23); + assertThat(config.validateStaleMillis()).isEqualTo(23_000L); + } + + @Test + void validateOnStaleSecs_loadsFromProperties() { + var properties = new Properties(); + properties.setProperty("validateOnStaleSecs", "17"); + + var config = new DataSourceConfig().load(properties); + + assertThat(config.validateOnStaleSecs()).isEqualTo(17); + assertThat(config.validateStaleMillis()).isEqualTo(17_000L); + } + private DataSourceConfig create() { return new DataSourceConfig() .setDriver("org.postgresql.Driver")