diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 7006fbd3..997c1743 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -11,7 +11,7 @@ - [ ] Tested -- [ ] Docs / README updated (if public API changed) +- [ ] Docs updated — if this adds, changes or removes anything a consumer calls, update the module's `.md` alongside the code (for example `lib/addons/.md`) and the matching README section. New addons ship with a new `.md`. diff --git a/README.md b/README.md index 6befe60d..4795551a 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,7 @@ JavaScript SDK for integrating with an [Optable Data Connectivity Node (DCN)](ht - [Return Value](#return-value) - [Input Type](#input-type) - [Geo-routing](#geo-routing) +- [Bot detection](#bot-detection) - [Demo Pages](#demo-pages) ## Installing @@ -1216,6 +1217,34 @@ if (host) { Keys are region codes, not country codes. Translating a visitor's country code to a region code (for example `GB`/`UK` → `EU`) is the caller's responsibility — the addon deliberately knows only regions. The caller also supplies the SDK `node`/`site`; this addon only resolves the host. +For the full region table and custom `GeoMap` usage, see the [geo-routing addon README](lib/addons/geo-routing.md). + +## Bot detection + +The bot detection addon identifies requests coming from known bots and crawlers, so a wrapper can skip work that only makes sense for real visitors — edge calls, identity resolution, analytics samples. It is a pure function over the user agent, with no network calls or storage access. + +```typescript +import { isBot } from "@optable/web-sdk/lib/dist/addons/botDetection"; + +if (isBot()) { + return; // Skip targeting and analytics for this request. +} +``` + +With no argument it reads `navigator.userAgent`; pass a string to test one explicitly. + +When the page also runs a Prebid RTD provider, prefer `SkipTargetingForBots()`. It calls `isBot()` and, for a bot, marks targeting as already done so the RTD module short-circuits instead of waiting for a targeting call that will never be made: + +```typescript +import { SkipTargetingForBots } from "@optable/web-sdk/lib/dist/edge/targeting"; + +if (!SkipTargetingForBots()) { + await sdk.targeting(); +} +``` + +Matching is substring-based and case-insensitive, covering generic crawlers, headless browsers, HTTP clients and Google's non-search agents. It is deliberately broad and user-agent only — a cost-saving filter, not a fraud signal. For the full match list, see the [bot detection addon README](lib/addons/botDetection.md). + ## Demo Pages The demo pages are working examples of both `identify` and `targeting` APIs, as well as an integration with the [Google Ad Manager 360](https://admanager.google.com/home/) ad server, enabling the targeting of ads served by GAM360 to audiences activated in the [Optable](https://optable.co/) DCN. diff --git a/lib/addons/botDetection.md b/lib/addons/botDetection.md new file mode 100644 index 00000000..947f3f25 --- /dev/null +++ b/lib/addons/botDetection.md @@ -0,0 +1,64 @@ +# Bot Detection Addon + +This addon identifies requests coming from known bots and crawlers, so a wrapper can skip work that only makes sense for real visitors — edge calls, identity resolution, analytics samples. + +It is a pure function over the user agent string. It performs no network calls, reads no storage, and has no side effects. + +## Usage + +```js +import { isBot } from "@optable/web-sdk/lib/dist/addons/botDetection"; + +if (isBot()) { + // Skip targeting, analytics and any other per-visitor work. + return; +} +``` + +With no argument it reads `navigator.userAgent`. Pass a string to test one explicitly, which is also how it is unit tested: + +```js +isBot("Mozilla/5.0 (compatible; Googlebot/2.1)"); // true +isBot("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)"); // false +``` + +## Skipping targeting for bots + +`SkipTargetingForBots()` is the companion helper for the common case. It calls `isBot()` and, when true, writes `OPTABLE_TARGETING_DONE` to `sessionStorage`. + +```js +import { SkipTargetingForBots } from "@optable/web-sdk/lib/dist/edge/targeting"; + +const skipped = SkipTargetingForBots(); +if (!skipped) { + await sdk.targeting(); +} +``` + +It returns whether the request was identified as a bot, and is a no-op for real visitors. Prefer it over a bare `isBot()` early-return when other code on the page can also trigger targeting. + +## What is matched + +The user agent is tested case-insensitively against a single pattern built from these substrings: + +| Category | Substrings | +| -------------------------------- | ---------------------------------------------------------------------------------------------- | +| Generic crawlers | `bot`, `crawler`, `spider`, `scraper` | +| Headless browsers and automation | `headless`, `phantomjs`, `selenium`, `webdriver` | +| HTTP clients and runtimes | `curl`, `wget`, `python`, `java`, `perl`, `ruby`, `go-http-client`, `okhttp`, `axios`, `fetch` | +| API tools | `postman`, `insomnia` | +| Google non-search agents | `googleother`, `google-extended`, `google-inspectiontool` | + +Matching is substring-based, so `Googlebot`, `bingbot` and `AhrefsBot` are all caught by `bot`. + +Two consequences worth knowing: + +- **It is deliberately broad.** `java` matches any user agent containing that substring, and the Google entries cover the non-search crawlers that should not consume edge calls. The bias is toward skipping work rather than toward precise classification. +- **It is user-agent only.** A bot that presents a browser user agent is not detected. Treat this as a cost-saving filter, not a security control or a fraud signal. + +## API + +| Export | Signature | Description | +| ---------------------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| `isBot` | `(userAgent?: string) => boolean` | True when the user agent looks like a known bot. Defaults to `navigator.userAgent`. | +| `SkipTargetingForBots` | `() => boolean` | Exported from `lib/edge/targeting`. Marks targeting done for bots so RTD short-circuits. Returns whether a bot was detected. | diff --git a/lib/addons/geo-routing.md b/lib/addons/geo-routing.md new file mode 100644 index 00000000..43b3133c --- /dev/null +++ b/lib/addons/geo-routing.md @@ -0,0 +1,68 @@ +# Geo-routing Addon + +This addon maps a visitor's region code to the Optable edge host that should serve them, so a single SDK bundle can route traffic to the regional edge closest to — and provisioned for — the visitor. + +It is a pure lookup. It performs no network calls, does no geolocation of its own, and has no side effects. + +## Usage + +```js +import { getGeoRouting } from "@optable/web-sdk/lib/dist/addons/geo-routing"; + +const host = getGeoRouting(visitorRegion); // "na.edge.optable.co" for "US" +if (host) { + const sdk = new OptableSDK({ host, node: "my-node", site: "my-site" }); +} +``` + +`getGeoRouting` returns `null` when the region is missing or unsupported. Either skip region-specific initialization, or fall back to a configured default: + +```js +const host = getGeoRouting(visitorRegion) ?? "na.edge.optable.co"; +``` + +## Supported regions + +| Region code | Edge host | +| ----------- | -------------------- | +| `AU` | `au.edge.optable.co` | +| `CA` | `ca.edge.optable.co` | +| `EU` | `eu.edge.optable.co` | +| `NA` | `na.edge.optable.co` | +| `US` | `na.edge.optable.co` | + +`US` and `NA` are aliases for the same North America edge. + +## Custom region maps + +Pass a `GeoMap` as the second argument to support region codes outside the default set, or to point them at different hosts: + +```js +import { getGeoRouting, DEFAULT_GEO_MAP } from "@optable/web-sdk/lib/dist/addons/geo-routing"; + +const host = getGeoRouting(visitorRegion, { + ...DEFAULT_GEO_MAP, + UK: "eu.edge.optable.co", + JP: "ap.edge.optable.co", +}); +``` + +Lookups use `Object.prototype.hasOwnProperty`, so an unexpected region such as `"constructor"` resolves to `null` rather than picking up an inherited `Object.prototype` member. + +## Resolving the region code + +**Translating a country to a supported region code is the caller's responsibility.** The addon knows regions, not the full country-to-region table, and does not detect the visitor's location. Supply the code from whatever source the page already has — a CDN geo header, a CMP, or a publisher-set global: + +```js +const host = getGeoRouting(window.optable?.countryCode); +``` + +The addon also resolves only the host. The SDK `node` and `site` are separate configuration and must be supplied by the caller. + +## API + +| Export | Signature | Description | +| ----------------- | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | +| `getGeoRouting` | `(region: string \| undefined, geoMap?: GeoMap) => string \| null` | Resolves the edge host for a region code. `null` when the region is missing or absent from the map. | +| `DEFAULT_GEO_MAP` | `GeoMap` | The built-in region-to-host table. Spread it to extend rather than replace it. | +| `GeoMap` | `Record` | Type alias for a region-code-to-host map. |