Add event environments and filtering

docs/docs/clients/dotnet/configuration.md

@@ -5,6 +5,8 @@ order: 1
 
 # Configuration
 
+Set a deployment environment once at startup, with optional per-event overrides. See [Environments](/docs/environments/) for configuration examples.
+
 There are a few ways to configure Exceptionless in your project. We'll cover them here or you can jump to app-specific examples: [Console App Example](/docs/clients/dotnet/guides/console-apps-example), [Web Server Example](/docs/clients/dotnet/guides/web-server-example).
 
 ---

docs/docs/clients/javascript/client-configuration.md

@@ -5,6 +5,8 @@ order: 1
 
 # Configuration
 
+Set a deployment environment once at startup, with optional per-event overrides. See [Environments](/docs/environments/) for configuration examples.
+
 - [Installation](#installation)
   - [Browser](#browser)
   - [Node.js](#nodejs)

docs/docs/environments.md

@@ -0,0 +1,73 @@
+---
+title: "Environments"
+---
+
+# Environments
+
+Use an environment to distinguish production, staging, development, or a custom deployment within one project. Each event can carry one optional top-level `environment` string. The same error still belongs to one stack, with one shared status and fixed version.
+
+## Configure your client
+
+Set a default during application startup. In the .NET client:
+
+```csharp
+client.Configuration.SetEnvironment("production");
+client.CreateLog("Deployment complete").SetEnvironment("staging").Submit();
+```
+
+You can also set `Exceptionless:Environment` in .NET configuration, or the `Exceptionless__Environment` environment variable. The hosting integration uses `IHostEnvironment.EnvironmentName` when no explicit environment is configured. A per-event value overrides the default.
+
+In the JavaScript client:
+
+```javascript
+await Exceptionless.startup((config) => {
+  config.apiKey = "API_KEY_HERE";
+  config.environment = "production";
+});
+
+await Exceptionless.createLog("Deployment complete")
+  .setEnvironment("staging")
+  .submit();
+```
+
+For direct API submissions:
+
+```json
+{
+  "type": "log",
+  "message": "Deployment complete",
+  "environment": "production"
+}
+```
+
+Names are trimmed, with a maximum of 64 characters. Events preserve the supplied casing: `" Production "` is stored and returned as `"Production"`. Filtering is case-insensitive, and aggregation keys are normalized to lowercase, so `Production` and `production` share one environment bucket. Custom names such as `qa-west` are supported. Empty, oversized, control-character, and non-string values are treated as unspecified. Historical events and older clients without this field remain unspecified; they are never assumed to be production.
+
+## Filter your data
+
+Choose **Manage filters → Environment** on Events, Stacks, Sessions, or Stream. Select one or more names, or **Unspecified** for events without an environment. Clear the selection to include all environments. The picker discovers names for the selected projects and time range; you can also enter a name that has no current events.
+
+Environment selections persist in URLs and saved views. Events, sessions, and stream tables have an optional Environment column. Event details show Environment in the overview when the event has one; **Machine & runtime** contains the existing `data.@environment` diagnostics.
+
+Environment searches and aggregations are available on every plan:
+
+| Search | Meaning |
+| --- | --- |
+| `environment:production` | Production events |
+| `(environment:production OR environment:staging)` | Either deployment |
+| `_missing_:environment` | Historical or unspecified events |
+| `_exists_:environment` | Events with an environment |
+| `environment:"qa west"` | A custom name containing spaces |
+
+Stack dashboard counts and charts use events matching the filter. Stack status remains shared, so changing status while viewing production also changes the same stack seen in staging. Automatic session detection separates the same user's activity by environment. Session heartbeats do not include an environment, so a heartbeat keeps the user's open sessions active in every environment.
+
+Notifications and the **New** view are not environment-aware yet. A stack is new once, when its first event arrives from any environment. An error first seen in development triggers the new error notification then, and is not reported as new again when it reaches production; the **New** view also dates it from that first occurrence. Critical error and regression notifications are sent for events from every environment.
+
+## Fixing stacks across deployments
+
+Use [fixed in version](/docs/versioning/) when environments run different releases. For example, if a stack is fixed in `2.4.0`, occurrences from production running `2.3.0` do not represent a regression just because staging already has the fix. Continue sending the application version with events. Plain **Fixed**, without a version, retains its existing behavior across all environments.
+
+## Rollout
+
+Deploy the server before adopting SDK versions that expose the new setting. The server adds mappings to existing event indices without rewriting historical events. Self-hosted installs should let the migration job finish before the new API and job instances process events: events written to an existing index before the mapping is added keep their environment but cannot be filtered or aggregated by it. Clients that do not send an environment continue to work. `data.@environment` and custom `data.environment` values retain their existing meanings.
+
+Before this release, the server stored an unrecognized root `environment` property as custom data in `data.environment`. A valid root `environment` string is now stored only as the event's environment, trimmed, so read it from `environment` instead of `data.environment`. Any other root value, such as null, an object, a number, a blank string, or an oversized name, is still stored as `data.environment`. If `data.environment` already exists, that value uses the next available key (`environment1`, `environment2`, and so on).

docs/docs/filtering-and-searching.md

@@ -6,6 +6,7 @@ title: "Filtering & Searching"
 
 - [Filter by Organization \& Project](#filter-by-organization--project)
 - [Filter by Time Frame](#filter-by-time-frame)
+- [Filter by Environment](#filter-by-environment)
 - [Filter / Search by Specific Criteria](#filter--search-by-specific-criteria)
 - [Searchable Fields \& Requirements](#searchable-fields--requirements)
 - [Multiple Queries](#multiple-queries)
@@ -30,6 +31,10 @@ Click on the calendar icon in the header to select from multiple preset time fra
 
 ![Exceptionless Filter Time Frame](img/filter-by-timeframe.png)
 
+## Filter by Environment
+
+Choose **Manage filters → Environment** to select production, staging, development, or a custom name. Select **Unspecified** for events without this property, or clear the selection to include all environments. See [Environments](/docs/environments/) for client configuration and shared stack behavior.
+
 ## Filter / Search by Specific Criteria
 
 Click the magnifying glass to search by specific criteria.
@@ -53,6 +58,7 @@ View a complete list of searchable terms, examples, and FAQs below.
 | stack              | `stack:54d8315ce6bb2d0500bcc7b4`                                                                                     | true                         | Stack id                                      |
 | reference          | `reference:12345678`                                                                                                 | true                         | Reference id                                  |
 | session            | `session:12345678`                                                                                                   | true                         | Session id                                    |
+| environment        | `environment:production` or `_missing_:environment` | true | Deployment environment |
 | type               | `type:error`                                                                                                         | true                         | Event type                                    |
 | source             | `source:"my log source"` or `"my log source"`                                                                        | false                        | Event source                                  |
 | level              | `level:Error`                                                                                                        | true                         | Log level                                     |

docs/docs/versioning.md

@@ -4,6 +4,8 @@ title: "Versioning"
 
 # Versioning
 
+Stacks and their fixed version are shared across [environments](/docs/environments/). Continue sending application versions so older deployments can report occurrences without incorrectly reopening a stack fixed in a newer version.
+
 You can mark error stacks fixed and they won't show up or notify you until they regress!
 
 ## How does this work?

src/Exceptionless.Core/Extensions/PersistentEventExtensions.cs

@@ -183,6 +183,7 @@ public static PersistentEvent ToSessionStartEvent(this PersistentEvent source, I
     {
         var startEvent = new PersistentEvent
         {
+            Environment = source.Environment,
             Date = source.Date,
             Geo = source.Geo,
             OrganizationId = source.OrganizationId,

src/Exceptionless.Core/Jobs/CloseInactiveSessionsJob.cs

@@ -50,20 +50,22 @@ protected override async Task<JobResult> RunInternalAsync(JobContext context)
         if (results.Documents.Count == 0)
             return JobResult.Success;
 
+        // A user's heartbeat applies to their sessions in every environment, and those sessions can be on
+        // different pages, so heartbeats are consumed only after every page has been processed.
+        var consumedHeartbeats = new Dictionary<string, DateTime>(StringComparer.OrdinalIgnoreCase);
+        var closeKeysToRemove = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
+        var existingSessionHeartbeatIds = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
         while (results.Documents.Count > 0 && !context.CancellationToken.IsCancellationRequested)
         {
             var inactivePeriodUtc = _timeProvider.GetUtcNow().UtcDateTime.Subtract(DefaultInactivePeriod);
             var sessionsToUpdate = new List<PersistentEvent>(results.Documents.Count);
-            var cacheKeysToRemove = new List<string>(results.Documents.Count * 2);
-            var existingSessionHeartbeatIds = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
 
             foreach (var (sessionStart, heartbeatResult) in await GetHeartbeatsBatchAsync(results.Documents))
             {
                 var lastActivityUtc = sessionStart.Date.UtcDateTime.AddSeconds((double)sessionStart.Value.GetValueOrDefault());
 
-                bool closeDuplicate = heartbeatResult?.CacheKey is not null && existingSessionHeartbeatIds.Contains(heartbeatResult.CacheKey);
-                if (heartbeatResult?.CacheKey is not null && !closeDuplicate)
-                    existingSessionHeartbeatIds.Add(heartbeatResult.CacheKey);
+                // Heartbeats are not scoped to an environment, so one user's sessions in different environments are not duplicates.
+                bool closeDuplicate = heartbeatResult?.CacheKey is not null && !existingSessionHeartbeatIds.Add($"{heartbeatResult.CacheKey}:{sessionStart.Environment}");
 
                 if (heartbeatResult is not null && (closeDuplicate || heartbeatResult.Close || heartbeatResult.ActivityUtc > lastActivityUtc))
                     sessionStart.UpdateSessionStart(heartbeatResult.ActivityUtc, isSessionEnd: closeDuplicate || heartbeatResult.Close || heartbeatResult.ActivityUtc <= inactivePeriodUtc);
@@ -75,9 +77,10 @@ protected override async Task<JobResult> RunInternalAsync(JobContext context)
                 sessionsToUpdate.Add(sessionStart);
                 if (heartbeatResult?.CacheKey is not null)
                 {
-                    cacheKeysToRemove.Add(heartbeatResult.CacheKey);
+                    // Keep the first value read: sessions on earlier pages only applied that one.
+                    consumedHeartbeats.TryAdd(heartbeatResult.CacheKey, heartbeatResult.ActivityUtc);
                     if (heartbeatResult.Close)
-                        cacheKeysToRemove.Add($"{heartbeatResult.CacheKey}-close");
+                        closeKeysToRemove.Add($"{heartbeatResult.CacheKey}-close");
                 }
 
                 Debug.Assert(sessionStart.Value is not null && sessionStart.Value >= 0, "Session start value cannot be a negative number.");
@@ -89,9 +92,6 @@ protected override async Task<JobResult> RunInternalAsync(JobContext context)
             if (sessionsToUpdate.Count > 0)
                 await _eventRepository.SaveAsync(sessionsToUpdate);
 
-            if (cacheKeysToRemove.Count > 0)
-                await _cache.RemoveAllAsync(cacheKeysToRemove);
-
             _logger.LogInformation("Closing {SessionClosedCount} of {SessionCount} sessions", sessionsToUpdate.Count, results.Documents.Count);
 
             // Sleep so we are not hammering the backend.
@@ -106,6 +106,16 @@ protected override async Task<JobResult> RunInternalAsync(JobContext context)
                 _lastActivity = _timeProvider.GetUtcNow().UtcDateTime;
             }
         }
+
+        // Sessions on unprocessed pages still need the heartbeats, so a cancelled run leaves them for the next run.
+        if (!context.CancellationToken.IsCancellationRequested)
+        {
+            // A heartbeat written while later pages were processed has not been applied, so only unchanged values are removed.
+            await Task.WhenAll(consumedHeartbeats.Select(heartbeat => _cache.RemoveIfEqualAsync(heartbeat.Key, heartbeat.Value)));
+            if (closeKeysToRemove.Count > 0)
+                await _cache.RemoveAllAsync(closeKeysToRemove);
+        }
+
         _logger.LogInformation("Done checking active sessions. Closed {SessionClosedCount} of {SessionCount} sessions", sessionsClosed, totalSessions);
 
         return JobResult.Success;

src/Exceptionless.Core/Migrations/010_AddEventEnvironment.cs

@@ -0,0 +1,36 @@
+using Exceptionless.Core.Extensions;
+using Exceptionless.Core.Models;
+using Exceptionless.Core.Repositories.Configuration;
+using Foundatio.Repositories.Elasticsearch.Extensions;
+using Foundatio.Repositories.Migrations;
+using Microsoft.Extensions.Logging;
+
+namespace Exceptionless.Core.Migrations;
+
+public sealed class AddEventEnvironment : MigrationBase
+{
+    private readonly ExceptionlessElasticConfiguration _configuration;
+
+    public AddEventEnvironment(ExceptionlessElasticConfiguration configuration, ILoggerFactory loggerFactory) : base(loggerFactory)
+    {
+        _configuration = configuration;
+        MigrationType = MigrationType.VersionedAndResumable;
+        Version = 10;
+    }
+
+    public override async Task RunAsync(MigrationContext context)
+    {
+        var response = await _configuration.Client.Indices.PutMappingAsync<PersistentEvent>(mapping => mapping
+            .Indices($"{_configuration.Events.Name}-v*-*")
+            .AllowNoIndices(true)
+            .IgnoreUnavailable(true)
+            .Properties(properties => properties.Text(ev => ev.Environment,
+                text => text.Analyzer(EventIndex.LOWER_KEYWORD_ANALYZER)
+                    .Fields(fields => fields.Keyword("keyword", keyword => keyword.Normalizer("lowercase"))))), context.CancellationToken);
+        _logger.LogRequest(response);
+        if (!response.IsValidResponse)
+        {
+            throw new InvalidOperationException("Unable to add the event environment mapping: " + response.DebugInformation);
+        }
+    }
+}

src/Exceptionless.Core/Models/Event.cs

@@ -11,6 +11,30 @@ namespace Exceptionless.Core.Models;
 [DebuggerDisplay("Type: {Type}, Date: {Date}, Message: {Message}, Value: {Value}, Count: {Count}")]
 public class Event : IData, IJsonOnDeserialized
 {
+    private string? _environment;
+
+    /// <summary>
+    /// The deployment environment, such as production, staging, or development.
+    /// Missing or invalid names remain unspecified. Machine and runtime information is stored separately in data.@environment.
+    /// </summary>
+    [StringLength(64)]
+    public string? Environment
+    {
+        get => _environment;
+        set => _environment = NormalizeEnvironment(value);
+    }
+
+    /// <summary>
+    /// Returns the trimmed deployment name, or null when the value cannot be a deployment name.
+    /// </summary>
+    public static string? NormalizeEnvironment(string? value)
+    {
+        string? name = value?.Trim();
+        return String.IsNullOrEmpty(name) || name.Length > 64 || name.Any(Char.IsControl)
+            ? null
+            : name;
+    }
+
     /// <summary>
     /// The event type (ie. error, log message, feature usage). Check <see cref="KnownTypes">Event.KnownTypes</see> for standard event types.
     /// Nullable in transit; the pipeline infers a default before save. Validated as required on repository save.
@@ -112,7 +136,7 @@ void IJsonOnDeserialized.OnDeserialized()
 
     protected bool Equals(Event other)
     {
-        return String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data);
+        return String.Equals(Environment, other.Environment) && String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data);
     }
 
     public override bool Equals(object? obj)
@@ -138,6 +162,10 @@ public override int GetHashCode()
             hashCode = (hashCode * 397) ^ (Geo?.GetHashCode() ?? 0);
             hashCode = (hashCode * 397) ^ Value.GetHashCode();
             hashCode = (hashCode * 397) ^ (Data?.GetCollectionHashCode(_exclusions) ?? 0);
+            if (Environment is not null)
+            {
+                hashCode = (hashCode * 397) ^ Environment.GetHashCode();
+            }
             return hashCode;
         }
     }

src/Exceptionless.Core/Models/EventSummaryModel.cs

@@ -2,6 +2,7 @@
 
 public record EventSummaryModel : SummaryData
 {
+    public string? Environment { get; set; }
     public DateTimeOffset Date { get; set; }
     public string ProjectId { get; set; } = null!;
     public string? ProjectName { get; set; }

src/Exceptionless.Core/Models/WebHookEvent.cs

@@ -14,6 +14,7 @@ public WebHookEvent(string baseUrl)
     public DateTimeOffset? OccurrenceDate { get; init; }
     public TagSet? Tags { get; init; }
     public string? Type { get; init; }
+    public string? Environment { get; init; }
     public string? Source { get; init; }
     public string? Message { get; init; }
     public string ProjectId { get; init; } = null!;