Skip to content

Perf/api mongo projections#651

Open
Kuchizu wants to merge 3 commits into
masterfrom
perf/api-mongo-projections
Open

Perf/api mongo projections#651
Kuchizu wants to merge 3 commits into
masterfrom
perf/api-mongo-projections

Conversation

@Kuchizu
Copy link
Copy Markdown
Member

@Kuchizu Kuchizu commented May 6, 2026

findDailyEventsPortion: $limit before $lookups when no content filter (was the 8.8s prod aggregation). DataLoader batch helpers gain optional projection arg.

return { 'event.assignee': String(assignee) };
})();

/** When false, $limit can move before the $lookups. */
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please explain in docs why it is useful

Comment thread src/dataLoaders.ts
collectionName: string,
ids: ReadonlyArray<string>
ids: ReadonlyArray<string>,
projection?: Record<string, 0 | 1>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no usages. Where is it used?

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes the findDailyEventsPortion MongoDB aggregation for the no-content-filters case by moving $limit earlier (before $lookup stages) and extends internal DataLoader batching helpers to optionally accept MongoDB projections to reduce fetched fields.

Changes:

  • In findDailyEventsPortion, conditionally applies $limit before $lookup stages when no content filters are present, keeping $match + $limit after lookups when filters exist.
  • Adds an optional projection parameter to DataLoader batch helpers (batchByIds / batchByField) and forwards it to MongoDB .find(...).
  • Bumps package version to 1.5.1.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
src/models/eventsFactory.js Moves $limit earlier in the daily-events aggregation when filters are absent to reduce lookup workload.
src/dataLoaders.ts Adds optional MongoDB projection support to batch helper methods used by DataLoaders.
package.json Version bump from 1.5.0 to 1.5.1.
Comments suppressed due to low confidence (1)

src/dataLoaders.ts:107

  • valuesMap is typed as Map<string, FieldType>, but it stores the actual field values (T[FieldType]). Updating it to Map<string, T[FieldType]> would make the intent clearer and keep types accurate when building the $in array and mapping results.
    type Doc = WithId<T>;
    const valuesMap = new Map<string, FieldType>();

    for (const value of values) {
      valuesMap.set(value.toString(), value);
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +407 to +408
const hasContentFilters =
escapedSearch.length > 0 ||
Comment thread src/dataLoaders.ts
Comment on lines 109 to +114
const queryResult = await this.dbConnection
.collection<T>(collectionName)
.find({
[fieldName]: { $in: Array.from(valuesMap.values()) },
} as any)
.find(
{ [fieldName]: { $in: Array.from(valuesMap.values()) } } as any,
projection ? { projection } : {}
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants