Skip to content

Add OneLake workspace client factory - #7275

Open
Akın Soysal (akeensoysal) wants to merge 10 commits into
Azure:mainfrom
akeensoysal:feature/onelake-workspace-endpoint
Open

Add OneLake workspace client factory#7275
Akın Soysal (akeensoysal) wants to merge 10 commits into
Azure:mainfrom
akeensoysal:feature/onelake-workspace-endpoint

Conversation

@akeensoysal

@akeensoysal Akın Soysal (akeensoysal) commented Aug 3, 2026

Copy link
Copy Markdown

Summary

Add DataLakeFileSystemClient::CreateForOneLakeWorkspace so OneLake consumers can construct a validated workspace-scoped DFS/Blob client pair without duplicating Fabric cloud, ring, regional, and workspace-host resolution.

  • validate compact and hyphenated workspace GUIDs plus supported OneLake endpoint grammar
  • construct explicit ring-aware workspace DFS and Blob endpoints
  • reject unsafe retry, customer-key, audience, credential, and URL configurations before token or transport activity
  • preserve workspace Blob hosts and pipelines on rename-returned clients
  • add offline endpoint, auth, routing, rename, and DNS-boundary tests
  • add a strict redacted live-contract harness for the 4-cloud x 4-ring x DFS/Blob matrix
  • add changelog and DefaultAzureCredential sample coverage

Related to #7274.

Tracking

Verification

  • clean WARNINGS_AS_ERRORS=ON build of Data Lake unit tests, live harness, and sample
  • OneLakeWorkspaceClientTest.*: 22/22 passed
  • registered OneLakeWorkspaceEndpointHarness: passed
  • offline 32-case validate matrix: 32/32 passed, 0 probes
  • offline 32-case mock matrix: 32/32 passed, exact DFS/Blob hosts
  • existing Data Lake rename playback tests: 4/4 passed
  • LLVM 11 formatting and git diff --check: passed
  • pinned AMQP test broker (d82a8645) restores from an empty cache through public Azure mirrors, builds for net8.0, starts, listens on port 25672, and accepts a connection
  • fork Core CI skips credentialed Cargo feed setup and uses vendored uAMQP (199/199 tests passed); trusted builds retain Rust AMQP with CFS routing

Pending validation and release conditions

  • Azure Storage maintainers and OneLake service owners confirm the API placement and supported cloud/ring matrix.
  • The generated Data Lake APIView completes the required Azure SDK API review. For this targeted additive change, C++ language-architect approval is sufficient unless the change is escalated to the Architecture Board.
  • Before release, OneLake service-owned live validation passes every owner-approved cloud/ring/API-family combination with no failures or skips. If the proposed 4 x 4 x 2 matrix is confirmed, the required result is 32 passed, 0 failed, 0 skipped.

CDC will update its Azure SDK submodule to the first upstream Azure SDK for C++ release tag containing this API; it will not pin a commit from this fork.

Pull Request Checklist

  • C++ Guidelines
  • Doxygen docs
  • Unit tests
  • No unwanted commits/changes
  • Descriptive title/description
    • PR is single purpose
    • Related issue listed
  • Comments in source
  • No typos
  • Update changelog
  • Not work-in-progress
  • External references or docs updated
  • Self review of PR done
  • Any breaking changes? None; the API is additive.

Add fail-closed OneLake endpoint resolution, explicit DFS and Blob client pairing, offline routing coverage, and a strict live-contract harness. Preserve ring-specific Blob hosts on rename results.\n\nRefs Azure#7274
@github-actions github-actions Bot added Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files) labels Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Thank you for your contribution Akın Soysal (@akeensoysal)! We will review the pull request and get back to you soon.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).
8 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Use Azure Core JSON support, preserve the sample capability boundary, fix multi-config test discovery, and register OneLake service terms with cspell.
Restore unrelated Core CI and AMQP changes to the main-branch versions so the pull request remains focused on the OneLake workspace client API.\n\nRelated to Azure#7274.
Skip private Cargo feed authentication for fork pull requests and build the supported vendored uAMQP implementation instead. Pin the AMQP test broker, restore it through the public Azure SDK NuGet mirror, and return pre-cancelled management opens before network setup.\n\nRelated to Azure#7274.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).
8 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds first-class Microsoft Fabric OneLake workspace support to azure-storage-files-datalake by introducing a validated factory (DataLakeFileSystemClient::CreateForOneLakeWorkspace) that resolves ring/cloud/workspace-scoped DFS+Blob endpoints and returns a correctly paired client configuration. It also expands test coverage (unit + offline harness) and includes a small set of ancillary changes to AMQP tests and CI to support fork scenarios.

Changes:

  • Added DataLakeFileSystemClient::CreateForOneLakeWorkspace with strict URL/workspace validation and explicit construction of workspace DFS/Blob endpoints.
  • Updated rename-result client construction to preserve the workspace blob host/pipeline behavior.
  • Added extensive unit tests plus a CTest + Python harness for validating/mock-probing the cloud/ring/API-family endpoint matrix; updated sample and build/pipeline wiring.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
sdk/storage/azure-storage-files-datalake/test/ut/onelake_workspace_client_test.cpp New unit tests for OneLake workspace factory behavior, validation, routing, rename behavior, and tenant discovery.
sdk/storage/azure-storage-files-datalake/test/ut/CMakeLists.txt Adds the new OneLake unit test source to the UT target.
sdk/storage/azure-storage-files-datalake/test/live/onelake_workspace_endpoint_live_test.py Python driver that runs the harness in validate/mock modes and verifies redaction/behavior.
sdk/storage/azure-storage-files-datalake/test/live/onelake_workspace_endpoint_live_test.cpp C++ harness that validates endpoint normalization and (optionally) probes endpoints; produces redacted JSON evidence.
sdk/storage/azure-storage-files-datalake/test/live/fixtures/valid-two-row.tsv Fixture: minimal valid manifest (2 rows) for harness tests.
sdk/storage/azure-storage-files-datalake/test/live/fixtures/missing.tsv Fixture: manifest missing required row(s) to validate negative-path behavior.
sdk/storage/azure-storage-files-datalake/test/live/fixtures/malformed.tsv Fixture: manifest row with wrong column count to validate parser error handling.
sdk/storage/azure-storage-files-datalake/test/live/fixtures/failed.tsv Fixture: manifest with an invalid endpoint to validate failure reporting/redaction.
sdk/storage/azure-storage-files-datalake/test/live/fixtures/duplicate.tsv Fixture: duplicate matrix key to validate duplicate detection.
sdk/storage/azure-storage-files-datalake/src/private/datalake_utilities.hpp Declares GetOneLakeWorkspaceEndpoints and the endpoints pair struct.
sdk/storage/azure-storage-files-datalake/src/datalake_utilities.cpp Implements OneLake URL/workspace parsing, validation, and DFS/Blob endpoint synthesis.
sdk/storage/azure-storage-files-datalake/src/datalake_file_system_client.cpp Adds the public factory implementation and updates rename logic to preserve blob host/pipeline via existing container client.
sdk/storage/azure-storage-files-datalake/src/datalake_directory_client.cpp Updates rename logic to preserve blob host/pipeline via existing blob client.
sdk/storage/azure-storage-files-datalake/samples/datalake_getting_started.cpp Adds a OneLake path that uses DefaultAzureCredential + the new OneLake factory when env vars are present.
sdk/storage/azure-storage-files-datalake/samples/CMakeLists.txt Links azure-identity for the sample due to DefaultAzureCredential usage.
sdk/storage/azure-storage-files-datalake/inc/azure/storage/files/datalake/datalake_file_system_client.hpp Adds the public CreateForOneLakeWorkspace API and Doxygen remarks.
sdk/storage/azure-storage-files-datalake/CMakeLists.txt Adds the harness executable and a Python-driven CTest for validate/mock coverage.
sdk/storage/azure-storage-files-datalake/CHANGELOG.md Documents the additive OneLake factory feature.
sdk/core/azure-core-amqp/test/ut/claim_based_security_tests.cpp Removes server listening start/stop calls from a CBS cancellation test.
sdk/core/azure-core-amqp/Test-Setup.ps1 Pins azure-amqp test broker commit and adjusts NuGet sources for restore.
sdk/core/azure-core-amqp/src/impl/uamqp/amqp/management.cpp Returns Cancelled early when opening management with a cancelled context.
eng/pipelines/templates/steps/cmake-build.yml Skips Rust cargo feed auth for fork PRs and adds fork-specific CMake args support.
.vscode/cspell.json Adds new words used by the OneLake cloud/ring matrix to the spellchecker dictionary.

Comment thread sdk/storage/azure-storage-files-datalake/src/datalake_utilities.cpp Outdated
Comment thread sdk/storage/azure-storage-files-datalake/CMakeLists.txt Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants