Skip to content

Server Side Encryption using KMS Key failing for validate_s3_path_exists #5930

@kartik2112

Description

@kartik2112

PySDK Version

  • PySDK V2 (2.x)
  • PySDK V3 (3.x)

Describe the bug
While running sagemaker.train.sft_trainer.SFTTrainer, internally, the function sagemaker.train.common_utils.finetune_utils._validate_s3_path_exists is called but if the s3 bucket referred has SSE (Server Side Encryption) enabled and the path doesn't exist, s3.put_object fails.

To reproduce

  • Add SSE using kms key to your target s3 bucket for SFT job
  • For any dataset, try running SFTTrainer job, it fails saying access denied.

Expected behavior
Since kms_key_id is an accepted parameter in SFTTrainer, _validate_s3_path_exists should succeed and proceed to launch the job. Note that after the actual training is done, the subsequent put_object for model files does not fail.

System information
A description of your system. Please provide:

  • SageMaker Python SDK version: 3.12.0
  • Python version: 3.12
  • CPU or GPU: CPU
  • Custom Docker image (Y/N): N

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions