Skip to content

various fixes for fiberchannel and autoscale prefix option - #11071

Closed
rg9975 wants to merge 26 commits into
apache:4.20from
rg9975:4.20-fiberchannel-patches
Closed

various fixes for fiberchannel and autoscale prefix option#11071
rg9975 wants to merge 26 commits into
apache:4.20from
rg9975:4.20-fiberchannel-patches

Conversation

@rg9975

@rg9975 rg9975 commented Jun 23, 2025

Copy link
Copy Markdown

Description

This PR fixes various issues in the fiberchannel/adaptive storage provider, including adding an optional interface for handling connecting disks in a 3 step process to improve performance on VMs with a large number of disks to connect. One additional change is included to provide an option to disable the fixed string prefix added to autoscale VMs to allow for longer generated names. A couple of small annotations were added to help builds when running in an m2e-compatible IDE.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Code has been tested and used in live environment with appropriate storage appliances.

How did you try to break this feature and the system with this change?

Code has been tested and used in live environment with appropriate storage appliances.

@codecov

codecov Bot commented Jun 23, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 1.52672% with 258 lines in your changes missing coverage. Please review.
✅ Project coverage is 16.22%. Comparing base (432a306) to head (abf6aae).
⚠️ Report is 165 commits behind head on 4.20.

Files with missing lines Patch % Lines
...pervisor/kvm/storage/MultipathSCSIAdapterBase.java 0.00% 133 Missing ⚠️
.../hypervisor/kvm/storage/KVMStoragePoolManager.java 0.00% 54 Missing ⚠️
.../datastore/driver/AdaptiveDataStoreDriverImpl.java 0.00% 54 Missing ⚠️
...ud/hypervisor/kvm/storage/KVMStorageProcessor.java 0.00% 11 Missing ⚠️
...ava/com/cloud/network/as/AutoScaleManagerImpl.java 33.33% 5 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               4.20   #11071      +/-   ##
============================================
- Coverage     16.23%   16.22%   -0.01%     
+ Complexity    13378    13377       -1     
============================================
  Files          5657     5657              
  Lines        498865   499036     +171     
  Branches      60545    60571      +26     
============================================
- Hits          81014    80992      -22     
- Misses       408820   409010     +190     
- Partials       9031     9034       +3     
Flag Coverage Δ
uitests 4.00% <ø> (ø)
unittests 17.08% <1.52%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@DaanHoogland
DaanHoogland requested review from Copilot, rp- and vishesh92 and removed request for rp- June 23, 2025 11:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes several issues and enhances functionality in both autoscale VM hostname generation and storage connectivity, including improvements for handling disk connections asynchronously and build configuration refinements.

  • Introduces an optional configuration for including a hostname prefix for autoscale VMs.
  • Implements asynchronous disk connection handling and improves error messaging and logging across storage adapter scripts.
  • Adds several m2e IDE build annotations and fixes path issues in Maven POMs.

Reviewed Changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
server/src/main/java/com/cloud/network/as/AutoScaleManagerImpl.java Adds hostname generation methods with and without prefix for autoscale VMs.
server/src/main/java/com/cloud/network/as/AutoScaleManager.java Introduces a new config key to enable/disable the autoscale hostname prefix.
scripts/storage/multipath/startConnectVolume.sh Implements a shell script to connect volumes, with minor spelling issues in comments.
scripts/storage/multipath/finishConnectVolume.sh Provides a finishing connect script with similar spelling issues as startConnectVolume.sh.
scripts/storage/multipath/disconnectVolume.sh Updates background execution behavior for disk disconnect operations.
scripts/storage/multipath/connectVolume.sh Refactors volume connect logic to use dedicated start/finish connect scripts and a waiting loop.
pom.xml, plugins/*/pom.xml, engine/schema/pom.xml Updates Maven POM configurations and ensures m2e IDE compatibility.
plugins/storage/volume/adaptive/src/main/java/org/apache/cloudstack/storage/datastore/driver/AdaptiveDataStoreDriverImpl.java Modifies persistence method signatures and logging for volume operations.
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/MultipathSCSIAdapterBase.java Adjusts script timeout values and error messages, and adds new script properties.
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java, KVMStoragePoolManager.java, AsyncPhysicalDiskConnectorDecorator.java Improves exception handling and implements support for asynchronous physical disk connections.

Comment thread scripts/storage/multipath/startConnectVolume.sh Outdated
Comment thread scripts/storage/multipath/finishConnectVolume.sh Outdated
Comment thread scripts/storage/multipath/connectVolume.sh Outdated
Rene Glover and others added 6 commits June 23, 2025 10:18
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…stack/storage/datastore/driver/AdaptiveDataStoreDriverImpl.java

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…/storage/MultipathSCSIAdapterBase.java

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@rp-

rp- commented Jul 3, 2025

Copy link
Copy Markdown
Contributor

I ran the Linstor integration tests against it and had not problems, so green light from me, from that perspective.

@rp- rp- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@shwstppr

shwstppr commented Sep 3, 2025

Copy link
Copy Markdown
Contributor

@rg9975 can you please resolve the conflicts?

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16132

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian Build Failed (tid-15046)

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian Build Failed (tid-15047)

@DaanHoogland

Copy link
Copy Markdown
Contributor

@rg9975 , the deploy fails:

10:03:25 fatal: [pr11071-t15047-kvm-ol8-mgmt2]: FAILED! => {"changed": true, "cmd": "/usr/bin/cloudstack-setup-databases 'cloud’:’******'@'10.0.33.124' ", "delta": "0:00:00.114740", "end": "2025-12-23 09:03:25.297505", "failed": true, "rc": 1, "start": "2025-12-23 09:03:25.182765", "stderr": "\n\nWe apologize for below error:\n***************************************************************\nCannot find @MSCONF@/db.properties\n***************************************************************\nPlease run:\n\n    cloudstack-setup-databases -h\n\nfor full help", "stderr_lines": ["", "", "We apologize for below error:", "***************************************************************", "Cannot find @MSCONF@/db.properties", "***************************************************************", "Please run:", "", "    cloudstack-setup-databases -h", "", "for full help"], "stdout": "Mysql user name:cloud                                                           [ \u001b[92mOK\u001b[0m ]\nMysql user password:******                                                      [ \u001b[92mOK\u001b[0m ]\nMysql server ip:10.0.33.124                                                     [ \u001b[92mOK\u001b[0m ]\nMysql server port:3306                                                          [ \u001b[92mOK\u001b[0m ]\nChecking Cloud database files ...                                               ", "stdout_lines": ["Mysql user name:cloud                                                           [ \u001b[92mOK\u001b[0m ]", "Mysql user password:******                                                      [ \u001b[92mOK\u001b[0m ]", "Mysql server ip:10.0.33.124                                                     [ \u001b[92mOK\u001b[0m ]", "Mysql server port:3306                                                          [ \u001b[92mOK\u001b[0m ]", "Checking Cloud database files ...                                               "]}

@rg9975

rg9975 commented Dec 24, 2025

Copy link
Copy Markdown
Author

@DaanHoogland nothing in this PR changed the files/scripts related to the output you posted. I don't find that output in any of the workflow job output, can you point to where that is?

@DaanHoogland

Copy link
Copy Markdown
Contributor

I have no idea @rg9975 , but my first thought went to the engine/schema/pom.xml (line 111)

Cannot find @MSCONF@/db.properties , does it get copied properly? It is not an intermitted issue. At least two of the prior runs showed the same.

Comment thread engine/schema/pom.xml Outdated
@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16159

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16169

@apache apache deleted a comment from blueorangutan Jan 27, 2026
@apache apache deleted a comment from blueorangutan Jan 27, 2026
@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian Build Failed (tid-15290)

@DaanHoogland

Copy link
Copy Markdown
Contributor

@rg9975 sorry, systemvms don’t come up in a lab env with this PR

@DaanHoogland

Copy link
Copy Markdown
Contributor

@rg9975 , from an lab env attempt:

2026-01-27 10:35:12,197 ERROR [c.c.s.StorageManagerImpl] (qtp1390913202-16:[ctx-e7c7f6e4, ctx-d42028a4, ctx-087c6d5f]) (logid:d3f57cd1) Failed to register system VM template for hypervisor: KVM amd64 com.cloud.utils
.exception.CloudRuntimeException: Checksum failed for local template file
        at com.cloud.upgrade.SystemVmTemplateRegistration.validateTemplateFileForHypervisorAndArch(SystemVmTemplateRegistration.java:782)
        at com.cloud.upgrade.SystemVmTemplateRegistration.validateAndRegisterTemplate(SystemVmTemplateRegistration.java:788)
        at com.cloud.storage.StorageManagerImpl.registerSystemVmTemplateForHypervisorArch(StorageManagerImpl.java:3632)
        at com.cloud.storage.StorageManagerImpl$4.doInTransactionWithoutResult(StorageManagerImpl.java:3662)
        at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25)
        at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48)
        at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
        at com.cloud.utils.db.Transaction.execute(Transaction.java:45)
        at com.cloud.storage.StorageManagerImpl.registerSystemVmTemplateOnFirstNfsStore(StorageManagerImpl.java:3642)
        at com.cloud.storage.StorageManagerImpl.discoverImageStore(StorageManagerImpl.java:3593)
 

@abh1sar abh1sar modified the milestones: 4.20.3, 4.20.4 Feb 16, 2026
@poddm

poddm commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

@rg9975 -

Should MultipathSCSIAdapterBase.java or FiberChannelAdapter.java implement AsyncPhysicalDiskConnectorDecorator?

this check in KVMStoragePoolManagerwould would never be true:

if (adaptor instanceof AsyncPhysicalDiskConnectorDecorator) {

Java requires the interface to be declared on the class (or a superclass) for instanceof to match. Without it, the entire async disk connection path is dead code.

@DaanHoogland

Copy link
Copy Markdown
Contributor

instanceof

@rg9975 , are you still looking at this?

@rg9975 rg9975 closed this Aug 4, 2026
@rg9975

rg9975 commented Aug 4, 2026

Copy link
Copy Markdown
Author

This has been addressed in later versions, canceling/closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants