Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

vmware: improve solidfire storage plugin integration and fix cases#7761

Merged
yadvr merged 2 commits intoapache:4.18apache/cloudstack:4.18from
shapeblue:solidfire-fixesshapeblue/cloudstack:solidfire-fixesCopy head branch name to clipboard
Aug 25, 2023
Merged

vmware: improve solidfire storage plugin integration and fix cases#7761
yadvr merged 2 commits intoapache:4.18apache/cloudstack:4.18from
shapeblue:solidfire-fixesshapeblue/cloudstack:solidfire-fixesCopy head branch name to clipboard

Conversation

@yadvr
Copy link
Member

@yadvr yadvr commented Jul 21, 2023 •
edited
Loading

This fixes the following cases in which Solidfire storage integration caused issues when using Solidfire datadisks with VMware:

  1. Take Volume Snapshot of Solidfire data disk
  2. Delete an active Instance with Solidfire data disk attached
  3. Attach used existing Solidfire data disk to a running/stopped VM
  4. Stop and Start an instance with Solidfire data disks attached
  5. Expand disk by resizing Solidfire data disk by providing size
  6. Expand disk by changing disk offering for the Solidfire data disk
Function Operation Description Fix Status
Create Take Volume Snapshot Take data disk volume snapshot on active instance ✅
Delete Delete an active Instance Delete an active Instance with attached 1:1 data disks ✅
Update Attach used existing data disk Attach a previous used data volume to a instance ✅
Update Stop and Start a Instance Stop and Start a Instance with attached 1:1 data disks ✅
Update Resize Data volume Expand data disk on demand ✅
Update Change disk offering for the data volume Expand data disk changing disk offering ✅
Update Migrate data volume Migrate a data volume between primaries storages Solidfire based Unsupported feature

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

How Has This Been Tested?

Tested in a ACS 4.17 adv zone env with VMware and Solidfire setup. Tested all the following cases:

Screenshot 2023-07-21 at 1 02 29 PM

@yadvr yadvr added this to the 4.18.1.0 milestone Jul 21, 2023
This was linked to issues Jul 21, 2023
SolidFire: VM restart with data disk from the Solidfire Storage failure #7172
Closed
[SolidFire] Cannot attach volume which is detached from VM #7148
Closed
This was referenced Jul 21, 2023
Closed
Closed
@weizhouapache
Copy link
Contributor

weizhouapache commented Jul 21, 2023

great work @rohityadavcloud

@yadvr
Copy link
Member Author

yadvr commented Jul 21, 2023

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented Jul 21, 2023

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

DaanHoogland
DaanHoogland approved these changes Jul 21, 2023
View reviewed changes
Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm, but i cannot test this. Can you extract the workarounds (and containg blocks) in separate methods and where feasible add unit tests?

shwstppr
shwstppr approved these changes Jul 21, 2023
View reviewed changes
Copy link
Contributor

@shwstppr shwstppr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code change looks good in terms of logic. Left some comments. May need manual testing with different storage providers

...chestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java Show resolved Hide resolved
...isors/vmware/src/main/java/com/cloud/hypervisor/vmware/manager/VmwareStorageManagerImpl.java Outdated Show resolved Hide resolved
...ns/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java Outdated Show resolved Hide resolved
...ain/java/org/apache/cloudstack/storage/datastore/driver/SolidFirePrimaryDataStoreDriver.java Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Jul 21, 2023 •
edited
Loading

Codecov Report

Merging #7761 (76e5345) into 4.18 (b37834f) will decrease coverage by 0.01%.
Report is 1 commits behind head on 4.18.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##               4.18    #7761      +/-   ##
============================================
- Coverage     13.06%   13.05%   -0.01%     
- Complexity     9084     9086       +2     
============================================
  Files          2720     2720              
  Lines        257370   257394      +24     
  Branches      40124    40130       +6     
============================================
- Hits          33614    33612       -2     
- Misses       219534   219558      +24     
- Partials       4222     4224       +2
Files Changed Coverage Δ
...stack/engine/orchestration/VolumeOrchestrator.java 1.89% <0.00%> (ø)
...visor/vmware/manager/VmwareStorageManagerImpl.java 6.20% <0.00%> (-0.03%) ⬇️
...oud/hypervisor/vmware/resource/VmwareResource.java 4.48% <0.00%> (-0.01%) ⬇️
...cloud/storage/resource/VmwareStorageProcessor.java 0.32% <0.00%> (-0.01%) ⬇️
...astore/driver/SolidFirePrimaryDataStoreDriver.java 0.00% <0.00%> (ø)
...ud/hypervisor/vmware/mo/HostDatastoreSystemMO.java 0.00% <0.00%> (ø)
...com/cloud/hypervisor/vmware/util/VmwareHelper.java 8.65% <ø> (ø)

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@blueorangutan
Copy link

blueorangutan commented Jul 21, 2023

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6532

@yadvr
Copy link
Member Author

yadvr commented Jul 21, 2023

@blueorangutan test matrix

@blueorangutan
Copy link

blueorangutan commented Jul 21, 2023

@rohityadavcloud a [SF] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Jul 21, 2023

[SF] Trillian test result (tid-7135)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43125 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7135-xenserver-71.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_cancel_host_maintenace_with_no_migration_jobs Error 309.66 test_host_maintenance.py

@blueorangutan
Copy link

blueorangutan commented Jul 21, 2023

[SF] Trillian test result (tid-7137)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45954 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7137-kvm-centos7.zip
Smoke tests completed. 106 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_migrate_VM_and_root_volume Error 76.80 test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks Error 53.63 test_vm_life_cycle.py
test_hostha_enable_ha_when_host_in_maintenance Error 303.84 test_hostha_kvm.py

@blueorangutan
Copy link

blueorangutan commented Jul 26, 2023

@rohityadavcloud a [SF] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Jul 26, 2023

[SF] Trillian Build Failed (tid-7172)

@blueorangutan
Copy link

blueorangutan commented Jul 26, 2023

[SF] Trillian test result (tid-7171)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37558 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7171-xenserver-71.zip
Smoke tests completed. 108 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

@blueorangutan
Copy link

blueorangutan commented Jul 27, 2023

[SF] Trillian test result (tid-7173)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48780 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7173-kvm-centos7.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_migrate_VM_and_root_volume Error 76.57 test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks Error 55.40 test_vm_life_cycle.py

@yadvr yadvr added the Severity:Critical Critical bug label Aug 4, 2023
@weizhouapache
Copy link
Contributor

weizhouapache commented Aug 10, 2023

@rohityadavcloud
is this ready for merge ?

@yadvr
Copy link
Member Author

yadvr commented Aug 11, 2023

Yes, @weizhouapache but @NuxRo needs to sign off with his testing. I'll change the PR to draft.

@yadvr yadvr marked this pull request as draft August 11, 2023 11:12
@weizhouapache
Copy link
Contributor

weizhouapache commented Aug 11, 2023

Yes, @weizhouapache but @NuxRo needs to sign off with his testing. I'll change the PR to draft.

Ok thanks @rohityadavcloud

@weizhouapache
Copy link
Contributor

weizhouapache commented Aug 17, 2023

cc @NuxRo
plaese kindly approve it if it is tested good

weizhouapache
weizhouapache approved these changes Aug 18, 2023
View reviewed changes
Copy link
Contributor

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@yadvr yadvr marked this pull request as ready for review August 24, 2023 07:41
@yadvr
Copy link
Member Author

yadvr commented Aug 24, 2023

@NuxRo completed QA on this, against a community-supplied 4.17 Solidfire env (before he went on holidays). Now with @andrijapanicsb to finalise review. Thanks.

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented Aug 24, 2023

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

blueorangutan commented Aug 24, 2023

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6877

@weizhouapache
Copy link
Contributor

weizhouapache commented Aug 24, 2023

@blueorangutan test rocky8 vmware-67u3

@blueorangutan
Copy link

blueorangutan commented Aug 24, 2023

@weizhouapache a [SF] Trillian-Jenkins test job (rocky8 mgmt + vmware-67u3) has been kicked to run smoke tests

@weizhouapache
Copy link
Contributor

weizhouapache commented Aug 24, 2023

@blueorangutan test ubuntu22 vmware-70u3

@blueorangutan
Copy link

blueorangutan commented Aug 24, 2023

@weizhouapache [SF] unsupported parameters provided. Supported mgmt server os are: centos7, centos6, suse15, alma8, ubuntu18, ubuntu22, ubuntu20, rocky8, alma9. Supported hypervisors are: kvm-centos6, kvm-centos7, kvm-rocky8, kvm-alma8, kvm-alma9, kvm-ubuntu18, kvm-ubuntu20, kvm-ubuntu22, kvm-suse15, vmware-55u3, vmware-60u2, vmware-65u2, vmware-67u3, vmware-70u1, vmware-70u2, vmware-70u3, vmware-80, vmware-80u1, xenserver-65sp1, xenserver-71, xenserver-74, xcpng74, xcpng76, xcpng80, xcpng81, xcpng82

@weizhouapache
Copy link
Contributor

weizhouapache commented Aug 24, 2023

@blueorangutan test rocky8 vmware-70u3

@blueorangutan
Copy link

blueorangutan commented Aug 24, 2023

@weizhouapache a [SF] Trillian-Jenkins test job (rocky8 mgmt + vmware-70u3) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Aug 24, 2023

[SF] Trillian test result (tid-7531)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 49474 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7531-vmware-67u3.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_upgrade_kubernetes_cluster Failure 560.83 test_kubernetes_clusters.py

@blueorangutan
Copy link

blueorangutan commented Aug 24, 2023

[SF] Trillian test result (tid-7532)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 47727 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7532-vmware-70u3.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_upgrade_kubernetes_cluster Failure 585.85 test_kubernetes_clusters.py

@yadvr yadvr merged commit e964395 into apache:4.18 Aug 25, 2023
@yadvr yadvr deleted the solidfire-fixes branch August 25, 2023 06:06
@stardom3645 stardom3645 mentioned this pull request Aug 29, 2023
Closed
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shwstppr shwstppr shwstppr approved these changes

@DaanHoogland DaanHoogland DaanHoogland approved these changes

@harikrishna-patnala harikrishna-patnala harikrishna-patnala approved these changes

@weizhouapache weizhouapache weizhouapache approved these changes

Assignees

@NuxRo NuxRo

@andrijapanicsb andrijapanicsb

Labels

component:orchestration component:primary-storage component:solidfire component:storage component:vmware Severity:Critical Critical bug

Projects

None yet

Milestone

4.18.1.0

Development

Successfully merging this pull request may close these issues.

SolidFire: VM restart with data disk from the Solidfire Storage failure [SolidFire] Cannot attach volume which is detached from VM

8 participants

@yadvr @weizhouapache @blueorangutan @shwstppr @DaanHoogland @harikrishna-patnala @NuxRo @andrijapanicsb
Morty Proxy This is a proxified and sanitized view of the page, visit original site.