Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

[KVM] Fix VM migration error due to VNC password on libvirt limiting versions#6404

Merged
nvazquez merged 3 commits intoapache:4.16apache/cloudstack:4.16from
shapeblue:fixmigratevmvncissueshapeblue/cloudstack:fixmigratevmvncissueCopy head branch name to clipboard
May 23, 2022
Merged

[KVM] Fix VM migration error due to VNC password on libvirt limiting versions#6404
nvazquez merged 3 commits intoapache:4.16apache/cloudstack:4.16from
shapeblue:fixmigratevmvncissueshapeblue/cloudstack:fixmigratevmvncissueCopy head branch name to clipboard

Conversation

@nvazquez
Copy link
Contributor

@nvazquez nvazquez commented May 22, 2022 •
edited
Loading

Description

This PR fixes a VM migration issue described on #6402 (comment)

image

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

2022-05-22 03:56:07,297 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-77987975 job-41/job-42 ctx-91220c68) (logid:c4ad92ef) VM instance {id: "5", name: "i-2-5-VM", uuid: 
"6a2339af-5236-49aa-b9a0-65a2c9ce3952", type="User"} state transited from [Migrating] to [Running] with event [OperationFailed]. VM's original host: Host {"id": "1", "name": "ref-trl-
3127-k-M7-nicolas-vazquez-kvm1", "uuid": "9c851ff3-06f5-46e0-a822-f029a375748a", "type"="Routing"}, new host: Host {"id": "1", "name": "ref-trl-3127-k-M7-nicolas-vazquez-kvm1", "uuid"
: "9c851ff3-06f5-46e0-a822-f029a375748a", "type"="Routing"}, host before state transition: Host {"id": "4", "name": "ref-trl-3141-k-M7-nicolas-vazquez-kvm1", "uuid": "90107936-3a2d-48
06-b5af-561e97620664", "type"="Routing"}
2022-05-22 03:56:07,300 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-77987975 job-41/job-42 ctx-91220c68) (logid:c4ad92ef) Hosts's actual total CPU: 6300 and CPU after a
pplying overprovisioning: 12600
2022-05-22 03:56:07,301 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-77987975 job-41/job-42 ctx-91220c68) (logid:c4ad92ef) Hosts's actual total RAM: (6.77 GB) 7264485376
 and RAM after applying overprovisioning: (6.77 GB) 7264485376
2022-05-22 03:56:07,301 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-77987975 job-41/job-42 ctx-91220c68) (logid:c4ad92ef) release cpu from host: 4, old used: 500,reserv
ed: 0, actual total: 6300, total with overprovisioning: 12600; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2022-05-22 03:56:07,301 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-77987975 job-41/job-42 ctx-91220c68) (logid:c4ad92ef) release mem from host: 4, old used: (512.00 MB
) 536870912,reserved: (0 bytes) 0, total: (6.77 GB) 7264485376; new used: (0 bytes) 0,reserved:(0 bytes) 0; movedfromreserved: false,moveToReserveredfalse
2022-05-22 03:56:07,305 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-77987975 job-41/job-42 ctx-91220c68) (logid:c4ad92ef) Invocation exception, caused by: com.cloud.u
tils.exception.CloudRuntimeException: Exception during migrate: org.libvirt.LibvirtException: unsupported configuration: VNC password is 22 characters long, only 8 permitted
2022-05-22 03:56:07,305 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-77987975 job-41/job-42 ctx-91220c68) (logid:c4ad92ef) Rethrow exception com.cloud.utils.exception.
CloudRuntimeException: Exception during migrate: org.libvirt.LibvirtException: unsupported configuration: VNC password is 22 characters long, only 8 permitted
2022-05-22 03:56:07,309 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-77987975 job-41/job-42) (logid:c4ad92ef) Done with run of VM work job: com.cloud.vm.VmWorkMigrate fo
r VM 5, job origin: 41
2022-05-22 03:56:07,310 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-77987975 job-41/job-42) (logid:c4ad92ef) Unable to complete AsyncJobVO: {id:42, userId: 2, accountId
: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyABpjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZRdxQXtPtzYqAgAGSgAJc3JjSG9zdElkTAAJY2x1c3RlcklkdAAQTG
phdmEvbGFuZy9Mb25nO0wABmhvc3RJZHEAfgABTAAFcG9kSWRxAH4AAUwAB3N0b3JhZ2V0AA9MamF2YS91dGlsL01hcDtMAAZ6b25lSWRxAH4AAXhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VyS
WRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAABXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAF
dmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAAAXNxAH4ABwAAAAAAAAAEcQB-AAlwcQB-AAk, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, ini
tMsid: 32986154140800, completeMsid: null, lastUpdated: null, lastPolled: null, created: Sun May 22 03:56:06 UTC 2022, removed: null}, job origin:41
com.cloud.utils.exception.CloudRuntimeException: Exception during migrate: org.libvirt.LibvirtException: unsupported configuration: VNC password is 22 characters long, only 8 permitte
d
        at com.cloud.vm.VirtualMachineManagerImpl.migrate(VirtualMachineManagerImpl.java:2795)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrate(VirtualMachineManagerImpl.java:2658)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrate(VirtualMachineManagerImpl.java:5883)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  • Upgrade cloudstack-agent packages on host 1, containing this fix -> then upgrade host 1
  • Attempt migrations to host 2 -> SUCCESS

@nvazquez nvazquez added this to the 4.17.0.0 milestone May 22, 2022
@nvazquez
Copy link
Contributor Author

nvazquez commented May 22, 2022

@blueorangutan package

@nvazquez nvazquez mentioned this pull request May 22, 2022
Merged
12 tasks
@blueorangutan
Copy link

blueorangutan commented May 22, 2022

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

blueorangutan commented May 22, 2022

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3466

@nvazquez
Copy link
Contributor Author

nvazquez commented May 22, 2022

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented May 22, 2022

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

blueorangutan commented May 22, 2022

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3468

@nvazquez
Copy link
Contributor Author

nvazquez commented May 22, 2022

@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented May 22, 2022

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@leolleeooleo
Copy link
Contributor

leolleeooleo commented May 22, 2022

It works.
But, not host 2, shoud be host 1.

  • Upgrade cloudstack-agent packages on host 2, containing this fix -> then upgrade host 1

@weizhouapache
Copy link
Contributor

weizhouapache commented May 22, 2022

It works. But, not host 2, shoud be host 1.

  • Upgrade cloudstack-agent packages on host 2, containing this fix -> then upgrade host 1

@leolleeooleo
The source host should have this fix before vm migration.

@leolleeooleo
Copy link
Contributor

leolleeooleo commented May 22, 2022

It works. But, not host 2, shoud be host 1.

  • Upgrade cloudstack-agent packages on host 2, containing this fix -> then upgrade host 1

@leolleeooleo The source host should have this fix before vm migration.

@weizhouapache
That's what i mean.
In @nvazquez test, the source host is host 1.

@nvazquez
Copy link
Contributor Author

nvazquez commented May 22, 2022 •
edited
Loading

Yes, was a mistake on the PR description, have updated it - thanks for testing @leolleeooleo. Please submit your review under Files changed -> Review

weizhouapache
weizhouapache approved these changes May 22, 2022
View reviewed changes
Copy link
Contributor

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@leolleeooleo
Copy link
Contributor

leolleeooleo commented May 22, 2022

Yes, was a mistake on the PR description, have updated it - thanks for testing @leolleeooleo. Please submit your review under Files changed -> Review

"then upgrade host 1" is right.

  • Upgrade cloudstack-agent packages on host 1, containing this fix -> then upgrade host 1

@blueorangutan
Copy link

blueorangutan commented May 22, 2022

Trillian test result (tid-4228)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29318 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6404-t4228-kvm-centos7.zip
Smoke tests completed. 92 look OK, 0 have errors
Only failed tests results shown below:

Test Result Time (s) Test File

@nvazquez
Copy link
Contributor Author

nvazquez commented May 23, 2022

Thanks for the review @leolleeooleo @weizhouapache - I have refactored it to a simpler approach

@nvazquez
Copy link
Contributor Author

nvazquez commented May 23, 2022

@blueorangutan package

1 similar comment
@nvazquez
Copy link
Contributor Author

nvazquez commented May 23, 2022

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented May 23, 2022

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

blueorangutan commented May 23, 2022

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3471

@weizhouapache weizhouapache reopened this May 23, 2022
@weizhouapache
Copy link
Contributor

weizhouapache commented May 23, 2022

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented May 23, 2022

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

weizhouapache
weizhouapache approved these changes May 23, 2022
View reviewed changes
Copy link
Contributor

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@blueorangutan
Copy link

blueorangutan commented May 23, 2022

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3472

@weizhouapache
Copy link
Contributor

weizhouapache commented May 23, 2022

@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented May 23, 2022

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

weizhouapache
weizhouapache approved these changes May 23, 2022
View reviewed changes
Copy link
Contributor

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manually tested OK

DaanHoogland
DaanHoogland approved these changes May 23, 2022
View reviewed changes
Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

...vm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtMigrateCommandWrapper.java
Comment on lines +153 to +156
// Limit the VNC password in case the length is greater than 8 characters
// Since libvirt version 8 VNC passwords are limited to 8 characters
String vncPassword = org.apache.commons.lang3.StringUtils.truncate(to.getVncPassword(), 8);
xmlDesc = replaceIpForVNCInDescFileAndNormalizePassword(xmlDesc, target, vncPassword);
Copy link
Contributor

@DaanHoogland DaanHoogland May 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

separate methos?

Suggested change
// Limit the VNC password in case the length is greater than 8 characters
// Since libvirt version 8 VNC passwords are limited to 8 characters
String vncPassword = org.apache.commons.lang3.StringUtils.truncate(to.getVncPassword(), 8);
xmlDesc = replaceIpForVNCInDescFileAndNormalizePassword(xmlDesc, target, vncPassword);
/**
* Limit the VNC password in case the length is greater than 8 characters
* Since libvirt version 8 VNC passwords are limited to 8 characters
*/
private XmlDescOrSuchAThing getXmlDesc(....) {
String vncPassword = org.apache.commons.lang3.StringUtils.truncate(to.getVncPassword(), 8);
return replaceIpForVNCInDescFileAndNormalizePassword(xmlDesc, target, vncPassword);
}

@nvazquez nvazquez merged commit b1c8b5a into apache:4.16 May 23, 2022
@blueorangutan
Copy link

blueorangutan commented May 23, 2022

Trillian test result (tid-4229)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29967 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6404-t4229-kvm-centos7.zip
Smoke tests completed. 92 look OK, 0 have errors
Only failed tests results shown below:

Test Result Time (s) Test File

weizhouapache pushed a commit to weizhouapache/cloudstack that referenced this pull request May 24, 2022
@nvazquez @weizhouapache
[KVM] Fix VM migration error due to VNC password on libvirt limiting …
b5a6485 
…versions (apache#6404)

* [KVM] Fix VM migration error due to VNC password on libvirt limiting versions

* Fix passwd value

* Simplify implementation
@shwstppr shwstppr mentioned this pull request Aug 12, 2022
Closed
Pearl1594 pushed a commit to shapeblue/cloudstack that referenced this pull request Sep 6, 2022
@shwstppr @slavkap
@weizhouapache @nvazquez
el8 fixes backport (apache#155)
de14141 
* Extract the IO_URING configuration into the agent.properties (apache#6253)

When using advanced virtualization the IO Driver is not supported. The
admin will decide if want to enable/disable this configuration from
agent.properties file. The default value is true

* kvm: truncate vnc password to 8 chars (apache#6244)

This PR truncates the vnc password of kvm vms to 8 chars to support latest versions of libvirt.

* merge fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* [KVM] Enable IOURING only when it is available on the host (apache#6399)

* [KVM] Disable IOURING by default on agents

* Refactor

* Remove agent property for iouring

* Restore property

* Refactor suse check and enable on ubuntu by default

* Refactor irrespective of guest OS

* Improvement

* Logs and new path

* Refactor condition to enable iouring

* Improve condition

* Refactor property check

* Improvement

* Doc comment

* Extend comment

* Move method

* Add log

* [KVM] Fix VM migration error due to VNC password on libvirt limiting versions (apache#6404)

* [KVM] Fix VM migration error due to VNC password on libvirt limiting versions

* Fix passwd value

* Simplify implementation

Co-authored-by: slavkap <51903378+slavkap@users.noreply.github.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
@weizhouapache weizhouapache mentioned this pull request Jan 6, 2023
Closed
shwstppr pushed a commit to shapeblue/cloudstack that referenced this pull request Mar 17, 2023
@nvazquez @yadvr
[KVM] Fix VM migration error due to VNC password on libvirt limiting …
4b7d229 
…versions (apache#6404)

* [KVM] Fix VM migration error due to VNC password on libvirt limiting versions

* Fix passwd value

* Simplify implementation

(cherry picked from commit b1c8b5a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DaanHoogland DaanHoogland DaanHoogland approved these changes

@weizhouapache weizhouapache weizhouapache approved these changes

@yadvr yadvr Awaiting requested review from yadvr

@shwstppr shwstppr Awaiting requested review from shwstppr

@Pearl1594 Pearl1594 Awaiting requested review from Pearl1594

@slavkap slavkap Awaiting requested review from slavkap

+1 more reviewer

@leolleeooleo leolleeooleo leolleeooleo left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

component:kvm component:migration type:bug

Projects

No open projects
Apache CloudStack 4.17.0
Status: Done

Milestone

4.17.0.0

Development

Successfully merging this pull request may close these issues.

5 participants

@nvazquez @blueorangutan @leolleeooleo @weizhouapache @DaanHoogland
Morty Proxy This is a proxified and sanitized view of the page, visit original site.