systemvmtemplate: use latest LTS kernel from buster-ports by yadvr · Pull Request #5073 · apache/cloudstack

yadvr · Jun 3, 2021

Use the latest v5.10 Linux kernel (LTS) from buster-ports and fixes systemvmtemplate build issue on main branch.

Signed-off-by: Rohit Yadav rohit.yadav@shapeblue.com

Use the latest v5.10 Linux kernel (LTS) from buster-ports Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> (cherry picked from commit 2dd1650276e7fa31fc3b920b67be8396647c279d) Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

weizhouapache · Jun 4, 2021

@rhtyd good.
maybe consider #3900 and set a smaller template size ?

yadvr · Jun 4, 2021

@weizhouapache I think it would be too much maintenance work for gain to maintain separate systemvmtemplate for each hypervisor. These days users can download 100MBs more in few extra seconds.

weizhouapache · Jun 4, 2021

tools/appliance/systemvmtemplate/scripts/apt_upgrade.sh

  apt-get autoclean
  apt-get clean
+  reboot
 }


@rhtyd might 'reboot' be an issue in packer processing ?

No see template.json

weizhouapache · Jun 4, 2021

tools/appliance/systemvmtemplate/scripts/configure_grub.sh


 function configure_grub() {
+  # Remove old kernel
+  apt-get remove -y --purge linux-image-amd64 linux-image-4.19.0-16-amd64 || true


@rhtyd two potential issues in my opinion
(1) the versions are different in debian releases (debian 10.9.0: kernel 4.19.0.16.debian 10.8.0: kernel 4.19.0.14,etc)
(2) might there be more than 2 versions ? for example, one new kernel by apt-get -q -y dist-upgrade, another new kernel by apt-get -q -y dist-upgrade -t buster-backports ?

I see, we can explore some general way of removing kernels that are not in use.

weizhouapache · Jun 4, 2021

@weizhouapache I think it would be too much maintenance work for gain to maintain separate systemvmtemplate for each hypervisor. These days users can download 100MBs more in few extra seconds.

@rhtyd yes, agree. let's use same template for hypervisors.

yadvr · Jun 7, 2021

tools/appliance/systemvmtemplate/template.json

-        "scripts/apt_upgrade.sh",
+        "scripts/apt_upgrade.sh"
+      ],
+      "expect_disconnect": true


@weizhouapache this handles for reboot case. You can read more about it here https://www.packer.io/docs/provisioners/shell

@rhtyd good. thanks for the info.

yadvr · Jun 7, 2021

tools/appliance/systemvmtemplate/scripts/configure_grub.sh


 function configure_grub() {
+  # Remove old kernel
+  apt-get remove -y --purge linux-image-amd64 linux-image-4* || true


@weizhouapache would this help, essentially saying remove all the old 4.x kernels. As the new/LTS Linux kernel are in the 5.x series - https://www.kernel.org/category/releases.html

@rhtyd ok, I will test it.

yadvr · Jun 17, 2021

Is this good to be merged @weizhouapache @Pearl1594

weizhouapache

@rhtyd yes, it is good to me.

Will we wait for debian 11 which is planned to be released in July ?

yadvr · Jun 17, 2021

@weizhouapache yes we should update to Debian 11 if/when it comes before 4.16. In the meanwhile we can use the same Linux kernel backport same as Debian 11.

weizhouapache · Jun 17, 2021

@weizhouapache yes we should update to Debian 11 if/when it comes before 4.16. In the meanwhile we can use the same Linux kernel backport same as Debian 11.

@rhtyd ok, let's merge it when Pearl approves.

the codename of debian11 is 'Bullseye', cool.
bullseye-backports already exists, good.
http://ftp.debian.org/debian/dists/bullseye-backports/

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr · Jun 17, 2021

@blueorangutan package

blueorangutan · Jun 17, 2021

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · Jun 17, 2021

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 264

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr · Jun 17, 2021

tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh

  [[ -f /bin/vhd-util ]] && return

-  wget --no-check-certificate https://github.com/shapeblue/cloudstack-nonoss/raw/master/vhd-util -O /bin/vhd-util
+  wget --no-check-certificate https://github.com/shapeblue/cloudstack-nonoss/raw/main/vhd-util -O /bin/vhd-util


jfyi @sureshanaparti - fixed this

noted, thanks @rhtyd

yadvr · Jun 17, 2021

@blueorangutan test matrix

blueorangutan · Jun 17, 2021

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

yadvr · Jun 17, 2021

@blueorangutan test matrix

blueorangutan · Jun 17, 2021

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

blueorangutan · Jun 18, 2021

Trillian test result (tid-985)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44329 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5073-t985-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	526.36	test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Error`	526.37	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	463.68	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	504.96	test_vpc_redundant.py

blueorangutan · Jun 18, 2021

Trillian test result (tid-988)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45596 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5073-t988-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 86 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_volume_usage	`Error`	82.34	test_usage.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	605.14	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	525.84	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	538.46	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	538.48	test_vpc_redundant.py

weizhouapache · Jul 15, 2021

@rhtyd good. are you going to build new systemvm template and upload to download.cloudstack.org ?

yadvr · Jul 15, 2021

@weizhouapache the current 4.16 systemvmtemplate from this PR has been already on download.cloudstack.org for weeks now, all PRs being tested against that systemvmtemplate

weizhouapache · Jul 15, 2021

@weizhouapache the current 4.16 systemvmtemplate from this PR has been already on download.cloudstack.org for weeks now, all PRs being tested against that systemvmtemplate

@rhtyd nice, good to know. thanks

yadvr added the component:systemvm-template label Jun 3, 2021

yadvr added this to the 4.16.0.0 milestone Jun 3, 2021

yadvr mentioned this pull request Jun 3, 2021

systemvmtemplate: reduce template by using cloud kernel #5070

Closed

5 tasks

yadvr requested a review from weizhouapache June 3, 2021 17:35

weizhouapache reviewed Jun 4, 2021

View reviewed changes

yadvr commented Jun 7, 2021

View reviewed changes

Update configure_grub.sh

db3d1f7

yadvr commented Jun 7, 2021

View reviewed changes

yadvr requested a review from weizhouapache June 7, 2021 10:09

weizhouapache mentioned this pull request Jun 7, 2021

systemvm with overlayfs #5083

Closed

12 tasks

yadvr requested a review from Pearl1594 June 16, 2021 03:25

weizhouapache approved these changes Jun 17, 2021

View reviewed changes

yadvr added 2 commits June 17, 2021 15:58

Merge remote-tracking branch 'origin/main' into 416-systemvmtemplate

53e4408

changes

d66051a

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

fix failing build

4c8f726

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr commented Jun 17, 2021

View reviewed changes

yadvr added the Severity:BLOCKER label Jun 17, 2021

yadvr requested a review from sureshanaparti June 17, 2021 13:04

apache deleted a comment from blueorangutan Jun 17, 2021

Update install_systemvm_packages.sh

c905bd5

yadvr merged commit cc27c70 into apache:main Jul 15, 2021

Search code, repositories, users, issues, pull requests...

Conversation

yadvr commented Jun 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weizhouapache commented Jun 4, 2021

Uh oh!

yadvr commented Jun 4, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Jun 4, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yadvr commented Jun 17, 2021

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented Jun 17, 2021

Uh oh!

weizhouapache commented Jun 17, 2021

Uh oh!

yadvr commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yadvr commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

yadvr commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 18, 2021

Uh oh!

blueorangutan commented Jun 18, 2021

Uh oh!

weizhouapache commented Jul 15, 2021

Uh oh!

yadvr commented Jul 15, 2021

Uh oh!

weizhouapache commented Jul 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yadvr commented Jun 3, 2021 •

edited

Loading