Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

marvin: make deployDataCenter.py script compatible with python 2 and 3#5139

Merged
nvazquez merged 1 commit intoapache:mainapache/cloudstack:mainfrom
shapeblue:fix-py2-3-compatibilityshapeblue/cloudstack:fix-py2-3-compatibilityCopy head branch name to clipboard
Jul 13, 2021
Merged

marvin: make deployDataCenter.py script compatible with python 2 and 3#5139
nvazquez merged 1 commit intoapache:mainapache/cloudstack:mainfrom
shapeblue:fix-py2-3-compatibilityshapeblue/cloudstack:fix-py2-3-compatibilityCopy head branch name to clipboard

Conversation

@Pearl1594
Copy link
Contributor

@Pearl1594 Pearl1594 commented Jun 22, 2021

Description

This PR fixes a compatibility issue noticed with deployDataCenter.py script when run with python3 - the script reports the following exception:

Exception Occurred  while persisting DC Settings: ['Traceback (most recent call last):\n', '  File "tools/marvin/marvin/deployDataCenter.py", line 76, in __persistDcConfig\n    pickle.dump(self.__cleanUp, file_to_write)\n', 'TypeError: write() argument must be str, not bytes\n']

And hence doesn't create the dc_entries.obj file

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

How Has This Been Tested?

Ran the deployDataCenter script using python2 and python3 - it runs successfully and generates the dc_entries.obj file in human readable format (which is controlled by the protocol number passed)

@Pearl1594 Pearl1594 requested a review from DaanHoogland June 22, 2021 12:04
@weizhouapache
Copy link
Contributor

weizhouapache commented Jun 22, 2021

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented Jun 22, 2021

@weizhouapache a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link

blueorangutan commented Jun 22, 2021

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 324

@weizhouapache
Copy link
Contributor

weizhouapache commented Jun 22, 2021

@blueorangutan test kvm-ubuntu20 kvm-ubuntu20

@blueorangutan
Copy link

blueorangutan commented Jun 22, 2021

@weizhouapache unsupported parameters provided. Supported mgmt server os are: centos6, centos7, centos8, ubuntu18, ubuntu20. Supported hypervisors are: kvm-centos6, kvm-centos7, kvm-centos8, kvm-ubuntu18, kvm-ubuntu20, xenserver-71, xenserver-65sp1, vmware-70u1, vmware-67u3, vmware-65u2, vmware-60u2, vmware-55u3, xcpng76, xcpng80, xcpng81, xenserver-74, xcpng74

@weizhouapache
Copy link
Contributor

weizhouapache commented Jun 22, 2021

@blueorangutan test ubuntu20 kvm-ubuntu20

@blueorangutan
Copy link

blueorangutan commented Jun 22, 2021

@weizhouapache a Trillian-Jenkins test job (ubuntu20 mgmt + kvm-ubuntu20) has been kicked to run smoke tests

@weizhouapache
Copy link
Contributor

weizhouapache commented Jun 22, 2021

@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented Jun 22, 2021

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Jun 22, 2021

Trillian Build Failed (tid-1042)

@blueorangutan
Copy link

blueorangutan commented Jun 23, 2021

Trillian test result (tid-1041)
Environment: kvm-ubuntu20 (x2), Advanced Networking with Mgmt server u20
Total time taken: 42515 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5139-t1041-kvm-ubuntu20.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Failure 528.61 test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Error 528.63 test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers Failure 540.18 test_vpc_redundant.py
test_05_rvpc_multi_tiers Failure 510.48 test_vpc_redundant.py

@yadvr
Copy link
Member

yadvr commented Jun 23, 2021

@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented Jun 23, 2021

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Jun 24, 2021

Trillian test result (tid-1043)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 67119 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5139-t1043-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_storage_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 77 look OK, 11 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_01_nic Error 154.29 test_nic.py
test_02_deploy_and_upgrade_kubernetes_cluster Failure 555.81 test_kubernetes_clusters.py
test_01_isolate_network_FW_PF_default_routes_egress_true Failure 341.47 test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true Failure 430.94 test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false Failure 406.04 test_routers_network_ops.py
ContextSuite context=TestVMWareStoragePolicies>:setup Error 0.00 test_storage_policy.py
test_02_create_template_with_checksum_sha1 Error 65.58 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.44 test_templates.py
test_04_create_template_with_checksum_md5 Error 65.50 test_templates.py
test_05_create_template_with_no_checksum Error 65.45 test_templates.py
test_04_extract_template Failure 128.49 test_templates.py
ContextSuite context=TestISOUsage>:setup Error 0.00 test_usage.py
test_01_volume_usage Error 194.53 test_usage.py
test_10_attachAndDetach_iso Failure 1510.29 test_vm_life_cycle.py
ContextSuite context=TestVPCRedundancy>:setup Error 0.00 test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup Error 0.00 test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
test_disable_oobm_ha_state_ineligible Error 1511.20 test_hostha_kvm.py

@Pearl1594
Copy link
Contributor Author

Pearl1594 commented Jun 24, 2021

@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented Jun 24, 2021

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

DaanHoogland
DaanHoogland approved these changes Jun 24, 2021
View reviewed changes
Copy link
Contributor

@DaanHoogland DaanHoogland left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, have to find some time to test this but low risk so I'm ok with merge.
(if travis passes)

@yadvr
Copy link
Member

yadvr commented Jun 29, 2021

Travis is passing except that one job
@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented Jun 29, 2021

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@yadvr
Copy link
Member

yadvr commented Jul 5, 2021

@blueorangutan package

@blueorangutan
Copy link

blueorangutan commented Jul 5, 2021

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@yadvr yadvr added this to the 4.16.0.0 milestone Jul 5, 2021
@blueorangutan
Copy link

blueorangutan commented Jul 5, 2021

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 466

@Pearl1594
Copy link
Contributor Author

Pearl1594 commented Jul 5, 2021

@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented Jul 5, 2021

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Jul 6, 2021

Trillian test result (tid-1199)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 101255 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5139-t1199-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_affinity_groups_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_async_job.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_with_userdata.py
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_domain_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_loadbalance.py
Intermittent failure detected: /marvin/tests/smoke/test_metrics_api.py
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_nested_virtualization.py
Intermittent failure detected: /marvin/tests/smoke/test_network_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic_adapter_type.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_portforwardingrules.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 63 look OK, 25 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestInternalLb>:setup Error 0.00 test_internal_lb.py
ContextSuite context=TestAccounts>:setup Error 0.00 test_accounts.py
ContextSuite context=TestAddVmToSubDomain>:setup Error 0.00 test_accounts.py
test_DeleteDomain Error 3.93 test_accounts.py
test_forceDeleteDomain Failure 3.92 test_accounts.py
ContextSuite context=TestRemoveUserFromAccount>:setup Error 4.95 test_accounts.py
ContextSuite context=TestDeployVmWithAffinityGroup>:setup Error 0.00 test_affinity_groups_projects.py
test_01_create_iso_with_checksum_sha1 Error 66.49 test_iso.py
test_02_create_iso_with_checksum_sha256 Error 66.70 test_iso.py
test_02_create_iso_with_checksum_sha256 Error 66.70 test_iso.py
test_03_create_iso_with_checksum_md5 Error 66.37 test_iso.py
test_04_create_iso_with_no_checksum Error 69.68 test_iso.py
test_01_create_iso Failure 1512.87 test_iso.py
ContextSuite context=TestISO>:setup Error 3023.52 test_iso.py
ContextSuite context=TestAsyncJob>:setup Error 0.00 test_async_job.py
ContextSuite context=TestLoadBalance>:setup Error 0.00 test_loadbalance.py
test_list_clusters_metrics Error 1518.67 test_metrics_api.py
test_list_vms_metrics Error 0.13 test_metrics_api.py
ContextSuite context=TestDeployVirtioSCSIVM>:setup Error 0.00 test_deploy_virtio_scsi_vm.py
ContextSuite context=TestDeployVMFromISO>:setup Error 0.00 test_deploy_vm_iso.py
ContextSuite context=TestDeployVmWithVariedPlanners>:setup Error 0.00 test_deploy_vms_with_varied_deploymentplanners.py
ContextSuite context=TestNetworkACL>:setup Error 0.00 test_network_acl.py
test_delete_account Error 1523.48 test_network.py
test_delete_network_while_vm_on_it Error 1.09 test_network.py
test_deploy_vm_l2network Error 1.10 test_network.py
test_l2network_restart Error 2.15 test_network.py
ContextSuite context=TestPortForwarding>:setup Error 3.27 test_network.py
ContextSuite context=TestPublicIP>:setup Error 0.96 test_network.py
test_reboot_router Failure 0.03 test_network.py
test_releaseIP Error 0.41 test_network.py
ContextSuite context=TestRouterRules>:setup Error 0.44 test_network.py
ContextSuite context=TestAdapterTypeForNic>:setup Error 0.00 test_nic_adapter_type.py
ContextSuite context=TestDomainsServiceOfferings>:setup Error 1514.44 test_domain_service_offerings.py
test_01_invalid_upgrade_kubernetes_cluster Failure 0.01 test_kubernetes_clusters.py
test_02_deploy_and_upgrade_kubernetes_cluster Failure 0.01 test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster Failure 0.01 test_kubernetes_clusters.py
test_04_basic_lifecycle_kubernetes_cluster Failure 0.01 test_kubernetes_clusters.py
test_05_delete_kubernetes_cluster Failure 0.01 test_kubernetes_clusters.py
test_06_deploy_invalid_kubernetes_ha_cluster Failure 0.01 test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster Failure 0.01 test_kubernetes_clusters.py
test_08_deploy_and_upgrade_kubernetes_ha_cluster Failure 0.01 test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster Failure 0.01 test_kubernetes_clusters.py
test_01_add_delete_kubernetes_supported_version Error 60.53 test_kubernetes_supported_versions.py
ContextSuite context=TestListIdsParams>:setup Error 0.00 test_list_ids_parameter.py
test_nic_secondaryip_add_remove Error 1515.49 test_multipleips_per_nic.py
ContextSuite context=TestNestedVirtualization>:setup Error 0.00 test_nested_virtualization.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup Error 0.00 test_password_server.py
ContextSuite context=TestPortForwardingRules>:setup Error 0.00 test_portforwardingrules.py
ContextSuite context=TestPrivateGwACL>:setup Error 0.00 test_privategw_acl.py
ContextSuite context=TestProjectSuspendActivate>:setup Error 1518.98 test_projects.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true Failure 341.80 test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false Failure 336.27 test_routers_network_ops.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Failure 520.23 test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Error 520.25 test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers Failure 531.46 test_vpc_redundant.py

@Pearl1594
Copy link
Contributor Author

Pearl1594 commented Jul 6, 2021

@blueorangutan test

@blueorangutan
Copy link

blueorangutan commented Jul 6, 2021

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

blueorangutan commented Jul 6, 2021

Trillian test result (tid-1207)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34453 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5139-t1207-kvm-centos7.zip
Smoke tests completed. 88 look OK, 0 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File

nvazquez
nvazquez approved these changes Jul 13, 2021
View reviewed changes
Copy link
Contributor

@nvazquez nvazquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nvazquez
Copy link
Contributor

nvazquez commented Jul 13, 2021

Merging based on approvals and test results

@nvazquez nvazquez merged commit 2e376f2 into apache:main Jul 13, 2021
@nvazquez nvazquez added component:marvin Python Warning... Python code Ahead! labels Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yadvr yadvr yadvr approved these changes

@DaanHoogland DaanHoogland DaanHoogland approved these changes

@nvazquez nvazquez nvazquez approved these changes

Assignees

No one assigned

Labels

component:marvin Python Warning... Python code Ahead!

Projects

None yet

Milestone

4.16.0.0

Development

Successfully merging this pull request may close these issues.

6 participants

@Pearl1594 @weizhouapache @blueorangutan @yadvr @nvazquez @DaanHoogland
Morty Proxy This is a proxified and sanitized view of the page, visit original site.