Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

How to configure the OpenVswitch using bond on Centos 7 or Oracle 7? #5426

New issue
Copy link
New issue
Copy link
Closed
Closed
How to configure the OpenVswitch using bond on Centos 7 or Oracle 7?#5426
Copy link
Labels
component:openvswitchtype:question
Milestone
unplanned
@xuanyuanaosheng

Description

@xuanyuanaosheng
xuanyuanaosheng
opened on Sep 9, 2021
Issue body actions
ISSUE TYPE
  • Enhancement Request
  • Documentation Report
OS / ENVIRONMENT
  1. OS Version: Red Hat Enterprise Linux Server release 7.9 (Maipo)
  2. Linux **********(hostname) 5.4.17-2102.204.4.4.el7uek.x86_64 fix rpm/deb build error #2 SMP Tue Aug 17 20:25:28 PDT 2021 x86_64 x86_64 x86_64 GNU/Linux
  3. cloudstack version: cloudstack-agent-4.15.1.0-1.el8.x86_64
  4. openvswitch-2.12.0-1.el7.x86_64

The cloudstack work node is using KVM + openvswitch (http://docs.cloudstack.apache.org/en/latest/installguide/hypervisor/kvm.html#configuring-the-network-bridges-for-openvswitch )

image

The related configuration is as below:

The switch configuration:

mode=4 and allowed vlan 136,290,291

interface Ethernet7/40
  description to server Jethro1 eth0/0
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 136,290-291
  channel-group 704 mode active
  no shutdown

First we create a main bridge connected to the bond0 interface. Next we create three fake bridges, each connected to a specific vlan tag.

# ovs-vsctl add-br cloudbr
# ovs-vsctl add-bond cloudbr bond0 ens2f0 ens2f1
# ovs-vsctl set port cloudbr trunks=136,290,291
# ovs-vsctl add-br mgmt0 cloudbr 136
# ovs-vsctl add-br cloudbr0 cloudbr 290
# ovs-vsctl add-br cloudbr1 cloudbr 291

The ifcfg-ens2f0 network configuration:

TYPE=Ethernet
BOOTPROTO=none
UUID=f8a7f9c7-b5fc-4887-a656-50b9e0cd787b
DEVICE=ens2f0
ONBOOT=yes

The ifcfg-ens2f1 network configuration:

TYPE=Ethernet
BOOTPROTO=none
UUID=0e8b0ab7-63c6-4704-9c27-01ee68098d6f
DEVICE=ens2f1
ONBOOT=yes

The ifcfg-bond0 configuration:

DEVICE=bond0
NAME=bond0
BOOTPROTO=none
ONBOOT=yes
HOTPLUG=no
TYPE=OVSBond
DEVICETYPE=ovs
OVS_BRIDGE=cloudbr
BOND_IFACES="ens2f0 ens2f1"
OVS_OPTIONS="bond_mode=balance-tcp lacp=active"

The ifcfg-mgmt0 configuration:

DEVICE=mgmt0
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=none
ONBOOT=yes
HOTPLUG=no
IPADDR=10.67.39.250
NETMASK=255.255.252.0
GATEWAY=10.67.39.254

The ifcfg-cloudbr0 configuration:

DEVICE=cloudbr0
BOOTPROTO=none
ONBOOT=yes
HOTPLUG=no
TYPE=OVSBridge
DEVICETYPE=ovs

The ifcfg-cloudbr1 configuration：

DEVICE=cloudbr1
BOOTPROTO=none
ONBOOT=yes
HOTPLUG=no
TYPE=OVSBridge
DEVICETYPE=ovs

The ovs-vsctl command is :

# ovs-vsctl show
fd8f7b60-f922-462f-b5c1-a9742e00bf89
    Bridge cloudbr
        Port cloudbr
            trunks: [136, 290, 291]
            Interface cloudbr
                type: internal
        Port "mgmt0"
            tag: 136
            Interface "mgmt0"
                type: internal
        Port "bond0"
            Interface "ens2f0"
            Interface "ens2f1"
        Port "cloudbr0"
            tag: 290
            Interface "cloudbr0"
                type: internal
        Port "cloudbr1"
            tag: 291
            Interface "cloudbr1"
                type: internal
    ovs_version: "2.12.0"

The nmcli command is

# nmcli d
DEVICE      TYPE         STATE         CONNECTION    
ens2f0      ethernet     connected     System ens2f0 
ens2f1      ethernet     connected     System ens2f1            
lo          loopback     unmanaged     --            
cloudbr     openvswitch  unmanaged     --            
cloudbr0    openvswitch  unmanaged     --            
cloudbr1    openvswitch  unmanaged     --            
mgmt0       openvswitch  unmanaged     --            
ovs-system  openvswitch  unmanaged     -- 

# nmcli c
NAME           UUID                                  TYPE      DEVICE 
System ens2f0  f8a7f9c7-b5fc-4887-a656-50b9e0cd787b  ethernet  ens2f0 
System ens2f1  0e8b0ab7-63c6-4704-9c27-01ee68098d6f  ethernet  ens2f1

The ip command is

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
4: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 48:df:37:5f:62:00 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::4adf:37ff:fe5f:6200/64 scope link 
       valid_lft forever preferred_lft forever
6: ens2f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 48:df:37:5f:62:08 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::4adf:37ff:fe5f:6208/64 scope link 
       valid_lft forever preferred_lft forever
18: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 0a:0e:20:80:8f:5d brd ff:ff:ff:ff:ff:ff
19: cloudbr: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 48:df:37:5f:62:00 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::4adf:37ff:fe5f:6200/64 scope link 
       valid_lft forever preferred_lft forever
20: mgmt0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 86:64:3b:b2:8c:48 brd ff:ff:ff:ff:ff:ff
    inet 10.67.39.250/22 brd 10.67.39.255 scope global mgmt0
       valid_lft forever preferred_lft forever
    inet6 fe80::8464:3bff:feb2:8c48/64 scope link 
       valid_lft forever preferred_lft forever
21: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 86:64:3b:b2:8c:48 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8464:3bff:feb2:8c48/64 scope link 
       valid_lft forever preferred_lft forever
22: cloudbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 86:64:3b:b2:8c:48 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8464:3bff:feb2:8c48/64 scope link 
       valid_lft forever preferred_lft forever

# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
4: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000
    link/ether 48:df:37:5f:62:00 brd ff:ff:ff:ff:ff:ff
6: ens2f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000
    link/ether 48:df:37:5f:62:08 brd ff:ff:ff:ff:ff:ff
18: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 0a:0e:20:80:8f:5d brd ff:ff:ff:ff:ff:ff
19: cloudbr: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 48:df:37:5f:62:00 brd ff:ff:ff:ff:ff:ff
20: mgmt0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 86:64:3b:b2:8c:48 brd ff:ff:ff:ff:ff:ff
21: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 86:64:3b:b2:8c:48 brd ff:ff:ff:ff:ff:ff
22: cloudbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 86:64:3b:b2:8c:48 brd ff:ff:ff:ff:ff:ff

The route command is

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.67.39.254    0.0.0.0         UG    0      0        0 mgmt0
10.67.36.0      0.0.0.0         255.255.252.0   U     0      0        0 mgmt0
169.254.0.0     0.0.0.0         255.255.0.0     U     1019   0        0 cloudbr
169.254.0.0     0.0.0.0         255.255.0.0     U     1020   0        0 mgmt0
169.254.0.0     0.0.0.0         255.255.0.0     U     1021   0        0 cloudbr0
169.254.0.0     0.0.0.0         255.255.0.0     U     1022   0        0 cloudbr1

The ovs-appctl bond/show command is

# ovs-appctl bond/show
---- bond0 ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
next rebalance: 736 ms
lacp_status: negotiated
lacp_fallback_ab: false
active slave mac: 48:df:37:5f:62:08(ens2f1)

slave ens2f0: enabled
  may_enable: true

slave ens2f1: enabled
  active slave
  may_enable: true
  hash 250: 6 kB load
Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    component:openvswitchtype:question

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      Morty Proxy This is a proxified and sanitized view of the page, visit original site.