- Getting Started Configuration
- OpenID & LDAP
- Custom TLS Certificate
- Custom OS Images
- Extra Files
- Network Interface Bonding
- Multipath disks
- Accessing your Cluster
- CephFS
- MDS Notifications
- CephFS Guard
- RBD
- S3
- S3 Multisite
- NVMe over Fabrics
- SMB
- NFS
- Ceph Keys
- Updating croit
- Replacing OSDs
- Replacing NIC
- PG numbers
- Updating Ceph
- Migrating or restoring croit container
- Backups
Networking & Switches
Getting the PXE configured
Admittedly, the biggest and probably only obstacle to using our software is to initially set up the network correctly so that a PXE boot is possible.
Depending on the switches used, the firmware version and the desired network configuration, you must set up the configuration according to the switch instructions.
Here are a few examples of typical configurations:
Arista Switches in MLAG LACP:
Switch A:
=======
interface Port-Channel101
switchport trunk native vlan 100
switchport trunk allowed vlan 100,101
switchport mode trunk
port-channel min-links 1
port-channel lacp fallback individual
port-channel lacp fallback timeout 5
mlag 101
!
interface Ethernet10/1
description "LACP to storage server #1"
switchport access vlan 100
channel-group 101 mode active
lacp port-priority 14000
!
Switch B:
=======
interface Port-Channel101
switchport trunk native vlan 100
switchport trunk allowed vlan 100,101
switchport mode trunk
port-channel min-links 1
port-channel lacp fallback individual
port-channel lacp fallback timeout 5
mlag 101
!
interface Ethernet10/1
description "LACP to storage server #1"
switchport access vlan 100
channel-group 101 mode active
lacp port-priority 15000
!
Juniper Switches in LACP:
Switch A:
=======
set interfaces xe-0/0/0 ether-options 802.3ad lacp force-up
Cisco Switches in LACP:
Switch A:
=======
interface Ethernet1/1
lacp rate fast
switchport
switchport access vlan 100
channel-group 1 mode active
no shutdown
interface port-channel1
switchport
switchport access vlan 100
no lacp suspend-individual
vpc 1
Dell Switches in LACP:
Switch A:
=======
interface ethernet1/1/14
no shutdown
channel-group 11 mode active
no switchport
interface port-channel11
no shutdown
switchport mode trunk
switchport access vlan 33
switchport trunk allowed vlan 30-31
lacp fallback enable
FS Switches in LACP:
SwitchA# configure terminal
SwitchA(config)# interface range HundredGigabitEthernet 1/1-2
SwitchA(config-if-range)# port-group 3 mode active
SwitchA(config-if-range)# lacp individual-port enable
SwitchA(config-if-range)# exit
SwitchA(config)# interface aggregateport 3
SwitchA(config-if-Aggregateport 3)#switch access vlan 10
SwitchA(config-if-Aggregateport 3)#
FS# show interface aggregateport 3 …
Aggregate Port Informations: Aggregate Number: 3
Name: "AggregatePort 3" Members: (count=2)
HundredGigabitEthernet 0/1 Link Status: Up Lacp Status: individual
HundredGigabitEthernet1/1 Link Status: Up Lacp Status: individual
Booting from a tagged VLAN
The examples above carry the boot network as the port's native (untagged) VLAN, which is the usual setup: the servers' firmware sends untagged DHCP, and so does the croit OS while it boots.
Where a switch cannot offer a native VLAN together with LACP fallback, or the site policy is "trunk ports only", croit can boot the servers over a tagged VLAN instead:
- Configure the VLAN in each server's firmware (UEFI setup, for example under the network device's VLAN configuration) so PXE itself runs tagged. The management node needs an interface on that VLAN too, for example
eth0.108. - In croit, open Config → Networks → Boot IP, pick the management node's address on that VLAN interface, expand show advanced and enter the tag as Boot VLAN ID.
Servers booting from then on run DHCP on the tagged VLAN and keep their management IP on a <interface>.<tag> sub-interface, which the NICs page shows as a VLAN on the boot NIC. Leave the field empty for an untagged boot network. Changing it only affects servers at their next boot.
