Preparations

Deployment

Deployment Models

• Baremetal - all servers listed are baremetal machines, at least a single
  switch to interconnect and an instance for deployment (VM, laptop, a small
  baremetal machine)
• Hyperconverged - at least a single baremetal node, to create all listed
  servers as VMs and additionally an instance for deployment.
• Hybrid - some baremetal nodes to virtualize servers (controllers, network, infrastructure) and some to use to deliver performance (compute, Ceph OSD nodes)

Baremetal and Hybrid approaches can be used for production environments

Hyperconverged Environments

Advantages:

• simple and fast
• single baremetal node needed
• all listed servers backed by VMs
• all listed networks backed by Open vSwitch / Linux Bridges
• no physical switch needed

Disadvantages:

• less insights into networking
• less performance possible

Approaches to create a hyperconverged environment

We will need at least a single baremetal node with either:

• KVM | libvirt
• single-node OpenStack environment
• single-node Proxmox environment
• single-node Incus environment
  ...

Proxmox

# Install post-pve-install.sh                                               
bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/tools/pve/post-pve-install.sh)"

# Prepare a new template for VMs
# https://www.croit.io/blog/how-to-use-cloud-images-for-faster-vm-deployment-in-proxmox-ve

TEMPL_ID=9000

wget -O /tmp/noble-server-cloudimg-amd64.img \
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img

qm create $TEMPL_ID \
--name ubuntu-2404-cloud-init \
--description "Ubuntu 24.04 Cloud Init template" \
--ostype l26 \
--cpu cputype=host \
--cores 1 \
--sockets 1 \
--memory 2048 \
--scsihw virtio-scsi-pci \
--net0 virtio,bridge=vmbr0

qm importdisk $TEMPL_ID /tmp/noble-server-cloudimg-amd64.img local-lvm

qm set $TEMPL_ID --scsi0 local-lvm:vm-$TEMPL_ID-disk-0
qm set $TEMPL_ID --boot c --bootdisk scsi0

qm set $TEMPL_ID --serial0 socket --vga serial0

qm set $TEMPL_ID --ide2 local-lvm:cloudinit

qm set $TEMPL_ID --ipconfig0 ip=dhcp
qm cloudinit update $TEMPL_ID

qm template $TEMPL_ID

# Install packages
apt install -y  \
 bind9-dnsutils \
 curl           \
 git            \
 htop           \
 jq             \
 tcpdump        \
 tmux           \
 tmuxp          \
 crudini        \
 vim            \
 yq

Proxmox

# Create bridges                                                            
cat > /etc/network/interfaces.d/wrx <<EOF
auto cpln01
iface cpln01 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto dpln0
iface dpln0 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brmgmt0
iface brmgmt0 inet static
        address 10.14.0.8/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brsrvc0
iface brsrvc0 inet static
        address 10.34.10.8/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brcpln0
iface brcpln0 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brdpln0
iface brdpln0 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brlmgt0
iface brlmgt0 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brxtrn0
iface brxtrn0 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brstrg0
iface brstrg0 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brstrg1
iface brstrg1 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
auto brpblc0
iface brpblc0 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
EOF

systemctl restart networking.service
brctl show

Let's Go!

git clone https://github.com/codecap/openstack-workshop.git

cd openstack-workshop
./scripts/print-ssh-config  >> ~/.ssh/config

Configuration

Link

resolve_conf:
  nameservers:
    - 10.14.0.10
  searchdomains: []
##############################################################################
network_conf:
  public:
    iface:       pblc0
    bridgename:  vimbr0
    addr:        10.10.61.0/24
    netmask:     255.255.255.0
    gateway:     10.10.61.1/24
    nameservers: [10.10.11.99]
  management:
    iface:   mgmt0
    addr:    10.14.0.0/24
  services:
    iface:   srvc0
    addr:    10.34.10.0/24
...
##############################################################################
server_conf:
  - hostname:    "dns"
...

Scripts

create-vm	creates a new VM defined by parameters
print-create-env-commands	reads conf/env.yaml file and prints the commands required to create the full environment
print-destroy-env-commands	reads conf/env.yaml file, queries the current state, and prints the commands required to destroy the entire environment

create-vm --id 10 --name dns.wrx.sckt.net --cpu 1 --ram 4096 --disks '[8]' --netconf '
     {
       "mgmt0": {"ip": "10.14.0.10/24",  "macaddr": "bc:24:11:99:04:0a"},
       "srvc0": {"ip": "10.34.10.10/24", "macaddr": "bc:24:11:99:06:0a"}
     }'

How to use scripts

# Create a new environment
print-create-env-commands | bash

# Destroy the whole environment
print-destroy-env-commands | bash

# If you need to rebuild only a set or a single server, use grep with a matching filter
print-destroy-env-commands | grep <NODE_FILTER> | bash
print-create-env-commands  | grep <NODE_FILTER> | bash

Create a workshop environment

BASE_PATH=https://raw.githack.com/codecap/openstack-workshop/main
PROXY=http://proxy.wrx.sckt.net:3128
INFRA_NODES="dns|proxy|registry"
DEPLOY_NODES="deployment|recorder"

# Infra nodes
print-create-env-commands  | grep -E "$INFRA_NODES" | bash
ssh -t dns      "sudo -i bash -c 'curl -L $BASE_PATH/scripts/infra/dns.sh      | bash'"
ssh -t proxy    "sudo -i bash -c 'curl -L $BASE_PATH/scripts/infra/proxy.sh    | bash'"
ssh -t registry "sudo -i bash -c 'curl -L $BASE_PATH/scripts/infra/registry.sh | bash'"

# Deployment nodes
print-create-env-commands  | grep -E "$DEPLOY_NODES" | bash
ssh -t recorder "sudo -i bash -c 'curl --proxy $PROXY -L $BASE_PATH/scripts/infra/recorder.sh | bash'"
scp /root/.ssh/id_rsa* deployment:/home/deploy/.ssh

# Environment nodes
print-create-env-commands  | grep -v -E "$INFRA_NODES|$DEPLOY_NODES"   | bash

Configure a Tunnel

To access environment

* Generate a new ECDSA key with **PuTTYgen** and save it on your system
* Put the public key in ~/.ssh/authorized_keys on:
  * hypervisor (root)
  * deployment (deploy)

* Start PuTTY. Go to Connection -> SSH -> Auth -> Credentials. Load the Private Key
* Go to Connection -> Data. Put "root" for "Auto-login username"
* Go to Session. Put the IP of your hypervisor
* Put the name for the session - "hypervisor". Save.
* Press "Open" button, a new session should be opened

* Create a new session with PuTTY
  * Put 10.14.0.24 into "Host Name" field
  * Under Connection -> Data, put "deploy" for "Auto-login username"
  * Under Connection -> SSH -> Auth -> Credentials load the private key
* Go to Connection -> SSH -> Tunnels
  * Choose Dynamic
  * Source Port: 8888
  * Press "Add" button
* Go to Connection -> Proxy
  * Set type to "SSH to proxy and port forward"
  * Set Proxy hostname to "hypervisor"
  * Set port to 22
* Go to Session
  * Put "deployment" in "Saved Sessions"
  * Press "Save"
  * Press "Open"

Configure Proxy

* Open Firefox
* Go to Settings
* Search for "proxy"
* Choose "Manual Configuration"
* Put "127.0.0.1" in SOCKS Host field, Port: 8888
* Choose SOCKS v5

SSH Config

# On deployment node create SSH config to access the workshop environment   
cat >> ~/.ssh/config <<EOF
Host hypervisor hypervisor.wrx.sckt.net
  User root

Host *.mgmt *.mgmt.wrx.sckt.net *.wrx.sckt.net
  User  deploy

Host *
  User                  deploy
  StrictHostKeyChecking no
  UserKnownHostsFile    /dev/null
EOF

Some Common Configs

# On deployment node check out the repository                               
cd ~
git clone https://github.com/codecap/openstack-workshop.git

ln -s ~/openstack-workshop/kolla-ansible openstack
ln -s ~/openstack-workshop/cephadm       ceph

# Install and configure tmuxp
sudo apt install pipx -y
pipx install tmuxp

cat >> ~/.profile <<EOF

alias wrx-stack='tmuxp load -y ~/openstack-workshop/conf/tmuxp.yaml'
EOF