How to Turn a GL.iNet Flint 3 with OpenWrt into QNetd for a Proxmox Cluster

1. The environment used

In my installation:

Código

Router:
GL.iNet Flint 3 / GL-BE9300
IP: 192.168.1.1
OpenWrt 23.05-SNAPSHOT
ARM64 architecture

Cluster:
Name: hectormox

Node 1:
miniproxmox
192.168.1.6

Node 2:
zimaproxmox
192.168.1.7

The minimal Debian system was installed at:

/opt/debian-qnetd

QNetd listens on the standard port:

TCP 5403

2. Connect to the Flint 3 over SSH

From Windows, we can use the SSH client included with PowerShell:

ssh root@192.168.1.1

Enter the router’s administrator password.

Once connected, we should see something similar to:

Código

BusyBox
OpenWrt
root@GL-BE9300:~#

Before installing anything, it is worth finding out exactly what we have.

3. Check the system, architecture, packages, and available space

Run:

bash

cat /etc/os-release
cat /etc/openwrt_release

uname -a
uname -m

command -v opkg
command -v apk

opkg print-architecture

df -h

On my Flint, I found a system with:

OpenWrt 23.05-SNAPSHOT

aarch64

and opkg as the package manager.

I also checked which tools were available:

bash

command -v ar
command -v tar
command -v gzip
command -v xz
command -v wget
command -v curl
command -v mount
command -v chroot

And, especially, whether OpenWrt already provided Corosync:

bash

opkg update
opkg list | grep -Ei 'corosync|qnetd|qdevice|libnss|libqb'

My firmware did not include corosync-qnetd.

So we could not simply run:

opkg install corosync-qnetd

as we would with a native OpenWrt package.

4. Why use a minimal Debian system inside OpenWrt

Instead of trying to compile Corosync and all its dependencies directly for OpenWrt, I chose a much simpler solution:

create a minimal ARM64 Debian system in a directory and run QNetd through chroot.

OpenWrt remains the router’s operating system. We do not install another kernel, virtualize anything, or replace the firmware.

The layout is simply:

Código

OpenWrt
└── /opt/debian-qnetd
    └── Debian 12 ARM64
        └── corosync-qnetd

In other words, the Flint continues to work as an OpenWrt router, but it has a small Debian environment from which we can run software that is not available in its repositories.

5. Install debootstrap on the Flint

First, update the package indexes:

opkg update

Install debootstrap:

opkg install debootstrap

Check that it is available:

bash

command -v debootstrap
debootstrap --version

Then check the available space:

df -h /overlay

In my case, several gigabytes were free, which was enough for a minimal Debian system.

6. GPG issue with Debian Bookworm

This is where one of the first issues specific to my OpenWrt version appeared.

debootstrap needs to verify the signatures of the Debian repositories, but the gpgv initially available on the router was too old for some signatures used by Bookworm.

The solution was to install:

opkg install gpgv2

We can check it with:

gpgv2 --version

We also need the official Debian keys.

Create a temporary directory:

bash

BASE="/tmp/qnetd-bootstrap"

mkdir -p "$BASE"
mkdir -p "$BASE/bin"

Download the debian-archive-keyring package for Debian Bookworm.

It is a good idea to check which package version is current when following this guide, rather than relying on a specific URL indefinitely. instead of permanently copying a specific URL.

Once it has been downloaded:

DEB="$BASE/debian-archive-keyring.deb"

Extract its contents:

bash

ar p "$DEB" data.tar.xz |
    xz -dc |
    tar -xf - -C "$BASE"

The key is available at:

bash

KEY="$BASE/usr/share/keyrings/debian-archive-keyring.gpg"
ls -l "$KEY"

To make sure debootstrap uses gpgv2, create a small temporary override:

ln -sf /usr/bin/gpgv2 "$BASE/bin/gpgv"

This only affects the PATH used when running debootstrap.

7. Create a Debian 12 ARM64 minbase system

Set the installation path:

TARGET="/opt/debian-qnetd"

Run:

bash

PATH="$BASE/bin:$PATH" \
debootstrap \
    --arch=arm64 \
    --variant=minbase \
    --keyring="$KEY" \
    bookworm \
    "$TARGET" \
    https://deb.debian.org/debian

This downloads and installs a minimal Debian 12 system in:

/opt/debian-qnetd

Check its contents:

ls -la /opt/debian-qnetd

Then enter it manually:

chroot /opt/debian-qnetd /bin/bash

To return to OpenWrt:

exit

8. Update Debian and install QNetd

From OpenWrt itself, we can run commands directly inside the chroot:

chroot "$TARGET" /usr/bin/apt-get update

Before installing it, check that the package is available:

bash

chroot "$TARGET" \
    /usr/bin/apt-cache policy corosync-qnetd

In my case, Debian offered:

corosync-qnetd 3.0.3-1

The initial installation found some pending dependencies in the minbase environment.

First, I repaired the package database with:

bash

chroot "$TARGET" \
    /usr/bin/apt-get --no-remove -y --fix-broken install

Then I installed:

bash

chroot "$TARGET" \
    /usr/bin/apt-get install -y \
    --no-install-recommends \
    corosync-qnetd \
    libnss3-tools

Among the dependencies Debian installed were NSS libraries, Perl, netbase and other required packages.

9. `/proc` and `/dev/pts` during installation

During installation, some warnings appeared concerning /proc and /dev/pts.

I mounted these temporarily:

bash

mount -o bind \
    /proc \
    /opt/debian-qnetd/proc

And:

bash

mount -o bind \
    /dev/pts \
    /opt/debian-qnetd/dev/pts

This helped us while preparing Debian.

We later confirmed that QNetd does not need these bind mounts to run normally, so they are not part of the final service.

10. Initialize the QNetd certificates

QNetd uses an NSS database to manage its certificates.

Our Debian system includes this utility:

corosync-qnetd-certutil

Initialize it from OpenWrt:

bash

chroot /opt/debian-qnetd \
    /usr/bin/corosync-qnetd-certutil \
    -i

From OpenWrt, the NSS database is located at:

/opt/debian-qnetd/etc/corosync/qnetd/nssdb

Check it with:

bash

ls -lah \
    /opt/debian-qnetd/etc/corosync/qnetd/nssdb

Inside the chroot, the same path is:

/etc/corosync/qnetd/nssdb

The public CA is located at:

/opt/debian-qnetd/etc/corosync/qnetd/nssdb/qnetd-cacert.crt

11. Check the user `coroqnetd`

The Debian package creates a dedicated user to run QNetd.

Check its IDs:

chroot /opt/debian-qnetd id coroqnetd

In my installation, I got:

UID 100

GID 101

This matters because later I will run QNetd with:

--userspec=100:101

Do not copy these numbers without checking them first.

Run this on each installation:

chroot /opt/debian-qnetd id coroqnetd

and use the values it returns.

We also check the NSS database permissions:

bash

DB="/opt/debian-qnetd/etc/corosync/qnetd/nssdb"
ls -ln "$DB"

In my case, I had to adjust:

bash

chgrp 101 \
    "$DB/pwdfile.txt" \
    "$DB/pkcs11.txt"

chmod 640 \
    "$DB/pwdfile.txt" \
    "$DB/pkcs11.txt"

Again, 101 was the specific GID in my installation.

12. Test QNetd manually

Before turning it into a service, it is important to check that the daemon actually starts.

I tested:

bash

timeout -s TERM 10 \
    chroot --userspec=100:101 \
    /opt/debian-qnetd \
    /usr/bin/corosync-qnetd \
    -f \
    -4 \
    -l 192.168.1.1 \
    -p 5403 \
    -s req

The parameters used are:

-f: run in the foreground.
-4: use IPv4.
-l 192.168.1.1: address where QNetd listens.
-p 5403: port.
-s req: require TLS in the version we are using.

Before copying this, check the help for the installed version:

bash

chroot /opt/debian-qnetd \
    /usr/bin/corosync-qnetd \
    -h

13. OpenWrt’s LD_PRELOAD issue

When running Debian programs from OpenWrt, warnings appeared about an OpenWrt firmware LD_PRELOAD variable.

To prevent the chroot process from inheriting OpenWrt-specific variables, I ended up running it with a clean environment:

bash

/usr/bin/env -i \
    PATH=/usr/sbin:/usr/bin:/sbin:/bin \
    ...

This detail became part of the final service.

14. Create the QNetd service in OpenWrt

This is one of the most important differences from a standard Debian system.

On Debian, we would use systemd.

On OpenWrt, we use procd and scripts under:

/etc/init.d/

Create:

bash

cat > /etc/init.d/qnetd-debian <<'EOF'
#!/bin/sh /etc/rc.common

USE_PROCD=1
START=98
STOP=10

start_service() {
    procd_open_instance

    procd_set_param command \
        /usr/bin/env -i \
        PATH=/usr/sbin:/usr/bin:/sbin:/bin \
        /usr/sbin/chroot --userspec=100:101 \
        /opt/debian-qnetd \
        /usr/bin/corosync-qnetd \
        -f -4 -l 192.168.1.1 -p 5403 -s req

    procd_set_param respawn 3600 5 5
    procd_set_param stdout 1
    procd_set_param stderr 1

    procd_close_instance
}
EOF

Set the permissions:

chmod 755 /etc/init.d/qnetd-debian

Enable automatic startup:

/etc/init.d/qnetd-debian enable

Check the links that were created:

ls -l /etc/rc.d/*qnetd*

On my Flint, these were:

/etc/rc.d/S98qnetd-debian

/etc/rc.d/K10qnetd-debian

Start the service:

/etc/init.d/qnetd-debian start

15. Check that QNetd is running

First, find the process:

pidof corosync-qnetd

Then check the port:

netstat -lntp 2>/dev/null | grep 5403

We should see:

192.168.1.1:5403 LISTEN

We can also check the full process:

ps w | grep '[c]orosync-qnetd'

And query procd directly:

bash

ubus call service list \
    '{"name":"qnetd-debian"}'

A healthy status contains:

"running": true

In my case, we could also see:

Código

/usr/bin/corosync-qnetd
-f
-4
-l 192.168.1.1
-p 5403
-s req

16. Check respawning

The service includes:

procd_set_param respawn 3600 5 5

This allows procd to restart QNetd if the process exits unexpectedly.

During testing, we killed the process manually and confirmed that it reappeared with a new PID and started listening again at:

192.168.1.1:5403

17. Prepare Proxmox

Now move to the first cluster node:

ssh root@192.168.1.6

Check the current status:

pvecm status

And:

corosync-qdevice-tool -s

My cluster is called:

hectormox

Previously, it had another QNetd at:

192.168.1.5:5403

The new one will be:

192.168.1.1:5403

18. Back up before changing QDevice

Before touching the certificates, I made a backup of:

/etc/corosync/qdevice

/etc/pve/corosync.conf

For example:

bash

BACKUP="/root/qdevice-backup-$(hostname)-$(date +%Y%m%d-%H%M%S).tar.gz"

tar -czf "$BACKUP" \
    -C / \
    etc/corosync/qdevice \
    etc/pve/corosync.conf

sha256sum "$BACKUP"

ls -lh "$BACKUP"

I do not recommend continuing without this backup.

19. Create a new NSS database on Proxmox

On miniproxmox:

bash

DIR="/root/qnetd-migration"
DB="$DIR/nssdb-new"
PASS="$DIR/pwdfile-new.txt"

mkdir -p "$DIR"
mkdir -m 700 "$DB"

: > "$PASS"
chmod 600 "$PASS"

Inicializamos NSS:

certutil -N \
    -d "sql:$DB" \
    -f "$PASS"

20. Copy the Flint CA to Proxmox

From miniproxmox:

bash

scp \
    root@192.168.1.1:/opt/debian-qnetd/etc/corosync/qnetd/nssdb/qnetd-cacert.crt \
    /root/qnetd-migration/qnetd-cacert.crt

Import the CA:

bash

certutil -A \
    -d "sql:$DB" \
    -n "QNet CA" \
    -t "CT,c,c" \
    -i "$DIR/qnetd-cacert.crt" \
    -f "$PASS"

Check:

bash

certutil -L \
    -d "sql:$DB" \
    -f "$PASS"

We should see:

QNet CA CT,c,c

21. Create the cluster CSR

Generate some entropy:

bash

dd if=/dev/urandom \
    of="$DIR/noise-new.txt" \
    bs=1024 \
    count=1 \
    status=none

Create the CSR:

bash

CSR="$DIR/qdevice-net-node.crq"
certutil -R \
    -d "sql:$DB" \
    -f "$PASS" \
    -z "$DIR/noise-new.txt" \
    -s "CN=hectormox" \
    -g 3072 \
    -o "$CSR"

Replace:

hectormox

with the actual cluster name.

Check it with:

pvecm status

22. Send the CSR to the Flint

From Proxmox:

bash

scp \
    "$CSR" \
    root@192.168.1.1:/opt/debian-qnetd/tmp/hectormox.crq

Connect to the Flint again:

ssh root@192.168.1.1

Then sign it:

bash

chroot /opt/debian-qnetd \
    /usr/bin/corosync-qnetd-certutil \
    -s \
    -c /tmp/hectormox.crq \
    -n hectormox

The signed certificate is located at:

/opt/debian-qnetd/etc/corosync/qnetd/nssdb/cluster-hectormox.crt

23. Retrieve the signed certificate

From miniproxmox:

bash

scp \
    root@192.168.1.1:/opt/debian-qnetd/etc/corosync/qnetd/nssdb/cluster-hectormox.crt \
    /root/qnetd-migration/cluster-hectormox.crt

We can convert it to PEM:

bash

cd /root/qnetd-migration

openssl x509 \
    -inform DER \
    -in cluster-hectormox.crt \
    -out cluster-hectormox.pem

Then verify it:

bash

openssl verify \
    -CAfile qnetd-cacert.crt \
    cluster-hectormox.pem

The expected result is:

OK

24. Import the cluster certificate

Add the signed certificate to NSS:

bash

certutil -A \
    -d "sql:$DB" \
    -n "Cluster Cert" \
    -t "u,u,u" \
    -i "$DIR/cluster-hectormox.crt" \
    -f "$PASS"

Check:

bash

certutil -L \
    -d "sql:$DB" \
    -f "$PASS"

The following entries should now appear:

QNet CA CT,c,c

Cluster Cert u,u,u

25. Export the certificate and private key to PKCS#12

Generate:

bash

pk12util \
    -o "$DIR/qdevice-net-node.p12" \
    -d "sql:$DB" \
    -k "$PASS" \
    -W "" \
    -n "Cluster Cert"

Expected result:

PKCS12 EXPORT SUCCESSFUL

This file contains a private key. Do not publish or share it.

26. Prepare the second Proxmox node

Copy to the second node:

bash

scp \
    /root/qnetd-migration/qnetd-cacert.crt \
    root@192.168.1.7:/root/qnetd-migration/

And:

bash

scp \
    /root/qnetd-migration/qdevice-net-node.p12 \
    root@192.168.1.7:/root/qnetd-migration/

On zimaproxmox:

bash

DIR="/root/qnetd-migration"
DB="$DIR/nssdb-new"
PASS="$DIR/pwdfile-new.txt"
CA="$DIR/qnetd-cacert.crt"
P12="$DIR/qdevice-net-node.p12"

mkdir -m 700 "$DB"

: > "$PASS"
chmod 600 "$PASS"

certutil -N \
    -d "sql:$DB" \
    -f "$PASS"

Import the CA:

bash

certutil -A \
    -d "sql:$DB" \
    -n "QNet CA" \
    -t "CT,c,c" \
    -i "$CA" \
    -f "$PASS"

Import the PKCS#12 file:

bash

pk12util \
    -i "$P12" \
    -d "sql:$DB" \
    -k "$PASS" \
    -W ""

The expected result is:

PKCS12 IMPORT SUCCESSFUL

Then check:

bash

certutil -L \
    -d "sql:$DB" \
    -f "$PASS"

27. Prepare a permanent NSS database

Before touching the database actually used by QDevice, I prepared another directory:

/root/qnetd-migration/nssdb-ready

On each node:

bash

DIR="/root/qnetd-migration"
READY="$DIR/nssdb-ready"

mkdir -m 700 "$READY"

cp -a \
    "$DIR/nssdb-new/." \
    "$READY/"

install -m 600 \
    "$DIR/pwdfile-new.txt" \
    "$READY/pwdfile.txt"

Check:

bash

certutil -L \
    -d "sql:$READY" \
    -f "$READY/pwdfile.txt"

We can also verify the identity:

bash

certutil -L \
    -d "sql:$READY" \
    -n "Cluster Cert" \
    -a |
openssl x509 \
    -noout \
    -subject \
    -issuer

In my case:

subject=CN=hectormox

issuer=CN=QNet CA

28. Replace the active credentials

We performed this step one node at a time.

Stop only QDevice:

systemctl stop corosync-qdevice

Do not stop Corosync.

Save the old database:

bash

ACTIVE="/etc/corosync/qdevice/net/nssdb"
BACKUP="${ACTIVE}.pre-glinet-$(date +%Y%m%d-%H%M%S)"
mv "$ACTIVE" "$BACKUP"

Copy the new one:

bash

cp -a \
    /root/qnetd-migration/nssdb-ready \
    "$ACTIVE"

Adjust the permissions:

bash

chown -R root:root "$ACTIVE"
chmod 700 "$ACTIVE"
chmod 600 "$ACTIVE"/*

Check:

bash

certutil -L \
    -d "sql:$ACTIVE" \
    -f "$ACTIVE/pwdfile.txt"

Then start QDevice:

systemctl start corosync-qdevice

Repeat the same procedure on the other node.

29. Change the IP address of the old QNetd

In my case, QDevice was already configured to use another server:

192.168.1.5

and I needed to switch it to the Flint:

192.168.1.1

I made another backup first:

bash

cp \
    /etc/pve/corosync.conf \
    "/root/corosync-pre-glinet-$(date +%Y%m%d-%H%M%S).conf"

The change was:

- host: 192.168.1.5

+ host: 192.168.1.1

We also increased:

- config_version: 3

+ config_version: 4

The relevant section looked like this:

bash

quorum {
  device {
    model: net
    net {
      algorithm: ffsplit
      host: 192.168.1.1
      tls: on
    }
    votes: 1
  }
}

Keep in mind that config_version must be incremented from the value used by each installation.

Do not copy 3 → 4 literally.

30. Restart QDevice only

After the change, Corosync adopted the new configuration, but the QDevice process kept using the old address until it was restarted.

Run:

systemctl restart corosync-qdevice

Check:

corosync-qdevice-tool -s

The expected result was:

Cluster name: hectormox

QNetd host: 192.168.1.1:5403

Algorithm: Fifty-Fifty split

State: Connected

Repeat the QDevice restart on the second node.

31. Check quorum

Run:

pvecm status

In our case, the final result was:

Expected votes: 3 Highest expected: 3 Total votes: 3 Quorum: 2 Quorate: Yes

And in the member information:

0x00000001 1 A,V,NMW 192.168.1.6

0x00000002 1 A,V,NMW 192.168.1.7

0x00000000 1 Qdevice

That is:

miniproxmox: 1 vote.
zimaproxmox: 1 vote.
QDevice/QNetd: 1 vote.

Total:

3 votes

and the required quorum is still:

2

32. Make a full backup of the Flint QNetd

Once everything was done, I made a full backup of the environment.

Set:

bash

SERVICE="/etc/init.d/qnetd-debian"
BACKUP="/overlay/qnetd-backup-completo-$(date +%Y%m%d-%H%M%S).tar.gz"

Stop QNetd briefly:

"$SERVICE" stop

Create the backup:

bash

tar -czf "$BACKUP" \
    -C / \
    --exclude=opt/debian-qnetd/proc \
    --exclude=opt/debian-qnetd/dev/pts \
    --exclude=opt/debian-qnetd/run \
    --exclude=opt/debian-qnetd/tmp \
    opt/debian-qnetd \
    etc/init.d/qnetd-debian \
    etc/rc.d/S98qnetd-debian \
    etc/rc.d/K10qnetd-debian

Start it again:

"$SERVICE" start

Check the integrity:

bash

tar -tzf "$BACKUP" >/dev/null
sha256sum "$BACKUP"
ls -lh "$BACKUP"

My final backup was approximately:

157 MB

33. Download the backup to Windows

From PowerShell:

bash

scp -O `
    root@192.168.1.1:/overlay/qnetd-backup-completo-XXXXXXXX.tar.gz `
    "$env:USERPROFILE\Downloads\"

Then calculate the hash:

bash

Get-FileHash `
    "$env:USERPROFILE\Downloads\qnetd-backup-completo-XXXXXXXX.tar.gz" `
    -Algorithm SHA256

The value must exactly match the one obtained earlier on the Flint.

The backup contains certificates and private keys, so it must be stored securely., so it must be stored securely.

34. Test the system after restarting the Flint

I did not want to consider the project finished just because it worked after starting it manually.

I restarted the Flint 3 from its web interface.

When it was available again, I connected once more:

ssh root@192.168.1.1

First, I checked that the script was still there:

wc -l /etc/init.d/qnetd-debian

Then:

pidof corosync-qnetd

I checked the port:

netstat -lntp 2>/dev/null | grep 5403

And finally, procd:

bash

ubus call service list \
    '{"name":"qnetd-debian"}'

In the final test, I got:

23 /etc/init.d/qnetd-debian

a valid PID:

28139

the port:

192.168.1.1:5403 LISTEN

and:

"running": true

This showed that QNetd had started automatically after the router rebooted.

35. Verify that Proxmox reconnects automatically

On miniproxmox:

corosync-qdevice-tool -s

The status automatically returned to:

QNetd host: 192.168.1.1:5403 State: Connected

And:

pvecm status

showed again:

Expected votes: 3

Total votes: 3

Quorum: 2

Quorate: Yes

Therefore, when we restart the Flint, the Proxmox nodes temporarily lose the QDevice vote, but once QNetd starts again, the QDevices reconnect automatically.

36. A curious issue we found

Something strange happened during one of the tests.

The service was shown as enabled:

/etc/init.d/qnetd-debian enabled

but QNetd was not running.

We checked:

pidof corosync-qnetd

but got no PID.

Nothing was listening either:

bash

netstat -lntp 2>/dev/null |
    grep 5403

So we checked the script:

wc -l /etc/init.d/qnetd-debian

And found:

0 /etc/init.d/qnetd-debian

The file had become empty.

However, these still existed:

/etc/rc.d/S98qnetd-debian

/etc/rc.d/K10qnetd-debian

That is why OpenWrt could report that the service was enabled even though there was actually nothing to run.

We rebuilt the script and repeated the full reboot test.

After that, it started correctly.

The takeaway is important:

checking that a service is enabled is not enough.

It is also a good idea to check:

bash

pidof corosync-qnetd
netstat -lntp 2>/dev/null | grep 5403
ubus call service list \ '{"name":"qnetd-debian"}'

37. Quick troubleshooting if QNetd will not start

First, check that the script exists and is not empty:

bash

wc -l /etc/init.d/qnetd-debian
cat /etc/init.d/qnetd-debian

Then:

/etc/init.d/qnetd-debian enabled

Find the process:

pidof corosync-qnetd

Port:

bash

netstat -lntp 2>/dev/null |
    grep 5403

Status in procd:

bash

ubus call service list \
    '{"name":"qnetd-debian"}'

To test QNetd outside procd:

bash

timeout 5 /usr/bin/env -i \
    PATH=/usr/sbin:/usr/bin:/sbin:/bin \
    /usr/sbin/chroot --userspec=100:101 \
    /opt/debian-qnetd \
    /usr/bin/corosync-qnetd \
    -f -4 \
    -l 192.168.1.1 \
    -p 5403 \
    -s req

echo "EXIT=$?"

If it returns:

124

it means the daemon was still running when timeout stopped it.

In that case, the problem is probably with the procd service, not QNetd.

38. Quick troubleshooting from Proxmox

First, check the services:

bash

systemctl is-active corosync
systemctl is-active corosync-qdevice

Then:

corosync-qdevice-tool -s

The correct result is:

State: Connected

And:

pvecm status

should show:

Total votes: 3

Quorate: Yes

If you see:

Cannot initialize CMAP service

that does not necessarily mean QNetd is failing.

It is a good idea to check this first:

systemctl is-active corosync

During my tests, we encountered exactly this kind of issue. It turned out to be unrelated to QNetd and was caused by NFS mounts on the Proxmox host.

39. Values to adapt to your installation

Do not copy all the values in this guide literally.

Flint address

In my case:

192.168.1.1

Proxmox nodes

192.168.1.6

192.168.1.7

Cluster name

hectormox

Check it with:

pvecm status

Architecture

My Flint is:

ARM64

but we should check with:

uname -m

UID and GID for `coroqnetd`

In my installation:

100:101

But check with:

bash

chroot /opt/debian-qnetd \
    id coroqnetd

Debian path

I chose:

/opt/debian-qnetd

It is not required.

Debian version

In this guide:

Debian 12 Bookworm

TLS

My version used:

-s req

Check it with:

bash

chroot /opt/debian-qnetd \
    /usr/bin/corosync-qnetd \
    -h

`config_version`

Do not copy:

3 → 4

Increment the current value in our corosync.conf.

40. Final result

After completing the whole process, the Flint 3 works like this:

Código

OpenWrt
└── Debian 12 ARM64 in chroot
    └── corosync-qnetd

QNetd:

192.168.1.1:5403

TLS

procd

respawn

automatic startup

And the cluster:

Código

miniproxmox  → QDevice ─┐
                         ├──→ Flint 3 / QNetd
zimaproxmox  → QDevice ─┘

The final result, which we can check with:

corosync-qdevice-tool -s

is:

QNetd host: 192.168.1.1:5403

State: Connected

And with:

pvecm status

we get:

Expected votes: 3

Total votes: 3

Quorum: 2

Quorate: Yes

The useful thing about this solution is that we have not turned the Flint 3 into a Debian server or replaced OpenWrt.

The Flint remains our OpenWrt router.

We simply make use of an always-on ARM64 device with enough storage and support for chroot, to create a minimal Debian system dedicated to running QNetd.

This gives us a physically independent third vote for a two-node Proxmox cluster without keeping another server running solely to act as the arbitrator.

Recommended gear

Device used in this project: GL.iNet GL-BE9300 Flint 3. If you need it, here is where to find it:

GL.iNet GL-BE9300 Flint 3 — Reviewed in the video: AliExpress España · AliExpress desde Europa · AliExpress Oficial GL.iNet · Amazon

1. The environment used

2. Connect to the Flint 3 over SSH

3. Check the system, architecture, packages, and available space

4. Why use a minimal Debian system inside OpenWrt

5. Install debootstrap on the Flint

6. GPG issue with Debian Bookworm

7. Create a Debian 12 ARM64 minbase system

8. Update Debian and install QNetd

9. /proc and /dev/pts during installation

10. Initialize the QNetd certificates

11. Check the user coroqnetd

12. Test QNetd manually

13. OpenWrt’s LD_PRELOAD issue

14. Create the QNetd service in OpenWrt

15. Check that QNetd is running

16. Check respawning

17. Prepare Proxmox

18. Back up before changing QDevice

19. Create a new NSS database on Proxmox

20. Copy the Flint CA to Proxmox

21. Create the cluster CSR

22. Send the CSR to the Flint

23. Retrieve the signed certificate

24. Import the cluster certificate

25. Export the certificate and private key to PKCS#12

26. Prepare the second Proxmox node

27. Prepare a permanent NSS database

28. Replace the active credentials

29. Change the IP address of the old QNetd

30. Restart QDevice only

31. Check quorum

32. Make a full backup of the Flint QNetd

33. Download the backup to Windows

34. Test the system after restarting the Flint

35. Verify that Proxmox reconnects automatically

36. A curious issue we found

37. Quick troubleshooting if QNetd will not start

38. Quick troubleshooting from Proxmox

39. Values to adapt to your installation

Flint address

Proxmox nodes

Cluster name

Architecture

UID and GID for coroqnetd

Debian path

Debian version

TLS

config_version

40. Final result

Recommended gear

Related video

Recommended gear

9. `/proc` and `/dev/pts` during installation

11. Check the user `coroqnetd`

UID and GID for `coroqnetd`

`config_version`