drbd起動時に下記の様な「/lib/drbd/drbd: line 148: /var/lib/linstor/loop_device_mapping: No such file or directory」というログが出力されている。
root@ubuntu02:~# systemctl status drbd
● drbd.service - DRBD -- please disable. Unless you are NOT using a cluster manager.
Loaded: loaded (/lib/systemd/system/drbd.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2021-12-20 14:07:35 JST; 2s ago
Process: 241197 ExecStart=/lib/drbd/drbd start (code=exited, status=0/SUCCESS)
Main PID: 241197 (code=exited, status=0/SUCCESS)
Dec 20 14:07:34 ubuntu02 drbd[241197]: /lib/drbd/drbd: line 148: /var/lib/linstor/loop_device_mapping: No such file or directory
Dec 20 14:07:34 ubuntu02 drbd[241216]: [
Dec 20 14:07:34 ubuntu02 drbd[241216]: create res: alfons
Dec 20 14:07:34 ubuntu02 drbd[241216]: prepare disk: alfons
Dec 20 14:07:34 ubuntu02 drbd[241216]: adjust disk: alfons
Dec 20 14:07:34 ubuntu02 drbd[241216]: adjust net: alfons
Dec 20 14:07:34 ubuntu02 drbd[241216]: ]
Dec 20 14:07:35 ubuntu02 drbd[241258]: WARN: stdin/stdout is not a TTY; using /dev/console
Dec 20 14:07:35 ubuntu02 drbd[241197]: ...done.
Dec 20 14:07:35 ubuntu02 systemd[1]: Finished DRBD -- please disable. Unless you are NOT using a cluster manager..
root@ubuntu02:~#
まず、 /var/lib/linstor ディレクトリは標準状態では存在していない。
root@ubuntu02:~# ls -l /var/lib/linstor
ls: cannot access '/var/lib/linstor': No such file or directory
root@ubuntu02:~#
# Set up/detach linstor loopback devices for LINSTOR file based pools
# FIXME I think setting up these loop device mappings should be done elsewhere,
# and not in the DRBD init file; possibly we need proper support in drbdadm.
handle_linstor_loopback()
{
local line dev file
# || [[ -n $line ]]: in case there is no newline at EOF
while read -r line || [[ -n $line ]] ; do
dev=${line%%:*}
file=${line#*:}
test -f "$file" && test -r "$file" && test -w "$file" || continue
# should -f be allowed?
[[ $dev = loop* ]] || [[ $dev = /dev/loop* ]] || continue
case "$1" in
# what about existing, but "wrong", mappings?
start) losetup "$dev" 2>/dev/null || losetup "$dev" "$file";;
stop) losetup "$dev" 2>/dev/null && losetup -d "$dev";;
esac
done < /var/lib/linstor/loop_device_mapping
# at least we tried.
return 0
}
「drbdadm primary shares」でPrimaryとして動作させたあと、他方のサーバに切り替えるため降格する「drbdadm secondary shares」を実行したところ、「0: State change failed: (-12) Device is held open by someone」というメッセージで失敗した。
root@ubuntu01:~# drbdadm secondary shares
0: State change failed: (-12) Device is held open by someone
Command 'drbdsetup-84 secondary 0' terminated with exit code 11
root@ubuntu01:~#
$ sudo -i
# apt install drbd-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
guile-2.2-libs libgc1c2 libgsasl7 libidn11 libkyotocabinet16v5 libmailutils6
libmysqlclient21 libntlm0 mailutils mailutils-common mysql-common postfix
ssl-cert
Suggested packages:
heartbeat mailutils-mh mailutils-doc procmail postfix-mysql postfix-pgsql
postfix-ldap postfix-pcre postfix-lmdb postfix-sqlite sasl2-bin
| dovecot-common resolvconf postfix-cdb postfix-doc openssl-blacklist
The following NEW packages will be installed:
drbd-utils guile-2.2-libs libgc1c2 libgsasl7 libidn11 libkyotocabinet16v5
libmailutils6 libmysqlclient21 libntlm0 mailutils mailutils-common
mysql-common postfix ssl-cert
0 upgraded, 14 newly installed, 0 to remove and 0 not upgraded.
Need to get 9,579 kB of archives.
After this operation, 66.5 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
＜略＞
root@ubuntu143:~# apt search drbd
Sorting... Done
Full Text Search... Done
collectd-core/focal 5.9.2.g-1ubuntu5 amd64
statistics collection and monitoring daemon (core system)
drbd-doc/focal 8.4~20151102-1 all
RAID 1 over TCP/IP for Linux (user documentation)
drbd-utils/focal,now 9.11.0-1build1 amd64 [installed]
RAID 1 over TCP/IP for Linux (user utilities)
nagios-plugins-contrib/focal 25.20191015+1ubuntu1 amd64
Plugins for nagios compatible monitoring systems
prometheus-hacluster-exporter/focal 0.4.0-2 amd64
Prometheus exporter for HA cluster services
root@ubuntu143:~#
root@ubuntu143:~# add-apt-repository ppa:linbit/linbit-drbd9-stack
This PPA contains DRBD9, drbd-utils, LINSTOR (client, python API, server).
This differs from official, production grade LINBIT repositories in several ways, including:
- We push RCs immediately to the PPA
- We don't push hotfixes, these usually have to wait until the next RC/release
- We only keep 2 LTS versions up to date (Bionic and Focal, but not Xenial)
For support and access to official repositories see:
https://www.linbit.com or write an email to: sales AT linbit.com
More info: https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack
Press [ENTER] to continue or Ctrl-c to cancel adding it.
Hit:1 http://jp.archive.ubuntu.com/ubuntu focal InRelease
Get:2 http://jp.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://jp.archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:4 http://jp.archive.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:5 http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu focal InRelease [24.4 kB]
Get:6 http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu focal/main amd64 Packages [2,344 B]
Get:7 http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu focal/main Translation-en [1,308 B]
Fetched 356 kB in 3s (137 kB/s)
Reading package lists... Done
root@ubuntu143:~#
root@ubuntu143:~# apt update
Hit:1 http://jp.archive.ubuntu.com/ubuntu focal InRelease
Get:2 http://jp.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://jp.archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:4 http://jp.archive.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:5 http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu focal InRelease
Fetched 328 kB in 1s (538 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
root@ubuntu143:~# apt list --upgradable
Listing... Done
drbd-utils/focal 9.19.1-1ppa1~focal1 amd64 [upgradable from: 9.11.0-1build1]
N: There is 1 additional version. Please use the '-a' switch to see it
root@ubuntu143:~# apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
drbd-utils
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 720 kB of archives.
After this operation, 75.8 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu focal/main amd64 drbd-utils amd64 9.19.1-1ppa1~focal1 [720 kB]
Fetched 720 kB in 3s (214 kB/s)
Preconfiguring packages ...
(Reading database ... 72754 files and directories currently installed.)
Preparing to unpack .../drbd-utils_9.19.1-1ppa1~focal1_amd64.deb ...
Unpacking drbd-utils (9.19.1-1ppa1~focal1) over (9.11.0-1build1) ...
Setting up drbd-utils (9.19.1-1ppa1~focal1) ...
Configuration file '/etc/drbd.d/global_common.conf'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
*** global_common.conf (Y/I/N/O/D/Z) [default=N] ? y
Installing new version of config file /etc/drbd.d/global_common.conf ...
Installing new version of config file /etc/init.d/drbd ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.13) ...
root@ubuntu143:~#
次に、「apt install drbd-dkms」でdrbd本体をインストールした。
root@ubuntu143:~# apt install drbd-dkms
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
autoconf automake autopoint autotools-dev binutils binutils-common
binutils-x86-64-linux-gnu build-essential cpp cpp-9 dctrl-tools debhelper
dh-autoreconf dh-strip-nondeterminism dkms dpkg-dev dwz fakeroot g++ g++-9
gcc gcc-9 gcc-9-base gettext intltool-debian libalgorithm-diff-perl
libalgorithm-diff-xs-perl libalgorithm-merge-perl libarchive-cpio-perl
libarchive-zip-perl libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev
libcc1-0 libcroco3 libcrypt-dev libctf-nobfd0 libctf0 libdebhelper-perl
libdpkg-perl libfakeroot libfile-fcntllock-perl
libfile-stripnondeterminism-perl libgcc-9-dev libgomp1 libisl22 libitm1
liblsan0 libltdl-dev libmail-sendmail-perl libmpc3 libquadmath0
libstdc++-9-dev libsub-override-perl libsys-hostname-long-perl libtool
libtsan0 libubsan1 linux-libc-dev m4 make manpages-dev po-debconf
Suggested packages:
autoconf-archive gnu-standards autoconf-doc binutils-doc cpp-doc
gcc-9-locales debtags dh-make menu debian-keyring g++-multilib
g++-9-multilib gcc-9-doc gcc-multilib flex bison gdb gcc-doc gcc-9-multilib
gettext-doc libasprintf-dev libgettextpo-dev glibc-doc bzr libtool-doc
libstdc++-9-doc gfortran | fortran95-compiler gcj-jdk m4-doc make-doc
libmail-box-perl
The following NEW packages will be installed:
autoconf automake autopoint autotools-dev binutils binutils-common
binutils-x86-64-linux-gnu build-essential cpp cpp-9 dctrl-tools debhelper
dh-autoreconf dh-strip-nondeterminism dkms dpkg-dev drbd-dkms dwz fakeroot
g++ g++-9 gcc gcc-9 gcc-9-base gettext intltool-debian
libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl
libarchive-cpio-perl libarchive-zip-perl libasan5 libatomic1 libbinutils
libc-dev-bin libc6-dev libcc1-0 libcroco3 libcrypt-dev libctf-nobfd0 libctf0
libdebhelper-perl libdpkg-perl libfakeroot libfile-fcntllock-perl
libfile-stripnondeterminism-perl libgcc-9-dev libgomp1 libisl22 libitm1
liblsan0 libltdl-dev libmail-sendmail-perl libmpc3 libquadmath0
libstdc++-9-dev libsub-override-perl libsys-hostname-long-perl libtool
libtsan0 libubsan1 linux-libc-dev m4 make manpages-dev po-debconf
0 upgraded, 66 newly installed, 0 to remove and 0 not upgraded.
Need to get 47.8 MB of archives.
After this operation, 208 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
＜略＞
drbd.ko:
Running module version sanity check.
- Original module
- No original module exists within this kernel
- Installation
- Installing to /lib/modules/5.4.0-90-generic/updates/dkms/
drbd_transport_tcp.ko:
Running module version sanity check.
- Original module
- No original module exists within this kernel
- Installation
- Installing to /lib/modules/5.4.0-90-generic/updates/dkms/
depmod..........
DKMS: install completed.
Processing triggers for install-info (6.7.0.dfsg.2-5) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for man-db (2.9.1-1) ...
root@ubuntu143:~#
root@ubuntu143:~# apt search drbd
Sorting... Done
Full Text Search... Done
collectd-core/focal 5.9.2.g-1ubuntu5 amd64
statistics collection and monitoring daemon (core system)
drbd-dkms/focal,now 9.1.4-1ppa1~focal1 all [installed]
RAID 1 over TCP/IP for Linux module source
drbd-doc/focal 8.4~20151102-1 all
RAID 1 over TCP/IP for Linux (user documentation)
drbd-module-source/focal 9.1.4-1ppa1~focal1 all
RAID 1 over TCP/IP for Linux module source
drbd-reactor/focal 0.5.0-1ppa1~focal1 amd64
Monitors DRBD resources via plugins.
drbd-utils/focal,now 9.19.1-1ppa1~focal1 amd64 [installed]
RAID 1 over TCP/IP for Linux (user utilities)
drbd8-utils/focal 2:9.19.1-1ppa1~focal1 amd64
transitional dummy package
linstor-common/focal 1.16.0-1ppa1~focal1 all
DRBD distributed resource management utility
linstor-controller/focal 1.16.0-1ppa1~focal1 all
DRBD distributed resource management utility
linstor-satellite/focal 1.16.0-1ppa1~focal1 all
DRBD distributed resource management utility
nagios-plugins-contrib/focal 25.20191015+1ubuntu1 amd64
Plugins for nagios compatible monitoring systems
prometheus-hacluster-exporter/focal 0.4.0-2 amd64
Prometheus exporter for HA cluster services
root@ubuntu143:~#
2022/04/22追記
Ubuntu 22.04 LTSが出たので状況を確認してみた。
user$ apt search drbd
Sorting... Done
Full Text Search... Done
drbd-doc/jammy 8.4~20220106-1 all
RAID 1 over TCP/IP for Linux (user documentation)
drbd-utils/jammy 9.15.0-1build2 amd64
RAID 1 over TCP/IP for Linux (user utilities)
monitoring-plugins-contrib/jammy 37.20211217ubuntu1 amd64
Plugins for nagios compatible monitoring systems
prometheus-hacluster-exporter/jammy 1.2.3-2 amd64
Prometheus exporter for HA cluster services
user@ubuntu2204:~$ modinfo drbd
filename: /lib/modules/5.15.0-25-generic/kernel/drivers/block/drbd/drbd.ko
alias: block-major-147-*
license: GPL
version: 8.4.11
description: drbd - Distributed Replicated Block Device v8.4.11
author: Philipp Reisner <phil@linbit.com>, Lars Ellenberg <lars@linbit.com>
srcversion: C369F58AE19642816B00C81
depends: lru_cache,libcrc32c
retpoline: Y
intree: Y
name: drbd
vermagic: 5.15.0-25-generic SMP mod_unload modversions
sig_id: PKCS#7
signer: Build time autogenerated kernel key
sig_key: 43:DE:0B:33:F7:CF:5A:CE:F6:19:E2:2A:DD:DD:1D:BD:4D:ED:8E:ED
sig_hashalgo: sha512
signature: 72:7A:0C:A3:E5:31:68:E8:FC:55:61:6F:FF:E1:11:B1:90:9F:8D:DF:
29:E0:16:9D:C5:6D:0A:D4:A6:48:88:87:E0:5B:46:73:96:24:8A:1D:
51:1E:39:4B:A4:20:65:42:2F:A7:2B:6A:ED:AE:8B:4E:FD:0F:E5:94:
7C:03:0D:53:1F:21:C4:09:AF:7A:91:32:23:0E:E2:3A:32:79:FF:0B:
36:E7:BC:7A:5E:72:A0:6E:05:2C:04:B6:4B:01:77:65:5A:43:EA:AB:
F0:BE:B0:97:2E:65:81:7B:90:67:76:E5:64:2D:EF:8A:9A:3F:21:DD:
CF:A6:C8:3B:87:4D:C1:1C:6C:F8:AD:F2:01:C2:DC:BD:CA:3F:85:4C:
A8:A6:B3:C5:83:C8:13:76:31:77:51:8C:37:3D:D4:77:00:29:8F:BE:
1A:E3:C4:1F:EE:96:1E:24:24:9A:77:BE:F4:7C:40:4D:BD:34:83:69:
8D:0C:E6:DA:FC:F0:65:74:52:86:EA:8F:26:2B:71:5B:BB:79:73:0C:
26:D1:66:4D:FC:71:B2:E4:B3:0B:4D:85:EC:CF:98:CB:E6:83:C6:79:
28:F1:BF:23:CC:30:68:0E:F4:50:85:04:D1:AF:87:31:E0:4E:A5:1D:
5B:F7:6F:18:34:17:E9:8D:84:6C:B5:2C:9F:6F:38:DA:4D:9D:4B:3F:
38:6D:38:5E:02:56:23:DB:B3:F4:06:6E:EE:52:F1:09:62:59:1A:BA:
57:1E:E7:10:0F:61:E1:86:5C:3E:ED:67:D6:AF:9F:CE:69:F3:38:D4:
EA:91:30:72:7F:B3:02:08:82:0F:C4:4B:8C:85:86:6E:4A:9F:E3:51:
78:A7:4D:C2:E1:E2:7D:20:F5:42:F6:A7:6F:76:76:04:B9:93:E7:99:
53:9A:3F:6D:61:AA:31:B2:04:59:BE:E4:16:E3:8C:4E:D0:58:0A:FE:
FF:1C:79:D6:5E:72:72:3D:A0:41:47:DC:04:8C:04:04:C6:E7:6A:55:
5A:E7:FC:B1:C5:B5:CD:C3:D3:03:3A:B8:3E:C1:24:87:93:DD:34:1A:
4B:97:3B:61:7C:CB:E9:34:90:C2:C5:C5:2A:79:74:D1:49:E6:50:A1:
9F:35:AA:14:38:1B:0F:01:27:4E:44:F5:0F:7C:28:41:77:0A:AB:A9:
0A:43:0C:CB:69:27:B8:E4:CA:FA:E9:01:12:A3:57:D2:7B:9F:9F:8E:
4D:A1:F9:5B:12:AD:F8:87:FA:49:F2:E3:72:D4:2A:A4:6F:EF:C7:13:
2D:96:BF:E9:7C:3F:63:5B:60:C7:33:7B:5C:FA:9A:3A:66:83:41:63:
B6:67:18:DC:D4:5F:D6:4B:5F:DA:F2:74
parm: allow_oos:DONT USE! (bool)
parm: disable_sendpage:bool
parm: proc_details:int
parm: minor_count:Approximate number of drbd devices (1-255) (uint)
parm: usermode_helper:string
user@ubuntu2204:$
