投稿者: osakanataro

投稿日:

CUDA対応のtensorflowインストールがわかりにくい件

AlmaLinux 9環境でNVIDIA GPUでCUDAが動く環境を作ったときに、とりあえず動作テストとしてtensorflowでも動かすか、と試してみたら、ドキュメントに騙された件についてメモ

まず、最初に参照したドキュメントは TensorFlow 2 をインストールする

CPU と GPUのどちらも「pip install tensorflow」でインストールできる、と書いてある

では、インストール

[testuser@vgpu ~]$ pip list
Package         Version
--------------- --------
dbus-python     1.2.18
distlib         0.3.2
distro          1.5.0
filelock        3.7.1
gpg             1.15.1
libcomps        0.1.18
nftables        0.1
packaging       20.9
pip             21.3.1
platformdirs    2.5.4
pycairo         1.20.1
PyGObject       3.40.1
pyparsing       2.4.7
python-dateutil 2.8.1
PyYAML          5.4.1
rpm             4.16.1.3
selinux         3.6
sepolicy        3.6
setools         4.4.4
setuptools      53.0.0
six             1.15.0
systemd-python  234
virtualenv      20.21.1
[testuser@vgpu ~]$ pip install tensorflow
Defaulting to user installation because normal site-packages is not writeable
Collecting tensorflow
  Downloading tensorflow-2.19.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (644.8 MB)
     |████████████████████████████████| 644.8 MB 18 kB/s
Collecting h5py>=3.11.0
  Downloading h5py-3.13.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)
     |████████████████████████████████| 4.6 MB 92.7 MB/s
Collecting absl-py>=1.0.0
  Downloading absl_py-2.2.2-py3-none-any.whl (135 kB)
     |████████████████████████████████| 135 kB 85.3 MB/s
Collecting typing-extensions>=3.6.6
  Downloading typing_extensions-4.13.2-py3-none-any.whl (45 kB)
     |████████████████████████████████| 45 kB 9.4 MB/s
Collecting grpcio<2.0,>=1.24.3
  Downloading grpcio-1.71.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.9 MB)
     |████████████████████████████████| 5.9 MB 87.9 MB/s
Collecting tensorflow-io-gcs-filesystem>=0.23.1
  Downloading tensorflow_io_gcs_filesystem-0.37.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.1 MB)
     |████████████████████████████████| 5.1 MB 89.6 MB/s
Collecting google-pasta>=0.1.1
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 27.5 MB/s
Requirement already satisfied: packaging in /usr/lib/python3.9/site-packages (from tensorflow) (20.9)
Collecting wrapt>=1.11.0
  Downloading wrapt-1.17.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
     |████████████████████████████████| 82 kB 5.8 MB/s
Collecting astunparse>=1.6.0
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting flatbuffers>=24.3.25
  Downloading flatbuffers-25.2.10-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: six>=1.12.0 in /usr/lib/python3.9/site-packages (from tensorflow) (1.15.0)
Collecting requests<3,>=2.21.0
  Downloading requests-2.32.3-py3-none-any.whl (64 kB)
     |████████████████████████████████| 64 kB 21.4 MB/s
Collecting termcolor>=1.1.0
  Downloading termcolor-3.1.0-py3-none-any.whl (7.7 kB)
Collecting ml-dtypes<1.0.0,>=0.5.1
  Downloading ml_dtypes-0.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.7 MB)
     |████████████████████████████████| 4.7 MB 98 kB/s
Collecting keras>=3.5.0
  Downloading keras-3.9.2-py3-none-any.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 62.7 MB/s
Requirement already satisfied: setuptools in /usr/lib/python3.9/site-packages (from tensorflow) (53.0.0)
Collecting numpy<2.2.0,>=1.26.0
  Downloading numpy-2.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.5 MB)
     |████████████████████████████████| 19.5 MB 53 kB/s
Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0dev,>=3.20.3
  Downloading protobuf-5.29.4-cp38-abi3-manylinux2014_x86_64.whl (319 kB)
     |████████████████████████████████| 319 kB 80.5 MB/s
Collecting opt-einsum>=2.3.2
  Downloading opt_einsum-3.4.0-py3-none-any.whl (71 kB)
     |████████████████████████████████| 71 kB 2.0 MB/s
Collecting tensorboard~=2.19.0
  Downloading tensorboard-2.19.0-py3-none-any.whl (5.5 MB)
     |████████████████████████████████| 5.5 MB 88.3 MB/s
Collecting gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1
  Downloading gast-0.6.0-py3-none-any.whl (21 kB)
Collecting libclang>=13.0.0
  Downloading libclang-18.1.1-py2.py3-none-manylinux2010_x86_64.whl (24.5 MB)
     |████████████████████████████████| 24.5 MB 55 kB/s
Collecting wheel<1.0,>=0.23.0
  Downloading wheel-0.45.1-py3-none-any.whl (72 kB)
     |████████████████████████████████| 72 kB 5.3 MB/s
Collecting optree
  Downloading optree-0.15.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (397 kB)
     |████████████████████████████████| 397 kB 74.3 MB/s
Collecting namex
  Downloading namex-0.0.9-py3-none-any.whl (5.8 kB)
Collecting rich
  Downloading rich-14.0.0-py3-none-any.whl (243 kB)
     |████████████████████████████████| 243 kB 78.5 MB/s
Collecting charset-normalizer<4,>=2
  Downloading charset_normalizer-3.4.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)
     |████████████████████████████████| 149 kB 79.2 MB/s
Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.4.0-py3-none-any.whl (128 kB)
     |████████████████████████████████| 128 kB 74.9 MB/s
Collecting idna<4,>=2.5
  Downloading idna-3.10-py3-none-any.whl (70 kB)
     |████████████████████████████████| 70 kB 33.1 MB/s
Collecting certifi>=2017.4.17
  Downloading certifi-2025.4.26-py3-none-any.whl (159 kB)
     |████████████████████████████████| 159 kB 78.6 MB/s
Collecting werkzeug>=1.0.1
  Downloading werkzeug-3.1.3-py3-none-any.whl (224 kB)
     |████████████████████████████████| 224 kB 67.2 MB/s
Collecting markdown>=2.6.8
  Downloading markdown-3.8-py3-none-any.whl (106 kB)
     |████████████████████████████████| 106 kB 55.2 MB/s
Collecting tensorboard-data-server<0.8.0,>=0.7.0
  Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl (6.6 MB)
     |████████████████████████████████| 6.6 MB 61.3 MB/s
Requirement already satisfied: pyparsing>=2.0.2 in /usr/lib/python3.9/site-packages (from packaging->tensorflow) (2.4.7)
Collecting importlib-metadata>=4.4
  Downloading importlib_metadata-8.7.0-py3-none-any.whl (27 kB)
Collecting MarkupSafe>=2.1.1
  Downloading MarkupSafe-3.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (20 kB)
Collecting markdown-it-py>=2.2.0
  Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
     |████████████████████████████████| 87 kB 29.4 MB/s
Collecting pygments<3.0.0,>=2.13.0
  Downloading pygments-2.19.1-py3-none-any.whl (1.2 MB)
     |████████████████████████████████| 1.2 MB 77.3 MB/s
Collecting zipp>=3.20
  Downloading zipp-3.21.0-py3-none-any.whl (9.6 kB)
Collecting mdurl~=0.1
  Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Installing collected packages: zipp, mdurl, typing-extensions, pygments, numpy, MarkupSafe, markdown-it-py, importlib-metadata, wheel, werkzeug, urllib3, tensorboard-data-server, rich, protobuf, optree, namex, ml-dtypes, markdown, idna, h5py, grpcio, charset-normalizer, certifi, absl-py, wrapt, termcolor, tensorflow-io-gcs-filesystem, tensorboard, requests, opt-einsum, libclang, keras, google-pasta, gast, flatbuffers, astunparse, tensorflow
Successfully installed MarkupSafe-3.0.2 absl-py-2.2.2 astunparse-1.6.3 certifi-2025.4.26 charset-normalizer-3.4.2 flatbuffers-25.2.10 gast-0.6.0 google-pasta-0.2.0 grpcio-1.71.0 h5py-3.13.0 idna-3.10 importlib-metadata-8.7.0 keras-3.9.2 libclang-18.1.1 markdown-3.8 markdown-it-py-3.0.0 mdurl-0.1.2 ml-dtypes-0.5.1 namex-0.0.9 numpy-2.0.2 opt-einsum-3.4.0 optree-0.15.0 protobuf-5.29.4 pygments-2.19.1 requests-2.32.3 rich-14.0.0 tensorboard-2.19.0 tensorboard-data-server-0.7.2 tensorflow-2.19.0 tensorflow-io-gcs-filesystem-0.37.1 termcolor-3.1.0 typing-extensions-4.13.2 urllib3-2.4.0 werkzeug-3.1.3 wheel-0.45.1 wrapt-1.17.2 zipp-3.21.0
[testuser@vgpu ~]$

そして、テストとして「python3 -c “import tensorflow as tf; print(tf.config.list_physical_devices(‘GPU’))”」を実行

[testuser@vgpu ~]$ python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
2025-05-15 11:35:31.898614: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1747276531.922384    2049 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1747276531.929813    2049 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
W0000 00:00:1747276531.948919    2049 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1747276531.948946    2049 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1747276531.948951    2049 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1747276531.948954    2049 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
2025-05-15 11:35:31.954990: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
W0000 00:00:1747276535.245548    2049 gpu_device.cc:2341] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[]
[testuser@vgpu ~]$

GPU対応については https://www.tensorflow.org/install/gpu を見ろ、とある・・・

こちらでは「pip install ‘tensorflow[and-cuda]’」でインストールしろ、とある


[testuser@vgpu ~]$ pip list
Package                      Version
---------------------------- ---------
absl-py                      2.2.2
astunparse                   1.6.3
certifi                      2025.4.26
charset-normalizer           3.4.2
dbus-python                  1.2.18
distlib                      0.3.2
distro                       1.5.0
filelock                     3.7.1
flatbuffers                  25.2.10
gast                         0.6.0
google-pasta                 0.2.0
gpg                          1.15.1
grpcio                       1.71.0
h5py                         3.13.0
idna                         3.10
importlib_metadata           8.7.0
keras                        3.9.2
libclang                     18.1.1
libcomps                     0.1.18
Markdown                     3.8
markdown-it-py               3.0.0
MarkupSafe                   3.0.2
mdurl                        0.1.2
ml_dtypes                    0.5.1
namex                        0.0.9
nftables                     0.1
numpy                        2.0.2
opt_einsum                   3.4.0
optree                       0.15.0
packaging                    20.9
pip                          21.3.1
platformdirs                 2.5.4
protobuf                     5.29.4
pycairo                      1.20.1
Pygments                     2.19.1
PyGObject                    3.40.1
pyparsing                    2.4.7
python-dateutil              2.8.1
PyYAML                       5.4.1
requests                     2.32.3
rich                         14.0.0
rpm                          4.16.1.3
selinux                      3.6
sepolicy                     3.6
setools                      4.4.4
setuptools                   53.0.0
six                          1.15.0
systemd-python               234
tensorboard                  2.19.0
tensorboard-data-server      0.7.2
tensorflow                   2.19.0
tensorflow-io-gcs-filesystem 0.37.1
termcolor                    3.1.0
typing_extensions            4.13.2
urllib3                      2.4.0
virtualenv                   20.21.1
Werkzeug                     3.1.3
wheel                        0.45.1
wrapt                        1.17.2
zipp                         3.21.0
[testuser@vgpu ~]$ pip install 'tensorflow[and-cuda]'
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: tensorflow[and-cuda] in ./.local/lib/python3.9/site-packages (2.19.0)
Requirement already satisfied: flatbuffers>=24.3.25 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (25.2.10)
Requirement already satisfied: google-pasta>=0.1.1 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (0.2.0)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (0.6.0)
Requirement already satisfied: termcolor>=1.1.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (3.1.0)
Requirement already satisfied: wrapt>=1.11.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (1.17.2)
Requirement already satisfied: opt-einsum>=2.3.2 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (3.4.0)
Requirement already satisfied: setuptools in /usr/lib/python3.9/site-packages (from tensorflow[and-cuda]) (53.0.0)
Requirement already satisfied: typing-extensions>=3.6.6 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (4.13.2)
Requirement already satisfied: six>=1.12.0 in /usr/lib/python3.9/site-packages (from tensorflow[and-cuda]) (1.15.0)
Requirement already satisfied: numpy<2.2.0,>=1.26.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (2.0.2)
Requirement already satisfied: absl-py>=1.0.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (2.2.2)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0dev,>=3.20.3 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (5.29.4)
Requirement already satisfied: requests<3,>=2.21.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (2.32.3)
Requirement already satisfied: keras>=3.5.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (3.9.2)
Requirement already satisfied: h5py>=3.11.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (3.13.0)
Requirement already satisfied: astunparse>=1.6.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (1.6.3)
Requirement already satisfied: ml-dtypes<1.0.0,>=0.5.1 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (0.5.1)
Requirement already satisfied: tensorboard~=2.19.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (2.19.0)
Requirement already satisfied: libclang>=13.0.0 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (18.1.1)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (1.71.0)
Requirement already satisfied: packaging in /usr/lib/python3.9/site-packages (from tensorflow[and-cuda]) (20.9)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in ./.local/lib/python3.9/site-packages (from tensorflow[and-cuda]) (0.37.1)
Collecting nvidia-cufft-cu12==11.2.3.61
  Downloading nvidia_cufft_cu12-11.2.3.61-py3-none-manylinux2014_x86_64.whl (192.5 MB)
     |████████████████████████████████| 192.5 MB 52 kB/s
Collecting nvidia-cublas-cu12==12.5.3.2
  Downloading nvidia_cublas_cu12-12.5.3.2-py3-none-manylinux2014_x86_64.whl (363.3 MB)
     |████████████████████████████████| 363.3 MB 50 kB/s
Collecting nvidia-cudnn-cu12==9.3.0.75
  Downloading nvidia_cudnn_cu12-9.3.0.75-py3-none-manylinux2014_x86_64.whl (577.2 MB)
     |████████████████████████████████| 577.2 MB 65 kB/s
Collecting nvidia-nvjitlink-cu12==12.5.82
  Downloading nvidia_nvjitlink_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl (21.3 MB)
     |████████████████████████████████| 21.3 MB 67 kB/s
Collecting nvidia-cuda-runtime-cu12==12.5.82
  Downloading nvidia_cuda_runtime_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl (895 kB)
     |████████████████████████████████| 895 kB 38.5 MB/s
Collecting nvidia-nccl-cu12==2.23.4
  Downloading nvidia_nccl_cu12-2.23.4-py3-none-manylinux2014_x86_64.whl (199.0 MB)
     |████████████████████████████████| 199.0 MB 53 kB/s
Collecting nvidia-cusparse-cu12==12.5.1.3
  Downloading nvidia_cusparse_cu12-12.5.1.3-py3-none-manylinux2014_x86_64.whl (217.6 MB)
     |████████████████████████████████| 217.6 MB 51 kB/s
Collecting nvidia-cuda-nvcc-cu12==12.5.82
  Downloading nvidia_cuda_nvcc_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl (22.5 MB)
     |████████████████████████████████| 22.5 MB 56 kB/s
Collecting nvidia-cuda-cupti-cu12==12.5.82
  Downloading nvidia_cuda_cupti_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl (13.8 MB)
     |████████████████████████████████| 13.8 MB 47 kB/s
Collecting nvidia-cuda-nvrtc-cu12==12.5.82
  Downloading nvidia_cuda_nvrtc_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl (24.9 MB)
     |████████████████████████████████| 24.9 MB 52 kB/s
Collecting nvidia-curand-cu12==10.3.6.82
  Downloading nvidia_curand_cu12-10.3.6.82-py3-none-manylinux2014_x86_64.whl (56.3 MB)
     |████████████████████████████████| 56.3 MB 49 kB/s
Collecting nvidia-cusolver-cu12==11.6.3.83
  Downloading nvidia_cusolver_cu12-11.6.3.83-py3-none-manylinux2014_x86_64.whl (130.3 MB)
     |████████████████████████████████| 130.3 MB 51 kB/s
Requirement already satisfied: wheel<1.0,>=0.23.0 in ./.local/lib/python3.9/site-packages (from astunparse>=1.6.0->tensorflow[and-cuda]) (0.45.1)
Requirement already satisfied: rich in ./.local/lib/python3.9/site-packages (from keras>=3.5.0->tensorflow[and-cuda]) (14.0.0)
Requirement already satisfied: namex in ./.local/lib/python3.9/site-packages (from keras>=3.5.0->tensorflow[and-cuda]) (0.0.9)
Requirement already satisfied: optree in ./.local/lib/python3.9/site-packages (from keras>=3.5.0->tensorflow[and-cuda]) (0.15.0)
Requirement already satisfied: certifi>=2017.4.17 in ./.local/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (2025.4.26)
Requirement already satisfied: idna<4,>=2.5 in ./.local/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./.local/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (2.4.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.local/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (3.4.2)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in ./.local/lib/python3.9/site-packages (from tensorboard~=2.19.0->tensorflow[and-cuda]) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in ./.local/lib/python3.9/site-packages (from tensorboard~=2.19.0->tensorflow[and-cuda]) (3.1.3)
Requirement already satisfied: markdown>=2.6.8 in ./.local/lib/python3.9/site-packages (from tensorboard~=2.19.0->tensorflow[and-cuda]) (3.8)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/lib/python3.9/site-packages (from packaging->tensorflow[and-cuda]) (2.4.7)
Requirement already satisfied: importlib-metadata>=4.4 in ./.local/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard~=2.19.0->tensorflow[and-cuda]) (8.7.0)
Requirement already satisfied: MarkupSafe>=2.1.1 in ./.local/lib/python3.9/site-packages (from werkzeug>=1.0.1->tensorboard~=2.19.0->tensorflow[and-cuda]) (3.0.2)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in ./.local/lib/python3.9/site-packages (from rich->keras>=3.5.0->tensorflow[and-cuda]) (2.19.1)
Requirement already satisfied: markdown-it-py>=2.2.0 in ./.local/lib/python3.9/site-packages (from rich->keras>=3.5.0->tensorflow[and-cuda]) (3.0.0)
Requirement already satisfied: zipp>=3.20 in ./.local/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard~=2.19.0->tensorflow[and-cuda]) (3.21.0)
Requirement already satisfied: mdurl~=0.1 in ./.local/lib/python3.9/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.5.0->tensorflow[and-cuda]) (0.1.2)
Installing collected packages: nvidia-nvjitlink-cu12, nvidia-cusparse-cu12, nvidia-cublas-cu12, nvidia-nccl-cu12, nvidia-cusolver-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cudnn-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-nvcc-cu12, nvidia-cuda-cupti-cu12
Successfully installed nvidia-cublas-cu12-12.5.3.2 nvidia-cuda-cupti-cu12-12.5.82 nvidia-cuda-nvcc-cu12-12.5.82 nvidia-cuda-nvrtc-cu12-12.5.82 nvidia-cuda-runtime-cu12-12.5.82 nvidia-cudnn-cu12-9.3.0.75 nvidia-cufft-cu12-11.2.3.61 nvidia-curand-cu12-10.3.6.82 nvidia-cusolver-cu12-11.6.3.83 nvidia-cusparse-cu12-12.5.1.3 nvidia-nccl-cu12-2.23.4 nvidia-nvjitlink-cu12-12.5.82
[testuser@vgpu ~]$
[testuser@vgpu ~]$ pip list
Package                      Version
---------------------------- ---------
absl-py                      2.2.2
astunparse                   1.6.3
certifi                      2025.4.26
charset-normalizer           3.4.2
dbus-python                  1.2.18
distlib                      0.3.2
distro                       1.5.0
filelock                     3.7.1
flatbuffers                  25.2.10
gast                         0.6.0
google-pasta                 0.2.0
gpg                          1.15.1
grpcio                       1.71.0
h5py                         3.13.0
idna                         3.10
importlib_metadata           8.7.0
keras                        3.9.2
libclang                     18.1.1
libcomps                     0.1.18
Markdown                     3.8
markdown-it-py               3.0.0
MarkupSafe                   3.0.2
mdurl                        0.1.2
ml_dtypes                    0.5.1
namex                        0.0.9
nftables                     0.1
numpy                        2.0.2
nvidia-cublas-cu12           12.5.3.2
nvidia-cuda-cupti-cu12       12.5.82
nvidia-cuda-nvcc-cu12        12.5.82
nvidia-cuda-nvrtc-cu12       12.5.82
nvidia-cuda-runtime-cu12     12.5.82
nvidia-cudnn-cu12            9.3.0.75
nvidia-cufft-cu12            11.2.3.61
nvidia-curand-cu12           10.3.6.82
nvidia-cusolver-cu12         11.6.3.83
nvidia-cusparse-cu12         12.5.1.3
nvidia-nccl-cu12             2.23.4
nvidia-nvjitlink-cu12        12.5.82
opt_einsum                   3.4.0
optree                       0.15.0
packaging                    20.9
pip                          21.3.1
platformdirs                 2.5.4
protobuf                     5.29.4
pycairo                      1.20.1
Pygments                     2.19.1
PyGObject                    3.40.1
pyparsing                    2.4.7
python-dateutil              2.8.1
PyYAML                       5.4.1
requests                     2.32.3
rich                         14.0.0
rpm                          4.16.1.3
selinux                      3.6
sepolicy                     3.6
setools                      4.4.4
setuptools                   53.0.0
six                          1.15.0
systemd-python               234
tensorboard                  2.19.0
tensorboard-data-server      0.7.2
tensorflow                   2.19.0
tensorflow-io-gcs-filesystem 0.37.1
termcolor                    3.1.0
typing_extensions            4.13.2
urllib3                      2.4.0
virtualenv                   20.21.1
Werkzeug                     3.1.3
wheel                        0.45.1
wrapt                        1.17.2
zipp                         3.21.0
[testuser@vgpu ~]$

だいぶ差分があった

そして実行


[testuser@vgpu ~]$ python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
2025-05-15 11:45:19.350181: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1747277119.373769    2108 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1747277119.381364    2108 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
W0000 00:00:1747277119.400331    2108 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1747277119.400358    2108 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1747277119.400362    2108 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1747277119.400365    2108 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
2025-05-15 11:45:19.406358: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
[testuser@vgpu ~]$

というわけで、tensorflowをnvidia GPU環境でインストールするときは 「pip install ‘tensorflow[and-cuda]’」でやる必要がある、という話でした

投稿日:

DSPラジオ ATS-Miniのfirmwareをアップデートしてみた

aliexpressで安く売ってたATS-Miniの派生品買ってみた

ループアンテナセットで3222円でした。

電源入れた時のfirmware versionは1.0.1と出ていた

バージョンアップできるのか探してみると https://github.com/esp32-si4732/ats-mini を発見

(あとからわかったのだが、最初にインストールされてたのは 同じgithubでも大文字の https://github.com/G8PTN/ATS_MINI/ で、こちらはソースコード非公開だった)

ドキュメントにあるように uv をインストールしてみた

osakanataro@ubuntuserver:~$ curl -LsSf https://astral.sh/uv/install.sh | sh
downloading uv 0.7.2 x86_64-unknown-linux-gnu
no checksums to verify
installing to /home/osakanataro/.local/bin
  uv
  uvx
everything's installed!
osakanataro@ubuntuserver:~$

ATS-MiniをLinuxサーバに接続してみるとデバイスが認識される

osakanataro@ubuntuserver:~$ lsusb
＜略＞
Bus 002 Device 006: ID 303a:1001 Espressif USB JTAG/serial debug unit
＜略＞
osakanataro@ubuntuserver:~$ lsusb --tree
＜略＞
/:  Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/11p, 480M
＜略＞
    |__ Port 007: Dev 006, If 0, Class=Communications, Driver=cdc_acm, 12M
    |__ Port 007: Dev 006, If 1, Class=CDC Data, Driver=cdc_acm, 12M
    |__ Port 007: Dev 006, If 2, Class=Vendor Specific Class, Driver=[none], 12M
＜略＞
osakanataro@ubuntuserver:~$

シリアルポート名が何になっているのかをdmesgの結果から確認

osakanataro@ubuntuserver:~$ sudo dmesg|tail
[sudo] osakanataro のパスワード:
[1695575.346617] usbcore: registered new interface driver cdc_acm
[1695575.346621] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
[1695776.000954] usb 2-7: USB disconnect, device number 5
[1695849.379292] usb 2-7: new full-speed USB device number 6 using xhci_hcd
[1695849.506664] usb 2-7: New USB device found, idVendor=303a, idProduct=1001, bcdDevice= 1.01
[1695849.506681] usb 2-7: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[1695849.506688] usb 2-7: Product: USB JTAG/serial debug unit
[1695849.506693] usb 2-7: Manufacturer: Espressif
[1695849.506697] usb 2-7: SerialNumber: FC:01:2C:CC:BD:08
[1695849.510971] cdc_acm 2-7:1.0: ttyACM0: USB ACM device
osakanataro@ubuntuserver:~$

osakanataro@ubuntuserver:~$ uvx --from esptool esptool.py --chip esp32s3 --port ttyACM0 --baud 921600 read_flash 0x0 ALL original-flash.bin
      Built esptool==4.8.1
Installed 13 packages in 30ms
esptool.py v4.8.1
Serial port SERIAL_PORT

A fatal error occurred: Could not open ttyACM0, the port is busy or doesn't exist.
([Errno 2] could not open port ttyACM0: [Errno 2] No such file or directory: 'ttyACM0')

Hint: Check if the port is correct and ESP connected

osakanataro@ubuntuserver:~$

あ・・・パス指定

osakanataro@ubuntuserver:~$ uvx --from esptool esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 921600 read_flash 0x0 ALL original-flash.bin
esptool.py v4.8.1
Serial port /dev/ttyACM0

A fatal error occurred: Could not open /dev/ttyACM0, the port is busy or doesn't exist.
([Errno 13] could not open port /dev/ttyACM0: [Errno 13] Permission denied: '/dev/ttyACM0')

Hint: Try to add user into dialout or uucp group.

osakanataro@ubuntuserver:~$

uv,uvxはユーザディレクトリにインストールされているのでsudoで使えない・・・

osakanataro@ubuntuserver:~$ sudo uvx --from esptool esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 921600 read_flash 0x0 ALL original-flash.bin
sudo: uvx: コマンドが見つかりません
osakanataro@ubuntuserver:~$

uucpグループとdialoutグループのどちらにも所属させたけどpermission denied

osakanataro@ubuntuserver:~$ sudo usermod -G uucp,dialout osakanataro
[sudo] osakanataro のパスワード:
osakanataro@ubuntuserver:~$ uvx --from esptool esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 921600 read_flash 0x0 ALL original-flash.bin
esptool.py v4.8.1
Serial port /dev/ttyACM0

A fatal error occurred: Could not open /dev/ttyACM0, the port is busy or doesn't exist.
([Errno 13] could not open port /dev/ttyACM0: [Errno 13] Permission denied: '/dev/ttyACM0')

Hint: Try to add user into dialout or uucp group.

osakanataro@ubuntuserver:~$ ls -l /dev/ttyACM0
crw-rw---- 1 root dialout 166, 0  5月  6 15:54 /dev/ttyACM0
osakanataro@ubuntuserver:~$

んー？？？

ぐぐると brltty がインストールされている場合に発生することがあるらしいのでアンインストールして再実行

osakanataro@ubuntuserver:~$ dpkg -l|grep tty
ii  brltty                                         6.6-4ubuntu5                             amd64        Access software for a blind person using a braille display
ii  libjetty9-java                                 9.4.53-1                                 all          Java servlet engine and webserver -- core libraries
osakanataro@ubuntuserver:~$
osakanataro@ubuntuserver:~$ dpkg -l|grep brltty
ii  brltty                                         6.6-4ubuntu5                             amd64        Access software for a blind person using a braille display
osakanataro@ubuntuserver:~$ sudo apt remove brltty
パッケージリストを読み込んでいます... 完了
依存関係ツリーを作成しています... 完了
状態情報を読み取っています... 完了
以下のパッケージが自動でインストールされましたが、もう必要とされていません:
  libpcre2-32-0
これを削除するには 'sudo apt autoremove' を利用してください。
以下のパッケージは「削除」されます:
  brltty
アップグレード: 0 個、新規インストール: 0 個、削除: 1 個、保留: 7 個。
この操作後に 10.5 MB のディスク容量が解放されます。
続行しますか? [Y/n] y
(データベースを読み込んでいます ... 現在 278045 個のファイルとディレクトリがインストールされています。)
brltty (6.6-4ubuntu5) を削除しています ...
man-db (2.12.0-4build2) のトリガを処理しています ...
エラー: タイムアウトしました
osakanataro@ubuntuserver:~$ uvx --from esptool esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 921600 read_flash 0x0 ALL original-flash.bin
esptool.py v4.8.1
Serial port /dev/ttyACM0

A fatal error occurred: Could not open /dev/ttyACM0, the port is busy or doesn't exist.
([Errno 13] could not open port /dev/ttyACM0: [Errno 13] Permission denied: '/dev/ttyACM0')

Hint: Try to add user into dialout or uucp group.

osakanataro@ubuntuserver:~$

違うらしい

再起動したあと再実行すると、あっけなく実行が出来た

osakanataro@ubuntuserver:~$ uvx --from esptool esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 921600 read_flash 0x0 ALL original-flash.bin
esptool.py v4.8.1
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: fc:01:2c:cc:bd:08
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Detected flash size: 16MB
16777216 (100 %)
16777216 (100 %)
Read 16777216 bytes at 0x00000000 in 273.8 seconds (490.2 kbit/s)...
Hard resetting via RTS pin...
osakanataro@ubuntuserver:~$ ls -l original-flash.bin
-rw-rw-r-- 1 osakanataro osakanataro 16777216  5月  6 17:26 original-flash.bin
osakanataro@ubuntuserver:~$

では、続いて、firmwareを書き込み

osakanataro@ubuntuserver:~/ats-mini$ ls ats-mini-v2.14
CHANGELOG.md  ats-mini.ino.bin  ats-mini.ino.bootloader.bin  ats-mini.ino.merged.bin  ats-mini.ino.partitions.bin
osakanataro@ubuntuserver:~/ats-mini$ ls -l ats-mini-v2.14
合計 8712
-rw-r--r-- 1 osakanataro osakanataro   11491  5月  6 12:46 CHANGELOG.md
-rw-r--r-- 1 osakanataro osakanataro  494048  5月  6 12:46 ats-mini.ino.bin
-rw-r--r-- 1 osakanataro osakanataro   20208  5月  6 12:46 ats-mini.ino.bootloader.bin
-rw-r--r-- 1 osakanataro osakanataro 8388608  5月  6 12:46 ats-mini.ino.merged.bin
-rw-r--r-- 1 osakanataro osakanataro    3072  5月  6 12:46 ats-mini.ino.partitions.bin
osakanataro@ubuntuserver:~/ats-mini$ uvx --from esptool esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 921600 --before default_reset --after hard_reset write_flash  -z --flash_mode keep --flash_freq keep --flash_size keep 0x0 ats-mini-v2.14/ats-mini.ino.merged.bin
esptool.py v4.8.1
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: fc:01:2c:cc:bd:08
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x007fffff...
Compressed 8388608 bytes to 299786...
Wrote 8388608 bytes (299786 compressed) at 0x00000000 in 20.2 seconds (effective 3315.7 kbit/s)...

A fatal error occurred: Packet content transfer stopped (received 0 bytes)
osakanataro@ubuntuserver:~/ats-mini$

えっ・・・・失敗？

もう１つある分割ファイルの方で再実行

osakanataro@ubuntuserver:~/ats-mini$ uvx --from esptool esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 921600 --before default_reset --after hard_reset write_flash  -z --flash_mode keep --flash_freq keep --flash_size keep 0x0 ats-mini-v2.14/ats-mini.ino.bootloader.bin 0x8000 ats-mini-v2.14/ats-mini.ino.partitions.bin 0x10000 ats-mini-v2.14/ats-mini.ino.bin
esptool.py v4.8.1
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: fc:01:2c:cc:bd:08
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00004fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00010000 to 0x00088fff...
Compressed 20208 bytes to 13058...
Wrote 20208 bytes (13058 compressed) at 0x00000000 in 0.2 seconds (effective 918.1 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 146...
Wrote 3072 bytes (146 compressed) at 0x00008000 in 0.0 seconds (effective 1196.3 kbit/s)...
Hash of data verified.
Compressed 494048 bytes to 277050...
Wrote 494048 bytes (277050 compressed) at 0x00010000 in 2.5 seconds (effective 1560.3 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
osakanataro@ubuntuserver:~/ats-mini$

無事に書き込み完了

FW2.14で起動したことを確認

投稿日:

dovecot / postfix と Active Directory連携時の動作調査手法

dovecot / postfix と Active Directoryを連携させようと設定してみたところ、最初はうまく動かなかった。

この動かない原因をどうやって調べていくか、というのを解説してるものがなく、非常に難儀したのでメモ書きとして残す

dovecot, postfixの現在の設定を確認

RHEL9の場合、 /etc/dovecot および /etc/postfix に設定ファイル群があるが、コメントやサブディレクトリにあるファイルとの結合により、最終的な設定が何になっているのかわかりにくい

「doveconf」および「postconf」を実行することで最終的な設定を確認することができる

また、デフォルト値と異なる部分は何かを「doveconf -n」「postconf -n」を実行することで確認できる。

なお、doveconfの場合、ssh_keyなど一部のパラメータについては「doveconf -P」と-Pオプションをつけないと実際の値が表示されない

dovecotのログ出力を増やす

ログ出力を増やすための設定がいろいろあった(Dovecot Logging)ため、 /etc/dovecot/conf.d/99-debug.conf と１つのファイルにまとめて必要ない場合は /etc/dovecot/conf.d/99-debug._conf と.conf という名前じゃなくすることで無効化できるようにした

[root@mail dovecot]# cat /etc/dovecot/conf.d/99-debug.conf
auth_debug=yes
auth_debug_passwords=yes
auth_verbose=yes
auth_verbose_passwords=yes
verbose_proctitle=yes
verbose_ssl=yes

[root@mail dovecot]#

設定後は「systemctl restart dovecot」で設定を有効にする

なお「doveadm log find」を実行するとdovecotのログがどのファイルに出力されているかを確認することができる

[root@mail dovecot]# doveadm log find
Looking for log files from /var/log
Debug: /var/log/maillog
Info: /var/log/maillog
Warning: /var/log/maillog
Error: /var/log/maillog
Fatal: /var/log/maillog
[root@mail dovecot]#

LDAP検索のログを増やす

dovecotの場合は dovecot本体への auth_verbose=yes 設定だけでLDAPでどういったqueryを投げているかも確認できる

ただ、もっと詳細を確認したい、という場合 /etc/dovecot/dovecot-ldap.conf.ext などのLDAP接続情報を書いたファイルに「debug_level」を追加することでログを増やすこともできる。

dovecot Common LDAP Settings for both auth and sieve
iredmail Turn on debug mode in Dovecot

dovecot標準値は「debug_level=0」。ログを増やす場合は「debug_level=1」、最大量に増やす場合は「debug_level=-1」とする

postfixの場合も同様に LDAP接続情報を書いたファイル /etc/postfix/ldap-mailbox.cf などに「debuglevel」を追加することでログを増やすことができる。

postfix ldap_table – Postfix LDAP client configuration

postfix標準値は「debuglevel=0」。増やす場合は「debuglevel=1」から最大量は「debuglevel=10」とする

postfixのログ出力を増やす

Postfix Debugging Howto に記載があるが正直めんどい

/etc/postfix/master.cf の smtpd起動に関して「-v」オプションか「-D」オプションをつける、という形となる。

とはいえ、postfix/dovecotを組み合わせた場合、ActiveDirectory/LDAP側の処理をdovecot側で行うということもあるので、まずはdovecot側の動作がちゃんとするのを先に確認したほうがよい。

dovecotでの認証確認

dovecotで認証動作を確認する場合、まずは「doveadm auth login ユーザ名」を実行する

[root@mail ~]# doveadm auth login testuser1@adsample.local
Password: ＜パスワード入力＞
passdb: testuser1@adsample.local auth succeeded
extra fields:
  user=testuser1@adsample.local
  uid=1000
  gid=1000
userdb extra fields:
  testuser1@adsample.local
  uid=1000
  gid=1000
  auth_mech=PLAIN
[root@mail ~]#

なお、「doveadm auth login testuser1@adsample.local パスワード」とパスワードをつけて実行すると入力しないで済むので検証時は便利（ログに残るので一時なパスワードにすること）

ただし、このdoveadm auth loginコマンドは模擬的に確認しているだけで、実際にログイン処理は行っていないようで、テストしたユーザのディレクトリがない場合でもディレクトリが作成されない。（ちゃんとログインした場合は自動的に作成される）

ldapsearchコマンドを使った検証

doveadm auth loginコマンドでうまく認証が実行できない場合、 ldapsearchコマンドを使って原因を調査していったりする。

その場合、 /etc/dovecot/conf.d/99-debug.conf の設定を有効にしてから行う。

期待通りに動作していない場合、ログにある下記のような「ldap」「base=～」「filter=～」という記述に注目する

May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): Server accepted connection (fd=14)
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): Sending version handshake
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: Handling LIST request
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: ldap(): Performing userdb lookup
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: ldap: iterate: base=cn=Users,dc=adsample,dc=local scope=subtree filter=(objectClass=posixAccount) fields=uid
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: ldap(): Finished userdb lookup
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: Finished

上記の場合注目するのは以下の部分

base=cn=Users,dc=adsample,dc=local scope=subtree filter=(objectClass=posixAccount) fields=uid

これをldapsearchコマンドに与える

-bオプションの後ろにbase=の後ろにある「cn=Users,dc=adsample,dc=local」
-sオプションの後ろにscope=の「subtree」
そしてfilterの「objectClass=posixAccount」

ldapsearch -x -H ldaps://192.168.122.10 -D "cn=vmail,cn=Users,dc=adsample,dc=local" -w "パスワード" -b "dc=adsample,dc=local" -s subtree objectClass=posixAccount

この出力結果を確認し、fields=で指定した「uid」という項目があるかを確認します

なければfilterで指定した条件が不適切、ということとなる

fieldsで適切かなぁ、というものが出るまでfilterの条件式の調整と、fieldsで選ぶ項目の調整を行う

doveadmでメールボックスの状態確認

dovecotで認識しているメールボックスの状態を確認する「doveadm mailbox status」コマンド

全ユーザについてとりあえず全ステータスをとる場合はfieldをallにすればいい、というので「doveadm mailbox status -A all ‘*’」を実行してみる

[root@mail ~]# doveadm mailbox status -A all '*'
testuser1@adsample.local Trash messages=0 recent=0 uidnext=2 uidvalidity=1745978754 unseen=0 highestmodseq=5 vsize=0 guid=40b0b82833051c68570600003a0de1d0 firstsaved=never
testuser1@adsample.local INBOX messages=16 recent=0 uidnext=18 uidvalidity=1745978753 unseen=0 highestmodseq=22 vsize=613873 guid=3fa5f40281851168ee0500003a0de1d0 firstsaved=1746666817
testuser2@adsample.local Drafts messages=4 recent=0 uidnext=5 uidvalidity=1745978410 unseen=0 highestmodseq=9 vsize=2313 guid=fe4beb08b4d91268980500003a0de1d0 firstsaved=1746066862
testuser2@adsample.local Sent messages=28 recent=0 uidnext=35 uidvalidity=1745978409 unseen=0 highestmodseq=37 vsize=10545 guid=a28a5e0b7c201268cb0600003a0de1d0 firstsaved=1746018428
testuser2@adsample.local Trash messages=0 recent=0 uidnext=12 uidvalidity=1745978408 unseen=0 highestmodseq=11 vsize=0 guid=ab2c081a361e1268260600003a0de1d0 firstsaved=never
testuser2@adsample.local INBOX messages=9 recent=0 uidnext=15 uidvalidity=1745978407 unseen=0 highestmodseq=33 vsize=10810 guid=7e51333727841168d30500003a0de1d0 firstsaved=1746084330
testuser3@adsample.local INBOX messages=2 recent=2 uidnext=3 uidvalidity=1746166331 unseen=2 highestmodseq=3 vsize=1330 guid=7d83dc163b621468950800003a0de1d0 firstsaved=1746166331
[root@mail ~]#

ん？？ vsizeは表示されていない？…マニュアルを再度確認すると -tオプションで messages, recent, unseen, vsizeだけを表示できるとのこと

[root@mail ~]# doveadm mailbox status -A all '*' -t
testuser1@adsample.local messages=16 recent=0 unseen=0 vsize=613873
testuser2@adsample.local messages=41 recent=0 unseen=0 vsize=23668
testuser3@adsample.local messages=2 recent=2 unseen=2 vsize=1330
testuser4@adsample.local messages=0 recent=0 unseen=0 vsize=0
vmail@adsample.local messages=0 recent=0 unseen=0 vsize=0
[root@mail ~]#

messages: メール総数
recent: Recentフラグがついてるメール数(新着メール=まだメールソフトに取り込んでない)
unseen: 未読メール数
vsize: メールの総容量(バイト)

投稿日:

acvite directory連携のdovecotでdoveadm quota get -Aが動かない

dovecot 2.2.19以降で登場した各ユーザのメールフォルダ内にあるindexファイルを使ったquotaを設定しようとした際に発見した出来事です。

doveadm quota get -Aの動作

doveadm quotaのマニュアルを見ると「doveadm quota get -A」を実行すると全ユーザの結果が表示されそうな気がするので実行してみたがされない

[root@mail dovecot]# doveadm quota get -A
Username Quota name Type Value Limit                                                   %
[root@mail dovecot]#

dovecotにdebug系ログ出力を有効にした状態での /var/log/maillog には下記のログ

May  2 11:18:59 mail dovecot[959]: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth
May  2 11:18:59 mail dovecot[959]: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so
May  2 11:18:59 mail dovecot[959]: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so
May  2 11:18:59 mail dovecot[959]: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth
May  2 11:18:59 mail dovecot[959]: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so
May  2 11:18:59 mail dovecot[959]: auth: Debug: Read auth token secret from /run/dovecot/auth-token-secret.dat
May  2 11:18:59 mail dovecot[959]: auth: Debug: ldap(/etc/dovecot/dovecot-ldap.conf.ext): LDAP initialization took 22 msecs
May  2 11:18:59 mail dovecot[959]: auth: Debug: master in: LIST#0111
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: Loading modules from directory: /usr/lib64/dovecot/auth
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: Loading modules from directory: /usr/lib64/dovecot/auth
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: ldap(/etc/dovecot/dovecot-ldap.conf.ext): LDAP initialization took 14 msecs
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): Server accepted connection (fd=14)
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): Sending version handshake
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: Handling LIST request
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: ldap(): Performing userdb lookup
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: ldap: iterate: base=cn=Users,dc=adsample,dc=local scope=subtree filter=(objectClass=posixAccount) fields=uid
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: ldap(): Finished userdb lookup
May  2 11:18:59 mail dovecot[959]: auth-worker(1542): Debug: conn unix:auth-worker (pid=1541,uid=97): auth-worker<1>: Finished

「objectClass=posixAccount」でフィルターをかけているが、Active DirectoryベースのLDAPサーバ標準では posixAccountは存在していないため、フィルター文字列を変える必要がある、という話である

確認のためldapsearchコマンドで出力がないことを確認

[root@mail dovecot]# ldapsearch -x -H ldaps://192.168.122.10 -D "cn=vmail,cn=Users,dc=adsample,dc=local" -w "パスワード" -b "dc=adsample,dc=local" -s subtree objectClass=posixAccount
# extended LDIF
#
# LDAPv3
# base <dc=adsample,dc=local> with scope subtree
# filter: objectClass=posixAccount
# requesting: ALL
#

# search reference
ref: ldaps://ForestDnsZones.adsample.local/DC=ForestDnsZones,DC=adsample,DC=lo
 cal

# search reference
ref: ldaps://DomainDnsZones.adsample.local/DC=DomainDnsZones,DC=adsample,DC=lo
 cal

# search reference
ref: ldaps://adsample.local/CN=Configuration,DC=adsample,DC=local

# search result
search: 2
result: 0 Success

# numResponses: 4
# numReferences: 3
[root@mail dovecot]#

どこの設定を変えればいいのか調べていくと userdb_ldap_iterate_fieldsとuserdb_ldap_iterate_filter で行っているので /etc/dovecot/dovecot-ldap.conf.ext に iterate_filter と iterate_attrs の設定を行う、ということがわかる

うまいことユーザ一覧っぽいのを取得するにはどうすればいいかな、とldapsearchコマンドをこねくり回して「ldapsearch -x -H ldaps://192.168.122.10 -D “cn=vmail,cn=Users,dc=adsample,dc=local” -w “パスワード” -b “dc=adsample,dc=local” -s subtree objectClass=user userPrincipalName」とすればいいかな、というのがわかった。

この結果をもとに、/etc/dovecot/dovecot-ldap.conf.ext に以下を追加してみたところおおむね期待通りの動作となった

iterate_filter=objectClass=user
iterate_attrs=userPrincipalName=user

これは、”objectClass=user”に該当するオブジェクトを表示させたあと、 userPrincipalName の値を dovecot上の user として認識させる、という意味合いの設定となる。

doveadm quota get -Aの実行結果

[root@mail dovecot]# doveadm quota get -A
Username                 Quota name Type    Value Limit                                          %
testuser1@adsample.local User quota STORAGE     9 10240                                          0
testuser1@adsample.local User quota MESSAGE    13     -                                          0
testuser2@adsample.local User quota STORAGE    14 10240                                          0
testuser2@adsample.local User quota MESSAGE    31     -                                          0
testuser3@adsample.local User quota STORAGE     0 10240                                          0
testuser3@adsample.local User quota MESSAGE     0     -                                          0
testuser4@adsample.local User quota STORAGE     0 10240                                          0
testuser4@adsample.local User quota MESSAGE     0     -                                          0
vmail@adsample.local     User quota STORAGE     0 10240                                          0
vmail@adsample.local     User quota MESSAGE     0     -                                          0
[root@mail dovecot]#

/etc/dovecot/conf.d/90-quota.conf を編集し、容量制限を1MBに変更

＜略＞
plugin {
  # 10MB quota limit
  quota = count:User quota
  quota_rule = *:storage=1M

  # This is required - it uses "virtual sizes" rather than "physical sizes"
  # for quota counting:
  quota_vsizes = yes
}

この状態でメールを送って容量を増やして確認・・・

[root@mail dovecot]# doveadm quota get -A
Username                 Quota name Type    Value Limit                                          %
testuser1@adsample.local User quota STORAGE   895  1024                                         87
testuser1@adsample.local User quota MESSAGE    16     -                                          0
testuser2@adsample.local User quota STORAGE   907  1024                                         88
testuser2@adsample.local User quota MESSAGE    38     -                                          0
testuser3@adsample.local User quota STORAGE     0  1024                                          0
testuser3@adsample.local User quota MESSAGE     0     -                                          0
testuser4@adsample.local User quota STORAGE     0  1024                                          0
testuser4@adsample.local User quota MESSAGE     0     -                                          0
vmail@adsample.local     User quota STORAGE     0  1024                                          0
vmail@adsample.local     User quota MESSAGE     0     -                                          0
[root@mail dovecot]#
投稿日:

postfixとactive directoryを連携させるときのLDA向け設定を検証した

postfix/dovecotを使ったメールサーバを作成する際、そのサーバ上でユーザ向けメールサービスを提供する場合にLDA設定を行う必要がある。

調べると dovecot-lda を使って保存するのがよさそうではある

参考資料
　dovecot側 「Dovecot LDA」「Dovecot LDA Examples」「Dovecot LDA with Postfix」

ただ、これをActive Directory/LDAPと連携させる場合の記述についてがいまいちはっきりしない。

ベースとするのは「Dovecot LDA with Postfix」の「Virtual users」にある記述で、/etc/postfix/master.cf に dovecotの記述を追加して、 /etc/postfix/main.cf に dovecot_destination_recipient_limit , virtual_mailbox_domains , virtual_transport の設定を追加する、と読める。

ただ、これだけだとメールを格納する場所について書いてないなぁ、と思いつつ試してみた

その結果、 virtual_transport=dovecot を設定した場合は、 postfix側で “virtual_mailbox_maps= ldap:/etc/postfix/ldap-mailbox.cf”とか”virtual_mailbox_base= /var/vmail”とかの設定を入れて /etc/postfix/ldap-mailbox.cf でLDAPに関する設定を書いたりする必要なく、dovecot側で行ったLDAP連携設定をもとにdovecot側で処理してくれる、ということが分かった

で・・・これの確認をするための副産物として、 virtual_transport=dovecotとしているのに virtual_mailbox_maps= ldap:/etc/postfix/ldap-mailbox.cf とかpostfix側でLDAPを直接見に行くような設定をしてしまうとどうなるの？というのを確認していた（意図せずに・・・

gihyoにあるそろそろLDAPにしてみないか？第１５回「FDS＋Postfixでメールサーバ管理」と「デージーネットのOSS postLDAPadmin Appendix」を参照しつついろいろ検討

まず、今回、/etc/postfix/master.cf の最下行にdovecotに関する２行を追加した

[root@mail postfix]# tail /etc/postfix/master.cf
#
#scalemail-backend unix -       n       n       -       2       pipe
#  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
#  ${nexthop} ${user} ${extension}
#
#mailman   unix  -       n       n       -       -       pipe
#  flags=FRX user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
#  ${nexthop} ${user}
dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/libexec/dovecot/dovecot-lda -f ${sender} -d ${recipient}
[root@mail postfix]#

そして、/etc/postfix/main.cf には以下を追加した

[root@mail postfix]# tail -10 /etc/postfix/main.cf
#
#
virtual_mailbox_domains = adsample.local
virtual_mailbox_maps= ldap:/etc/postfix/ldap-mailbox.cf
virtual_mailbox_base= /var/vmail
virtual_uid_maps= static:1000
virtual_gid_maps= static:1000
virtual_transport= dovecot
dovecot_destination_recipient_limit = 1

[root@mail postfix]#

で、postfix側のLDAP連携設定は以下とした

[root@mail postfix]# cat /etc/postfix/ldap-mailbox.cf
#server_host=192.168.122.10
server_host=ldaps://192.168.122.10
#server_port=636
#search_base=dc=adsample,dc=local
search_base=cn=Users,dc=adsample,dc=local
scope=sub
#query_filter = (&(objectClass=user)(mail=%s))
query_filter = (&(objectClass=user)(userPrincipalName=%s))
result_attribute = samAccountName
result_filter = /var/mail/%s/Maildir/

bind=yes
bind_dn=vmail@adsample.local
bind_pw=パスワード
version=3
#start_tls=yes
#debuglevel=10
[root@mail postfix]#

とりあえず「debuglevel=10」というのは調査中に有効にしていた値

virtual_mailbox_maps で取得できる情報として期待されているものは メールを保存するディレクトリ名 の模様

コメントとなっている「query_filter = (&(objectClass=user)(mail=%s))」はActive Directoryベースだとmailってないので使わない。ユーザ名のみの場合は「samAccountName=%s」、ドメイン名付きの場合は「userPrincipalName=%s」かな、というところで設定

取得できてるかどうかは「postmap -q ユーザ名 ldap:/～」で確認

[root@mail ~]# postmap -q testuser1@adsample.local ldap:/etc/postfix/ldap-mailbox.cf
/var/mail/testuser1/Maildir/
[root@mail ~]#

想定しているディレクトリ名が出力されればOK

-vオプションをつけると検索内容の詳細が確認できる。

[root@mail ~]# postmap -v -q testuser1@adsample.local ldap:/etc/postfix/ldap-mailbox.cf
postmap: name_mask: all
postmap: inet_addr_local: configured 2 IPv4 addresses
postmap: inet_addr_local: configured 2 IPv6 addresses
postmap: dict_ldap_open: Using LDAP source /etc/postfix/_ldap-mailbox.cf
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: server_host = ldaps://192.168.122.10
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: server_port = 389
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: version = 3
postmap: dict_ldap_open: /etc/postfix/_ldap-mailbox.cf server_host URL is ldaps://192.168.122.10
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: scope = sub
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: search_base = cn=Users,dc=adsample,dc=local
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: timeout = 10
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: query_filter = (&(objectClass=user)(userPrincipalName=%s))
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: result_format = <NULL>
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: result_filter = /var/mail/%s/Maildir/
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: domain =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: terminal_result_attribute =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: leaf_result_attribute =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: result_attribute = samAccountName
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: special_result_attribute =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: bind = yes
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: bind_dn = vmail@adsample.local
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: bind_pw = パスワード
postmap: cfg_get_bool: /etc/postfix/_ldap-mailbox.cf: cache = off
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: cache_expiry = -1
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: cache_size = -1
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: recursion_limit = 1000
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: expansion_limit = 0
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: size_limit = 0
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: dereference = 0
postmap: cfg_get_bool: /etc/postfix/_ldap-mailbox.cf: chase_referrals = off
postmap: cfg_get_bool: /etc/postfix/_ldap-mailbox.cf: start_tls = off
postmap: cfg_get_bool: /etc/postfix/_ldap-mailbox.cf: tls_require_cert = off
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: tls_ca_cert_file =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: tls_ca_cert_dir =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: tls_cert =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: tls_key =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: tls_random_file =
postmap: cfg_get_str: /etc/postfix/_ldap-mailbox.cf: tls_cipher_suite =
postmap: cfg_get_int: /etc/postfix/_ldap-mailbox.cf: debuglevel = 0
postmap: dict_open: ldap:/etc/postfix/_ldap-mailbox.cf
postmap: dict_ldap_lookup: In dict_ldap_lookup
postmap: dict_ldap_lookup: No existing connection for LDAP source /etc/postfix/_ldap-mailbox.cf, reopening
postmap: dict_ldap_connect: Connecting to server ldaps://192.168.122.10
postmap: dict_ldap_connect: Actual Protocol version used is 3.
postmap: dict_ldap_connect: Binding to server ldaps://192.168.122.10 with dn vmail@adsample.local
postmap: dict_ldap_connect: Successful bind to server ldaps://192.168.122.10 with dn vmail@adsample.local
postmap: dict_ldap_connect: Cached connection handle for LDAP source /etc/postfix/_ldap-mailbox.cf
postmap: dict_ldap_lookup: /etc/postfix/_ldap-mailbox.cf: Searching with filter (&(objectClass=user)(userPrincipalName=testuser1@adsample.local))
postmap: dict_ldap_get_values[1]: Search found 1 match(es)
postmap: dict_ldap_get_values[1]: search returned 1 value(s) for requested result attribute sAMAccountName
postmap: dict_ldap_get_values[1]: Leaving dict_ldap_get_values
postmap: dict_ldap_lookup: Search returned /var/mail/testuser1/Maildir/
/var/mail/testuser1/Maildir/
postmap: dict_ldap_close: Closed connection handle for LDAP source /etc/postfix/_ldap-mailbox.cf
[root@mail ~]#

それっぽい動作をすることを確認して、postfixを再起動してみたところ”virtual_transport= dovecot”設定を無視して postfix側でLDAPを見に行くことを確認。(/etc/postfix/ldap-mailbox.cfにdebuglevel=10を設定すると、 /var/log/maillog に詳細ログが出ているので)

どうやら、2025年5月時点では postfix/dovecotでActive Directory連携するときは、postfix側ではLDAP連携をする必要はないようだ（aliasなどをAD側で設定している場合は別)

最終的に行った設定内容

結局のところ「Dovecot LDA with Postfix」の Virutal users記載ベースで/etc/postfix/master.cfの末尾にdovecot用設定2行追加。(元ネタでは /usr/local/libexec以下にあるけどRHEL9では/usr/libexecという違いに注意)

[root@mail ~]# tail /etc/postfix/master.cf
#
#scalemail-backend unix -       n       n       -       2       pipe
#  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
#  ${nexthop} ${user} ${extension}
#
#mailman   unix  -       n       n       -       -       pipe
#  flags=FRX user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
#  ${nexthop} ${user}
dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/libexec/dovecot/dovecot-lda -f ${sender} -d ${recipient}
[root@mail ~]#

/etc/postfix/main.cf に以下を追加

[root@mail ~]# tail -5 /etc/postfix/main.cf
#
virtual_mailbox_domains = adsample.local
virtual_transport= dovecot
dovecot_destination_recipient_limit = 1
[root@mail ~]#

なお、この時にdovecot側のLDAP連携設定は以下となっている。(メールソフト側でIMAP/POP3用に入力するユーザ名をドメイン名付きにしたい場合はuserPrincipalName、ユーザ名のみにしたい場合は samAccountName を使う) 

[root@mail ~]# cat /etc/dovecot/dovecot-ldap.conf.ext
dn= cn=vmail,cn=Users,dc=adsample,dc=local
dnpass= パスワード
#pass_filter= (samAccountName=%u)
#user_filter= (samAccountName=%u)
user_filter= (userPrincipalName=%u)
pass_filter= (userPrincipalName=%u)
user_attrs = =uid=1000, =gid=1000
pass_attrs = =uid=1000, =gid=1000

auth_bind=yes
uris=ldaps://192.168.122.10
base= cn=Users,dc=adsample,dc=local
scope=subtree

[root@mail ~]# cat /etc/dovecot/conf.d/auth-ldap.conf.ext
# Authentication for LDAP users. Included from 10-auth.conf.
#
# <doc/wiki/AuthDatabase.LDAP.txt>

passdb {
  driver = ldap

  # Path for LDAP configuration file, see example-config/dovecot-ldap.conf.ext
  args = /etc/dovecot/dovecot-ldap.conf.ext
}

# "prefetch" user database means that the passdb already provided the
# needed information and there's no need to do a separate userdb lookup.
# <doc/wiki/UserDatabase.Prefetch.txt>
#userdb {
#  driver = prefetch
#}

userdb {
  driver = ldap
  args = /etc/dovecot/dovecot-ldap.conf.ext

  # Default fields can be used to specify defaults that LDAP may override
  #default_fields = home=/home/virtual/%u
  default_fields = uid=vmail gid=vmail
}

# If you don't have any user-specific settings, you can avoid the userdb LDAP
# lookup by using userdb static instead of userdb ldap, for example:
# <doc/wiki/UserDatabase.Static.txt>
#userdb {
  #driver = static
  #args = uid=vmail gid=vmail home=/var/vmail/%u
#}
[root@mail ~]#

また /etc/dovecot/conf.d/10-mail.conf に mail_location = maildir:/var/mail/%n/Maildir という設定を追加している。

[root@mail ~]# diff -u /etc/dovecot/conf.d/10-mail.conf.org /etc/dovecot/conf.d/10-mail.conf
--- /etc/dovecot/conf.d/10-mail.conf.org        2025-04-25 03:13:54.044373479 +0900
+++ /etc/dovecot/conf.d/10-mail.conf    2025-04-30 10:59:12.661404241 +0900
@@ -27,7 +27,7 @@
 #
 # <doc/wiki/MailLocation.txt>
 #
-#mail_location =
+mail_location = maildir:/var/mail/%n/Maildir

 # If you need to set multiple mailbox locations or want to change default
 # namespace settings, you can do it by defining namespace sections.
[root@mail ~]#

エラー対処

/var/log/maillog を見てたら下記のようなpermission問題があった

May  1 18:28:19 mail dovecot[924]: auth: Debug: userdb out: USER#0111#011testuser2@adsample.local#011uid=1000#011gid=1000
May  1 18:28:19 mail dovecot[2244]: lda(testuser2@adsample.local)<2244></OWMMLM+E2jECAAAOg3h0A>: msgid=<9da96806-84e5-4f26-9752-acf16b48d4dc@adsample.local>: saved mail to INBOX
May  1 18:28:19 mail postfix/pipe[2238]: AD3C12037F14: to=<testuser2@adsample.local>, orig_to=<testuser2@adosakana.local>, relay=dovecot, delay=0.13, delays=0.02/0/0/0.11, dsn=2.0.0, status=sent (delivered via dovecot service (lda(testuser2@adsample.local): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission))
May  1 18:28:19 mail postfix/qmgr[2205]: AD3C12037F14: removed

エラー対処を調べると2019年1月のiredmailフォーラムの「Re: Error: net_connect_unix(/var/run/dovecot/stats-writer) failed」 が出てくる

これだと /etc/dovecot/dovecot.conf に直接追加しているが、 /etc/dovecot/conf.d/に新しいファイルを作って追加することにした

[root@mail conf.d]# vi /etc/dovecot/conf.d/12-stat.conf
[root@mail conf.d]# cat /etc/dovecot/conf.d/12-stat.conf
service stats {
    unix_listener stats-reader {
        user = vmail
        group = vmail
        mode = 0660
    }

    unix_listener stats-writer {
        user = vmail
        group = vmail
        mode = 0660
    }
}
[root@mail conf.d]# systemctl restart dovecot
[root@mail conf.d]#

これで出力されなくなった

メール送信時の細かい設定について

/etc/dovecot/dovecot-ldap.conf.ext と /etc/dovecot/conf.d/auth-ldap.conf.ext のそれぞれでuid/gidに関する設定を入れているが、それぞれの必要性について再検証

というのは、設定してるなかで、片方だけ設定を入れているとmissingログが出てたから両方入れた・・・はずだったんだけど、それって正しいの？という検証

まずは/etc/dovecot/conf.d/99-debug.conf にdebugログ出力設定を書いて 実施

[root@mail ~]# cat /etc/dovecot/conf.d/99-debug.conf
auth_debug=yes
auth_debug_passwords=yes
auth_verbose=yes
auth_verbose_passwords=yes
verbose_proctitle=yes
verbose_ssl=yes

[root@mail ~]#

ケース1

conf.d/auth-ldap.conf.ext に 「default_fields = uid=vmail gid=vmail」
dovecot-ldap.conf.ext で user_atrrs,pass_attrsの設定なし

この時メールを送信すると以下のような”result: uid missing”, “result: homeDirectory missing; uidNumber missing; gidNumber missing”ログがある

May  2 09:45:39 mail dovecot[1564]: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011session=BnUjeBw0hcjAqHoB#011lip=192.168.122.12#011rip=192.168.122.1#011lport=143#011rport=51333
May  2 09:45:39 mail dovecot[1564]: auth: Debug: client passdb out: CONT#0111
May  2 09:45:39 mail dovecot[1564]: auth: Debug: client in: CONT#0111#011AHRlc3R1c2VyMkBhZHNhbXBsZS5sb2NhbABkaWdpdGFsMTIzQSM= (previous base64 data may contain sensitive data)
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): Performing passdb lookup
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): bind search: base=cn=Users,dc=adsample,dc=local filter=(userPrincipalName=testuser2@adsample.local)
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): no fields returned by the server
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): result:  uid missing
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): Finished passdb lookup
May  2 09:45:39 mail dovecot[1564]: auth: Debug: auth(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): Auth request finished
May  2 09:45:39 mail dovecot[1564]: auth: Debug: client passdb out: OK#0111#011user=testuser2@adsample.local
May  2 09:45:39 mail dovecot[1564]: auth: Debug: master in: REQUEST#0112287206401#0111566#0111#011bef1c5eb0e8a2b05cf297e143bfb1cc6#011session_pid=1569#011request_auth_token
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): Performing userdb lookup
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): user search: base=cn=Users,dc=adsample,dc=local scope=subtree filter=(userPrincipalName=testuser2@adsample.local) fields=homeDirectory,uidNumber,gidNumber
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): no fields returned by the server
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): result:  homeDirectory missing; uidNumber missing; gidNumber missing
May  2 09:45:39 mail dovecot[1564]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<BnUjeBw0hcjAqHoB>): Finished userdb lookup
May  2 09:45:39 mail dovecot[1564]: auth: Debug: master userdb out: USER#0112287206401#011testuser2@adsample.local#011uid=1000#011gid=1000#011auth_mech=PLAIN#011auth_token=aaa072efd8fc1ceb4051c9c51b1fdbc9437e7f81

ケース2

conf.d/auth-ldap.conf.ext に default_fields 設定なし
dovecot-ldap.conf.ext で “user_attrs = =uid=1000, =gid=1000″,”pass_attrs = =uid=1000, =gid=1000” 設定あり

この場合、ログに xxx missing が出ない？


May  2 09:54:48 mail dovecot[959]: auth: Debug: client in: CONT#0111#011AHRlc3R1c2VyMkBhZHNhbXBsZS5sb2NhbABkaWdpdGFsMTIzQSM= (previous base64 data may contain sensitive data)
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): Performing passdb lookup
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): bind search: base=cn=Users,dc=adsample,dc=local filter=(userPrincipalName=testuser2@adsample.local)
May  2 09:54:48 mail dovecot[959]: auth: Debug: master in: USER#0111#011testuser1@adsample.local#011service=lda
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser1@adsample.local): Performing userdb lookup
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser1@adsample.local): user search: base=cn=Users,dc=adsample,dc=local scope=subtree filter=(userPrincipalName=testuser1@adsample.local) fields=
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): result: objectClass=top,top,top,top cn=testuser2 givenName=testuser2 distinguishedName=CN=testuser2,CN=Users,DC=adsample,DC=local instanceType=4 whenCreated=20250417094639.0Z whenChanged=20250430015944.0Z displayName=testuser2 uSNCreated=12616 uSNChanged=40983 name=testuser2 objectGUID=�u&#016$|-E�Z#034\O?�#036 userAccountControl=66048 badPwdCount=0 codePage=0 countryCode=0 badPasswordTime=133893655225944004 lastLogoff=0 lastLogon=133894260848965097 pwdLastSet=133893567990887154 primaryGroupID=513 objectSid=<no values> accountExpires=9223372036854775807 logonCount=16 sAMAccountName=testuser2 sAMAccountType=805306368 userPrincipalName=testuser2@adsample.local objectCategory=CN=Person,CN=Schema,CN=Configuration,DC=adsample,DC=local dSCorePropagationData=20250418015428.0Z,20250418015428.0Z lastLogonTimestamp=133904519848472342; objectGUID,uSNCreated,objectCategory,objectClass,primaryGroupID,cn,givenName,objectSid,sAMAccountType,dSCorePropagationData,userAccountControl,name,codePage,lastLogon,logonCount,countryCode,lastLogoff,uSNChanged,pwdLastSet,distinguishedName,sAMAccountName,whenChanged,userPrincipalName,instanceType,badPwdCount,accountExpires,whenCreated,displayName,badPasswordTime,lastLogonTimestamp unused
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser1@adsample.local): result: objectClass=top,top,top,top cn=testuser1 givenName=testuser1 distinguishedName=CN=testuser1,CN=Users,DC=adsample,DC=local instanceType=4 whenCreated=20250417094618.0Z whenChanged=20250425001141.0Z displayName=testuser1 uSNCreated=12609 uSNChanged=36883 name=testuser1 objectGUID=#037��#�!#020F�j#020#010��#011x userAccountControl=66048 badPwdCount=0 codePage=0 countryCode=0 badPasswordTime=133900339076624909 lastLogoff=0 lastLogon=133900339256453379 pwdLastSet=133893567784742554 primaryGroupID=513 objectSid=<no values> accountExpires=9223372036854775807 logonCount=0 sAMAccountName=testuser1 sAMAccountType=805306368 userPrincipalName=testuser1@adsample.local objectCategory=CN=Person,CN=Schema,CN=Configuration,DC=adsample,DC=local dSCorePropagationData=20250418015428.0Z,20250418015428.0Z lastLogonTimestamp=133900135017739905 mail=testuser1@example.com; objectGUID,uSNCreated,objectCategory,objectClass,primaryGroupID,cn,givenName,objectSid,sAMAccountType,dSCorePropagationData,userAccountControl,name,mail,codePage,lastLogon,logonCount,countryCode,lastLogoff,uSNChanged,pwdLastSet,distinguishedName,sAMAccountName,whenChanged,userPrincipalName,instanceType,badPwdCount,accountExpires,whenCreated,displayName,badPasswordTime,lastLogonTimestamp unused
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser1@adsample.local): Finished userdb lookup
May  2 09:54:48 mail dovecot[959]: auth: Debug: userdb out: USER#0111#011testuser1@adsample.local#011uid=1000#011gid=1000
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): Finished passdb lookup
May  2 09:54:48 mail dovecot[959]: auth: Debug: auth(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): Auth request finished
May  2 09:54:48 mail dovecot[959]: auth: Debug: client passdb out: OK#0111#011user=testuser2@adsample.local#011uid=1000#011gid=1000
May  2 09:54:48 mail dovecot[1423]: lda(testuser1@adsample.local)<1423><jFPYJdgXFGiPBQAAOg3h0A>: msgid=<c923fd79-8135-46e3-aa56-afd1a1282ff8@adsample.local>: saved mail to INBOX
May  2 09:54:48 mail postfix/pipe[1422]: 7FF342020373: to=<testuser1@adsample.local>, orig_to=<testuser1@adosakana.local>, relay=dovecot, delay=0.15, delays=0.03/0.01/0/0.12, dsn=2.0.0, status=sent (delivered via dovecot service)
May  2 09:54:48 mail postfix/qmgr[884]: 7FF342020373: removed
May  2 09:54:48 mail dovecot[959]: auth: Debug: master in: REQUEST#0113980787713#0111424#0111#011d2548dbb496c7cf3c62e2637869d2d69#011session_pid=1425#011request_auth_token
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): Performing userdb lookup
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): user search: base=cn=Users,dc=adsample,dc=local scope=subtree filter=(userPrincipalName=testuser2@adsample.local) fields=
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): result: objectClass=top,top,top,top cn=testuser2 givenName=testuser2 distinguishedName=CN=testuser2,CN=Users,DC=adsample,DC=local instanceType=4 whenCreated=20250417094639.0Z whenChanged=20250430015944.0Z displayName=testuser2 uSNCreated=12616 uSNChanged=40983 name=testuser2 objectGUID=�u&#016$|-E�Z#034\O?�#036 userAccountControl=66048 badPwdCount=0 codePage=0 countryCode=0 badPasswordTime=133893655225944004 lastLogoff=0 lastLogon=133894260848965097 pwdLastSet=133893567990887154 primaryGroupID=513 objectSid=<no values> accountExpires=9223372036854775807 logonCount=16 sAMAccountName=testuser2 sAMAccountType=805306368 userPrincipalName=testuser2@adsample.local objectCategory=CN=Person,CN=Schema,CN=Configuration,DC=adsample,DC=local dSCorePropagationData=20250418015428.0Z,20250418015428.0Z lastLogonTimestamp=133904519848472342; objectGUID,uSNCreated,objectCategory,objectClass,primaryGroupID,cn,givenName,objectSid,sAMAccountType,dSCorePropagationData,userAccountControl,name,codePage,lastLogon,logonCount,countryCode,lastLogoff,uSNChanged,pwdLastSet,distinguishedName,sAMAccountName,whenChanged,userPrincipalName,instanceType,badPwdCount,accountExpires,whenCreated,displayName,badPasswordTime,lastLogonTimestamp unused
May  2 09:54:48 mail dovecot[959]: auth: Debug: ldap(testuser2@adsample.local,192.168.122.1,<D2PcmBw0S8rAqHoB>): Finished userdb lookup
May  2 09:54:48 mail dovecot[959]: auth: Debug: master userdb out: USER#0113980787713#011testuser2@adsample.local#011uid=1000#011gid=1000#011auth_mech=PLAIN#011auth_token=68518eb1e5a886522b5dcabaae828704633386c9

missing 出力がない？？

そもそも、同じログレベル出力なのに、出力された内容に差がありすぎるんですが・・・

dovecot-ldap.conf.ext に “user_attrs = =uid=1000, =gid=1000″,”pass_attrs = =uid=1000, =gid=1000” を設定するだけでいける？