esxcli storage core device list
現在認識してるストレージについてのいろんな情報
[root@esxi10:~] esxcli storage core device list eui.9fa59185cb34c2ab6c9ce9003faf076a Display Name: Nimble iSCSI Disk (eui.9fa59185cb34c2ab6c9ce9003faf076a) Has Settable Display Name: true Size: 5242880 Device Type: Direct-Access Multipath Plugin: NMP Devfs Path: /vmfs/devices/disks/eui.9fa59185cb34c2ab6c9ce9003faf076a Vendor: Nimble Model: Server Revision: 1.0 SCSI Level: 5 Is Pseudo: false Status: on Is RDM Capable: true Is Local: false Is Removable: false Is SSD: false Is VVOL PE: false Is Offline: false Is Perennially Reserved: false Queue Full Sample Size: 0 Queue Full Threshold: 0 Thin Provisioning Status: yes Attached Filters: VAAI Status: supported Other UIDs: vml.01000000003966613539313835636233346332616236633963653930303366616630373661536572766572 Is Shared Clusterwide: true Is Local SAS Device: false Is SAS: false Is USB: false Is Boot USB Device: false Is Boot Device: false Device Max Queue Depth: 128 No of outstanding IOs with competing worlds: 32 Drive Type: unknown RAID Level: unknown Number of Physical Drives: unknown Protection Enabled: false PI Activated: false PI Type: 0 PI Protection Mask: NO PROTECTION Supported Guard Types: NO GUARD SUPPORT DIX Enabled: false DIX Guard Type: NO GUARD SUPPORT Emulated DIX/DIF Enabled: false [root@esxi10:~]
個別で見る場合は「esxcli storage core device list -d ストレージ指定」
[root@esxi10:~] esxcli storage core device list -d eui.9fa59185cb34c2ab6c9ce9003faf076a eui.9fa59185cb34c2ab6c9ce9003faf076a Display Name: Nimble iSCSI Disk (eui.9fa59185cb34c2ab6c9ce9003faf076a) Has Settable Display Name: true Size: 5242880 Device Type: Direct-Access Multipath Plugin: NMP Devfs Path: /vmfs/devices/disks/eui.9fa59185cb34c2ab6c9ce9003faf076a Vendor: Nimble Model: Server Revision: 1.0 SCSI Level: 5 Is Pseudo: false Status: on Is RDM Capable: true Is Local: false Is Removable: false Is SSD: false Is VVOL PE: false Is Offline: false Is Perennially Reserved: false Queue Full Sample Size: 0 Queue Full Threshold: 0 Thin Provisioning Status: yes Attached Filters: VAAI Status: supported Other UIDs: vml.01000000003966613539313835636233346332616236633963653930303366616630373661536572766572 Is Shared Clusterwide: true Is Local SAS Device: false Is SAS: false Is USB: false Is Boot USB Device: false Is Boot Device: false Device Max Queue Depth: 128 No of outstanding IOs with competing worlds: 32 Drive Type: unknown RAID Level: unknown Number of Physical Drives: unknown Protection Enabled: false PI Activated: false PI Type: 0 PI Protection Mask: NO PROTECTION Supported Guard Types: NO GUARD SUPPORT DIX Enabled: false DIX Guard Type: NO GUARD SUPPORT Emulated DIX/DIF Enabled: false [root@esxi10:~]
ESXiで認識しているファイルシステム一覧
esxcli storage filesystem list
[root@esxi10:~] esxcli storage filesystem list Mount Point Volume Name UUID Mounted Type Size Free ---------------------------------------------------- ---------------------------- -------------------------------------- ------- ------ ------------- ------------- /vmfs/volumes/5bbc594e-3e173ecc-b3e7-90b11c536183 esxstore-vsphere10 5bbc594e-3e173ecc-b3e7-90b11c536183 true VMFS-5 5497289703424 2186855579648 /vmfs/volumes/b49358e5-23dc34a5-1c9f-5af965bf7cb0 b49358e5-23dc34a5-1c9f-5af965bf7cb0 true vfat 261853184 79454208 /vmfs/volumes/4aed47ef-d4e7ee55-f394-88684bc4443c 4aed47ef-d4e7ee55-f394-88684bc4443c true vfat 261853184 79417344 /vmfs/volumes/5d47ebcd-a4552bf1-c3e2-90b11c536183 5d47ebcd-a4552bf1-c3e2-90b11c536183 true vfat 299712512 84172800 [root@esxi10:~]
指定したVMFSの情報を確認
「esxcli storage vmfs host list –volume-label 名前」
[root@esxi10:~] esxcli storage vmfs host list --volume-label esxstore-vsphere10 Heartbeat UUID MAC Address Driver Version Heartbeat Offset Locking Mode Status ----------------------------------- ----------------- -------------- ---------------- ------------ ------ 5d6c6d91-51b61938-424a-90b11c536183 90:b1:1c:53:61:83 5.61 3178496 ATS-only Normal [root@esxi10:~]
ストレージアクセスのQueue関連設定変更
現在の値確認は「esxcli storage core device list -d ストレージ指定」で行う。
「Device Max Queue Depth」を「128」に設定する
[root@esxi10:~]esxcli storage core device set -d eui.9fa59185cb34c2ab6c9ce9003faf076a -m 128 [root@esxi10:~]
「No of outstanding IOs with competing worlds」を「32」に設定する
[root@esxi10:~]esxcli storage core device set -d eui.9fa59185cb34c2ab6c9ce9003faf076a -O 32 [root@esxi10:~]
ストレージパス関連
esxcli storage core path list
[root@esxi10:~] esxcli storage core path list iqn.1998-01.com.vmware:esxi10-753076d4-00023d000001,iqn.2007-11.com.nimblestorage:esxstore-vsphere10-v371c7edc5d1bd1e3.000000ab.6a07af3f,t,2460-eui.9fa59185cb34c2ab6c9ce9003faf076a UID: iqn.1998-01.com.vmware:esxi10-753076d4-00023d000001,iqn.2007-11.com.nimblestorage:esxstore-vsphere10-v371c7edc5d1bd1e3.000000ab.6a07af3f,t,2460-eui.9fa59185cb34c2ab6c9ce9003faf076a Runtime Name: vmhba38:C0:T1:L0 Device: eui.9fa59185cb34c2ab6c9ce9003faf076a Device Display Name: Nimble iSCSI Disk (eui.9fa59185cb34c2ab6c9ce9003faf076a) Adapter: vmhba38 Channel: 0 Target: 1 LUN: 0 Plugin: NMP State: active Transport: iscsi Adapter Identifier: iqn.1998-01.com.vmware:esxi10-753076d4 Target Identifier: 00023d000001,iqn.2007-11.com.nimblestorage:esxstore-vsphere10-v371c7edc5d1bd1e3.000000ab.6a07af3f,t,2460 Adapter Transport Details: iqn.1998-01.com.vmware:esxi10-753076d4 Target Transport Details: IQN=iqn.2007-11.com.nimblestorage:esxstore-vsphere10-v371c7edc5d1bd1e3.000000ab.6a07af3f Alias= Session=00023d000001 PortalTag=2460 Maximum IO Size: 131072 [root@esxi10:~]
各ストレージのパス選択アルゴリズム確認
「esxcli storage nmp device list」
[root@esxi10:~] esxcli storage nmp device list
eui.9fa59185cb34c2ab6c9ce9003faf076a
Device Display Name: Nimble iSCSI Disk (eui.9fa59185cb34c2ab6c9ce9003faf076a)
Storage Array Type: VMW_SATP_ALUA
Storage Array Type Device Config: {implicit_support=on;explicit_support=off; explicit_allow=on;alua_followover=on; action_OnRetryErrors=off; {TPG_id=0,TPG_state=AO}}
Path Selection Policy: VMW_PSP_MRU
Path Selection Policy Device Config: Current Path=vmhba38:C0:T1:L0
Path Selection Policy Device Custom Config:
Working Paths: vmhba38:C0:T1:L0
Is USB: false
[root@esxi10:~]
使用できるPSP一覧
[root@esxi10:~] esxcli storage nmp psp list Name Description ------------- --------------------------------- VMW_PSP_MRU Most Recently Used Path Selection VMW_PSP_RR Round Robin Path Selection VMW_PSP_FIXED Fixed Path Selection [root@esxi10:~]
指定できるSATP一覧
[root@esxi10:~] esxcli storage nmp satp list Name Default PSP Description ------------------- ------------- ------------------------------------------------------- VMW_SATP_ALUA VMW_PSP_MRU Supports non-specific arrays that use the ALUA protocol VMW_SATP_MSA VMW_PSP_MRU Placeholder (plugin not loaded) VMW_SATP_DEFAULT_AP VMW_PSP_MRU Placeholder (plugin not loaded) VMW_SATP_SVC VMW_PSP_FIXED Placeholder (plugin not loaded) VMW_SATP_EQL VMW_PSP_FIXED Placeholder (plugin not loaded) VMW_SATP_INV VMW_PSP_FIXED Placeholder (plugin not loaded) VMW_SATP_EVA VMW_PSP_FIXED Placeholder (plugin not loaded) VMW_SATP_ALUA_CX VMW_PSP_RR Placeholder (plugin not loaded) VMW_SATP_SYMM VMW_PSP_RR Placeholder (plugin not loaded) VMW_SATP_CX VMW_PSP_MRU Placeholder (plugin not loaded) VMW_SATP_LSI VMW_PSP_MRU Placeholder (plugin not loaded) VMW_SATP_DEFAULT_AA VMW_PSP_FIXED Supports non-specific active/active arrays VMW_SATP_LOCAL VMW_PSP_FIXED Supports direct attached devices [root@esxi10:~]
パス選択アルゴリズムの変更
[root@esxi10:~] esxcli storage nmp device set --psp=VMW_PSP_RR --device=eui.9fa59185cb34c2ab6c9ce9003faf076a [root@esxi10:~]
変更確認
[root@esxi10:~] esxcli storage nmp device list -d eui.9fa59185cb34c2ab6c9ce9003faf076a
eui.9fa59185cb34c2ab6c9ce9003faf076a
Device Display Name: Nimble iSCSI Disk (eui.9fa59185cb34c2ab6c9ce9003faf076a)
Storage Array Type: VMW_SATP_ALUA
Storage Array Type Device Config: {implicit_support=on;explicit_support=off; explicit_allow=on;alua_followover=on; action_OnRetryErrors=off; {TPG_id=0,TPG_state=AO}}
Path Selection Policy: VMW_PSP_RR
Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
Path Selection Policy Device Custom Config:
Working Paths: vmhba38:C3:T1:L0, vmhba38:C0:T1:L0
Is USB: false
[root@esxi10:~]
HBAドライバパラメータ調整
インストールされているドライババージョン等を確認
[root@esxi10:~] esxcli software vib list|grep qla ima-qla4xxx 2.02.18-1vmw.600.0.0.2494585 VMware VMwareCertified 2019-08-05 scsi-qla4xxx 5.01.03.2-7vmw.600.0.0.2494585 VMware VMwareCertified 2019-08-05 [root@esxi10:~]
HBAで指定できるパラメータ確認
「esxcli system module parameters list -m モジュール名」
[root@esxi10:~] esxcli system module parameters list -m qlnativefc Name Type Value Description -------------------------- ---- ----- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ql2name int Option to have different node and port name for ISP82xxpass value based on the port used on the FCoE switch. ql2xallocfwdump int Option to enable allocation of memory for a firmware dump during HBA initialization. Memory allocation requirements vary by ISP type. Default is 1 - allocate memory. ql2xattemptdumponpanic int Attempt fw dump for each function on PSOD Default is 0 - Don't attempt fw dump. ql2xbypass_log_throttle int Option to bypass log throttling.Default is 0 - Throttling enabled. 1 - Log all errors. ql2xcmdtimeout int Timeout value in seconds for scsi command, default is 20 ql2xcmdtimermin int Minimum command timeout value. Default is 30 seconds. ql2xdbwr int Option to specify scheme for request queue posting. 0 -- Regular doorbell. 1 -- (Default) CAMRAM doorbell (faster). ql2xdevdiscgoldfw int Option to enable device discovery with golden firmware Default is 0 - no discovery. 1 - discover device. ql2xdisablenpiv int Option to disable/enable NPIV feature globally. 1 - NPIV disabled. Default is 0 - NPIV enabled. ql2xdontresethba int 1: Do not reset on failure, 0: Reset on failure, default is 0 ql2xdontresethba_83xx int 1: Do not reset on failure, 0: Reset on failure, default is 0 ql2xenablehba_err_chk int Enable T10-CRC-DIF Error isolation by HBA: Default is 2. 0 -- Error isolation disabled 1 -- Error isolation enabled only for DIX Type 0 2 -- Error isolation enabled for all Types ql2xenablemd int 82XX/8044: Option to enable/disable the minidump capture capability. 0 - disable minidump capture 1 - (Default) enable minidump capture ql2xenablemsi2422 int Enables MSI interrupt scheme on 2422sDefault is 0 - disable MSI-X/MSI. 1 - enable MSI-X/MSI. ql2xenablemsi24xx int Enables MSIx/MSI interrupt scheme on 24xx cardsDefault is 1 - enable MSI-X/MSI. 0 - disable MSI-X/MSI. ql2xenablemsix int Set to enable MSI or MSI-X interrupt mechanism. 0 = enable traditional pin-based interrupt mechanism. 1 = enable MSI-X interrupt mechanism (Default). 2 = enable MSI interrupt mechanism. ql2xenablesmartsan int Enable HP Smart SAN management feature.Default is 1. 0 - Disable the Smart SAN feature. 1 - Enable the Smart SAN feature. ql2xexecution_throttle int IOCB exchange count for HBA.Default is 0, set intended value to override Firmware defaults. ql2xextended_error_logging int Option to enable extended error logging, Default is 0 - no logging. 1 - log errors. ql2xfabricpriorityqos int Option to set QoS mode Default is 2. 0 - Fabric Priority QoS is disabled 1 - Queue-based Fabric Priority QoS enabled. 2 - IOCB-based Fabric Priority QoS enabled. ql2xfdmienable int Enables FDMI registratons Default is 1 - perfom FDMI. 0 - no FDMI. ql2xfwloadbin int Option to specify location from which to load ISP firmware: 2 -- load firmware via the requestFirmware() (hotplug) interface. 1 -- load firmware from flash. 0 -- use default semantics. ql2xiidmaenable int Enables iIDMA settings Default is 1 - perform iIDMA. 0 - no iIDMA. ql2xintrdelaytimer int ZIO: Waiting time for Firmware before it generates an interrupt to the host to notify completion of request. ql2xioctltimeout int IOCTL timeout value in seconds for pass-thur commands. Default is 66 seconds. ql2xioctltimertest int IOCTL timer test enable - set to enable ioctlcommand timeout value to trigger before fw cmdtimeout value. Default is disabled ql2xloadfwbin int Option to Load Firmware from the flash/driver source for qla82xx Default is 1, 0 = load firmware bundled with driver source 1 = load from Flash ql2xloginretrycount int Specify an alternate value for the NVRAM login retry count. ql2xlogintimeout int Login timeout value in seconds. ql2xmaxlun int Defines the maximum LUNs to register with the SCSI midlayer. Default is 256. Maximum allowed is 65535. ql2xmaxqdepth int Maximum queue depth to report for target devices. ql2xmaxsgs int Maximum scatter/gather entries per request,Default is the Max the OS Supports. ql2xmdcapmask int 82XX/8044: Option to set the driver capture mask for f/w minidump as follows: 0x00 - use f/w recommended capture mask 0x03 - capture mask 0x3 0x0F - capture mask 0x0F 0x1F - (Default) capture mask 0x1F 0x7F - capture mask 0x7F ql2xmqcpuaffinity int Enables CPU affinity settings for the driver Default is 1 to turn on the cpu affinity.Set to 0 for no affinity of request and response IO. ql2xmqqos int Enables QoS settings Default is 1. Set it to enable queues in MQ QoS mode. ql2xoperationmode int Option to disable ZIO mode for ISP24XX: Default is 1, set 0 to disable ql2xplogiabsentdevice int Option to enable PLOGI to devices that are not present after a Fabric scan. This is needed for several broken switches. Default is 0 - no PLOGI. 1 - perfom PLOGI. ql2xshiftctondsd int Set to control shifting of command type processing based on total number of DSD. ql2xt10difvendor int Vendor unique T10 DIF support. Default is enabled. 0 -- Disabled. 1 -- Enabled. ql2xt10enable int Enable T10-CRC-DIF Default is 1 - DIF Support Enabled. 0 - Disable it ql2xt10guard int Adapter guard type supported Default is CRC checksumming and IP checksumming. ql2xt10protmask int Adapter protection mask Default is 0x55 - T10 PI Type 1/3, DIX Type 1/3 protection. ql2xusedefmaxrdreq int Default is 0 - adjust PCIe Maximum Read Request Size. 1 - use system default. ql2xuseshadowregisters int For ISP27XX/ISP28XX : Option to use shadow register for request/response queues : Default is 0. 0 -- Don't use shadow registers for the request/response queue manipulation 1 -- Use shadow registers for the request/response queue manipulation ql2xvmidexpiration int Expiration time for VMID in minutes. 0 is no expiration. Default is 240 minutes (4 hours) with no I/O leads to de-registration. ql2xvmidsupport int Enable VMID support in driver. Default is disabled. 0 -- Disabled. 1 -- Enabled. ql2xvmidtgtind int Ignore target settings for QoS VMID. Default is disabled. 0 -- Disabled. 1 -- Enabled. qlport_down_retry int Maximum number of command retries to a port that returns a PORT-DOWN status. [root@esxi10:~]
「esxcfg-module –show-info モジュール名」
[root@esxi10:~] esxcfg-module --show-info qlnativefc esxcfg-module module information input file: /usr/lib/vmware/vmkmod/qlnativefc License: BSD Cannot access module instance. [root@esxi10:~]
値の設定
[root@esxi10:~] esxcfg-module --get-options qlnativefc qlnativefc enabled = 1 options = '' [root@esxi10:~]
