We have ess3500 hybrid model, we want to define NVME disks as MetaDataOnly, and NL-SAS disks as DataOnly in the same storage pool - system.
Today mmvdisk does not allow it, but we can do it in Scale:
[root@ems4 var]# mmvdisk vs define --vs vs_ESS4_ESS35K_DA2_system1 --rg rg_ESS4_ESS35K_01 --code 8+2p --block-size 4M --set-size 25% --da DA2 --nsd-usage dataOnly --storage-pool system
mmvdisk: The storage pool can not be system when NSD usage is set to 'dataOnly'.
mmvdisk: Command failed. Examine previous error messages to determine cause.
In early days of ESS LE - the setup create 2 storage pools - system (metadataonly - 3% of the capacity), data (dataonly - 97% of the capacity).
Later versions create only system pool - which was defined as DataAndMetaData.
But since customers are moving the metadata to ess3500 NVME disks, we had to add the NVME disks to the File-System - and only way that mmvdisk support is DataAndMetaData, so we add them , and then change them to MetaDataOnly, and this way we have dedicade disks for MD (NVME) , and Data disks (DataOnly).
[root@gssio1e35k-ess7 ~]# mmlspool gpfs7 --block-size auto
Storage pools in file system at '/data/gpfs7':
Name Id BlkSize Data Meta Total Data Free Data Total Meta Free Meta
system 0 8 MB yes yes 9.819P 5.395P ( 55%) 47.27T 40.94T ( 87%)
[root@gssio1e35k-ess7 ~]# mmdf gpfs7
disk disk size failure holds holds free in KB free in KB
name in KB group metadata data in full blocks in fragments
--------------- ------------- -------- -------- ----- -------------------- -------------------
Disks in storage pool: system (Maximum disk size allowed is 19.34 PB)
RG001LG006VS013 690359369728 1 no yes 422202007552 ( 61%) 2442330672 ( 0%)
RG001LG001VS002 3172040704 1 yes no 2707136512 ( 85%) 40930576 ( 1%)
RG005VS001 2636802162688 1 no yes 1276963037184 ( 48%) 19627898160 ( 1%)
RG006VS001 2636802162688 1 no yes 1276630548480 ( 48%) 20593450320 ( 1%)
RG001LG003VS002 3172040704 1 yes no 2707447808 ( 85%) 40512448 ( 1%)
RG001LG006VS012 690359369728 1 no yes 422196641792 ( 61%) 2437469456 ( 0%)
RG001LG001VS003 3172040704 1 yes no 2707447808 ( 85%) 40515440 ( 1%)
RG001LG006VS011 690359369728 1 no yes 422261440512 ( 61%) 2442862496 ( 0%)
RG001LG003VS003 3172040704 1 yes no 2707267584 ( 85%) 40414416 ( 1%)
RG001LG006VS010 690359369728 1 no yes 422203088896 ( 61%) 2437393408 ( 0%)
RG001LG001VS004 3172040704 1 yes no 2707193856 ( 85%) 40768688 ( 1%)
RG001LG005VS009 690359369728 1 no yes 422277447680 ( 61%) 2441728672 ( 0%)
RG001LG003VS004 3172040704 1 yes no 2707775488 ( 85%) 40155952 ( 1%)
RG001LG005VS008 690359369728 1 no yes 422233997312 ( 61%) 2442661536 ( 0%)
RG001LG001VS005 3172040704 1 yes no 2707955712 ( 85%) 39941856 ( 1%)
RG001LG005VS007 690359369728 1 no yes 422222012416 ( 61%) 2439752400 ( 0%)
RG001LG003VS005 3172040704 1 yes no 2706817024 ( 85%) 41160736 ( 1%)
RG001LG005VS006 690359369728 1 no yes 422221447168 ( 61%) 2444175152 ( 0%)
RG001LG004VS005 3172040704 2 yes no 2706284544 ( 85%) 40531056 ( 1%)
RG001LG002VS005 3172040704 2 yes no 2705850368 ( 85%) 40890544 ( 1%)
RG001LG004VS004 3172040704 2 yes no 2706497536 ( 85%) 40362192 ( 1%)
RG001LG002VS004 3172040704 2 yes no 2706907136 ( 85%) 40228816 ( 1%)
RG001LG004VS003 3172040704 2 yes no 2706890752 ( 85%) 39960128 ( 1%)
RG001LG002VS003 3172040704 2 yes no 2706776064 ( 85%) 40152896 ( 1%)
RG001LG004VS002 3172040704 2 yes no 2706259968 ( 85%) 40574560 ( 1%)
RG001LG002VS002 3172040704 2 yes no 2706243584 ( 85%) 40278896 ( 1%)
------------- -------------------- -------------------
(pool total) 10847231934464 5974722420736 ( 55%) 60397101472 ( 1%)
============= ==================== ===================
(data) 10796479283200 5931411668992 ( 55%) 59749722272 ( 1%)
(metadata) 50752651264 43310751744 ( 85%) 647379200 ( 1%)
============= ==================== ===================
(total) 10847231934464 5974722420736 ( 55%) 60397101472 ( 1%)
Inode Information
-----------------
Total number of used inodes in all Inode spaces: 275120519
Total number of free inodes in all Inode spaces: 1171011193
Total number of allocated inodes in all Inode spaces: 1446131712
Total of Maximum number of inodes in all Inode spaces: 1579937792
So, every time we need to expand the Data with ESS nodes, or expand the Metadata with NVME disks - we need to define the as DataAndMetaData and change it according to the disk usage.
