Identificare il Disco Fisico associato ad un OSD in Ceph

From RVM Wiki
Jump to navigation Jump to search

Generico

Si usa 

ceph osd metadata

I campi che interesando sono 

"id"
"bluefs_db_dev_node"
"bluestore_bdev_dev_node"

In sintesi: 

"devices"
"device_paths"

Indicano tutti i dischi che interesano l'OSD (dai, db e wal)

Se si vuole sapere quali sono i LV ineterssati, vedere 

   "bluefs_db_dev_node": "/dev/dm-7",
   "bluefs_db_partition_path": "/dev/dm-7",
   "bluestore_bdev_dev_node": "/dev/dm-8",
   "bluestore_bdev_partition_path": "/dev/dm-8",

Che corrispondono a: 

ls /dev/disk/by-id/| grep 'dm-7\|dm-8'

lrwxrwxrwx 1 root root  10 Aug 11 19:04 dm-name-ceph--6f43a8ea--7a77--4868--ab6b--7dba36fe6ade-osd--db--4a24a2d2--5039--4760--a787--24b3445314dc -> ../../dm-7
lrwxrwxrwx 1 root root  10 Aug 11 19:04 dm-name-ceph--f6f66a9d--a368--4445--a21b--0d80dba60d70-osd--block--d7799c89--1211--45f8--9d96--91a835440093 -> ../../dm-8
lrwxrwxrwx 1 root root  10 Aug 11 19:04 dm-uuid-LVM-Fnud2KPpjz9SHQ0qwujyMkaYuMQmpkOR978WtyDhEgNfWFoLvYasjNuLRn0Ppc09 -> ../../dm-8
lrwxrwxrwx 1 root root  10 Aug 11 19:04 dm-uuid-LVM-oe9KlYHqixxStkxfAi8i1AOc7vHmP94BASMddcIGM743HiGeVpgluUGCM6zxWzqm -> ../../dm-7

Che quindi sono: 

sudo lvs

  LV                                             VG                                        Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
#...
  osd-db-de0e77dc-2d5d-4428-b3d7-913d7e5991be    ceph-6f43a8ea-7a77-4868-ab6b-7dba36fe6ade -wi-ao----   3.20g                                                    
#...
  osd-block-d7799c89-1211-45f8-9d96-91a835440093 ceph-f6f66a9d-a368-4445-a21b-0d80dba60d70 -wi-ao---- <32.00g                                                    
#...

Questo comando salva dettagli: 

rm osd.details; for ID in $OSD; do sudo ceph osd metadata $ID | grep 'device_paths\|osd_objectstore\|default_device_class\|\"id\"' >> osd.details; echo -e "\n" >> osd.details; done; cat osd.details


OSD Bluestore

Se l'OSD è di tipo bluestore, sarà gestito da un LV, che avrà un proprio VG con un solo PV.

Ad esempio se l'OSD è 20 proseguire come di seguito o andare direttamente allo script onliner alla fine.

  • Identificare il link al block device:
readlink /var/lib/ceph/osd/ceph-20/block

/dev/ceph-9d1ad516-c99d-4bea-a08d-cc248f47987f/osd-block-906c9230-0990-4ef3-864b-f00e0003fdf6
  • Vedere il dettaglio del VG per trovare il PV:
vgdisplay -v ceph-9d1ad516-c99d-4bea-a08d-cc248f47987f

  --- Volume group ---
  VG Name               ceph-9d1ad516-c99d-4bea-a08d-cc248f47987f
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  17
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               894.25 GiB
  PE Size               4.00 MiB
  Total PE              228928
  Alloc PE / Size       228928 / 894.25 GiB
  Free  PE / Size       0 / 0   
  VG UUID               6mbBYN-kwTB-qAPW-Dq0d-tSBZ-Zmnv-PfKqm1
   
  --- Logical volume ---
  LV Path                /dev/ceph-9d1ad516-c99d-4bea-a08d-cc248f47987f/osd-block-906c9230-0990-4ef3-864b-f00e0003fdf6
  LV Name                osd-block-906c9230-0990-4ef3-864b-f00e0003fdf6
  VG Name                ceph-9d1ad516-c99d-4bea-a08d-cc248f47987f
  LV UUID                7wtfHT-JGs3-lIks-ev2l-6atd-U2XU-wpyOND
  LV Write Access        read/write
  LV Creation host, time crostor03, 2021-04-26 16:05:34 +0200
  LV Status              available
  # open                 4
  LV Size                894.25 GiB
  Current LE             228928
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1
   
  --- Physical volumes ---
  PV Name               /dev/sdg     
  PV UUID               eQi3SF-s30C-8fY2-o490-W9eD-CyOI-OyEfOV
  PV Status             allocatable
  Total PE / Free PE    228928 / 0
  • Il disco è l'unico PV usato, cioè :
/dev/sdg
  • Con uno script:
export OSDID=20; vgdisplay -v $(readlink /var/lib/ceph/osd/ceph-${OSDID}/block  | cut -f 3 --delimiter='/') | grep 'PV Name' | tr -s ' '| cut -f 4 --delimiter=' '

/dev/sdg

OSD Filestore con vecchio formato montato

Se l'OSD è di formato filestore, sarà montato: 

export OSDID=18; df -h | grep ${OSDID}

/dev/sdg1             927G  400G  527G  44% /var/lib/ceph/osd/ceph-18

Quindi il disco fisico è 

/dev/sdg

Identificazione proprietà del disco

  • Seriale
smartctl -i /dev/sdg | grep Serial
  • Slot SCSI:
ls /dev/disk/by-path | grep sdg

Riferimenti

  • [ceph-users Need to replace OSD. How do I find physical disk]
Retrieved from "https://kb.rvmgroup.it/index.php?title=Identificare_il_Disco_Fisico_associato_ad_un_OSD_in_Ceph&oldid=10797"