今天今天今天检查1/2919039objectsdegraded(0.000%),1pgdegradedOBJECT_UNFOUND1/973013objectsunfound(0.000%)pg2.2bhas1unfoundobjectsOSD_SCRUB_ERRORS17scruberrorsPG_DAMAGEDPossibledatadamage:1pgrecovery_unfound,8pgsinconsistent,1pgrepairpg2.2bisactive+recovery_unfound+degraded,acting[14,22,4],1unfoundpg2.44isactive+clean+inconsistent,acting[14,8,21]pg2.73isactive+clean+inconsistent,acting[25,14,8]pg2.80isactive+clean+scrubbing+deep+inconsistent+repair,acting[4,8,14]pg2.83isactive+clean+不一致,作用[14,13,6]pg2.aeisactive+clean+不一致,作用[14,3,2]pg2.c4isactive+clean+不一致,作用[8,21,14]pg2.daisactive+clean+不一致,acting[23,14,15]pg2.faisactive+clean+inconsistent,acting[14,23,25]PG_DEGRADEDDegradeddataredundancy:1/2919039objectsdegraded(0.000%),1pgdegradedpg2.2bisactive+recovery_unfound+degraded,acting[14,22,4],1unfound从输出中发现pg2.2b是active+recovery_unfound+degraded,acting[14,22,4],1unfound下面我们来看看pg2.2b,看看这个pg的思路信息。[root@k8snode001~]#cephpgdump_jsonpools|grep2.2bdumpedall2.2b24871101953319840330483048active+recovery_unfound+degraded2020-07-2308:56:07.66990310373'544837010373:7312614[14,22,4]14[14,22,4]1410371'54372582020-07-2308:56:06.63701210371'54372582020-07-2308:56:06.6370120可以看到现在只有一份2.查看pgmap[root@k8snode001~]#cephpgmap2.2bosdmape10373pg2.2b(2.2b)->up[14,22,4]acting[14,22,4]从pgmap可以看出pg2.2b被分发到osd[14,22,4]3.查看存储池状态[root@k8snode001~]#cephosdpoolstatsk8s-1poolk8s-1id21/1955664objectsdegraded(0.000%)1/651888objectsunfound(0.000%)clientio271KiB/swr,0op/srd,52op/swr[root@k8snode001~]#cephosdpoollsdetail|pool12'k8s-1replicatedsize3min_size1crush_rule0object_hashrjenkinspg_num256pgp_num256last_change88flagshashpspool,selfmanaged_snapsstripe_width0应用程序rbd4。尝试恢复pg2.2b丢失的剧情[root@k8snode001~]#cephpgrepair2.2b如果修复一直不成功,可以查看卡住PG的具体信息,主要关注recovery_state,命令如下[root@k8snode001~]#cephpg2。2bquery{"..."recovery_state":[{"name":"Started/Primary/Active","enter_time":"2020-07-2114:17:05.855923","might_have_unfound":[],"recovery_progress":{"backfill_targets":[],"waiting_on_backfill":[],"last_backfill_started":"MIN","backfill_info":{"begin":"MIN","end":"MIN","objects":[]},"peer_backfill_info":[],"backfills_in_flight":[],"recovering":[],"pg_backend":{"pull_from_peer":[],"pushing":[]}},"scrub":{"scrubber.epoch_start":"10370","scrubber.active":false,"scrubber.state":"INACTIVE","scrubber.start":"MIN","scrubber.end":"MIN","scrubber.max_end":"MIN","scrubber.subset_last_update":"0'0","scrubber.deep":false,"scrubber.waiting_on_whom":[]}},{"name":"Started","enter_time":"2020-07-2114:17:04.814061"}],"agent_state":{}}如果无法修复;两种解决方法,回滚旧版本或者直接删除5.解决方法回滚旧版本[root@k8snode001~]#cephpg2.2bmark_unfound_lostdelete[root@k8snode001~]#cephpg2.2bmark_unfound_lostdelete6.验证我这里是直接删除了,然后重建pg在ceph集群中,稍微看一下,pg状态变成active+clean[root@k8snode001~]#cephpg2.2bquery{"state":"active+clean","snap_trimq":"[]","snap_trimq_len":0,"epoch":11069,"up":[12,22,4],再次查看集群状态[root@k8snode001~]#cephhealthdetailHEALTH_OK【小编推荐】为什么有的网站是http,有的网站是https,区别是5G发展进入攻坚期,2021年这几大领域应用值得期待注意。效率直线上升!推荐5款免费好用的Windows工具。春节回家要不要做核酸分离检测?教你找出2021年值得关注的10大互联网安全工具
