在运维过程中，很多时候，业务应用会出现假死，应用流程正常，但无法提供服务。这时候监控进程就没有意义了，所以需要监控接口。接口监控的方法有很多种。可以使用链路监控。编写监控脚本由于是整体使用prometheus进行监控，所以直接使用blackbox_exporter来监控接口。blackbox_exporter可以通过http、https、dns、tcp和ICMP检测目标。dns、tcp、ICMP比较简单。这里主要是为了监控一个登录界面，所以使用https来检测。具体配置方法如下Deployblockbox_exporter#创建blackbox_exporter的目录cd/usr/local/blackbox_exporter#下载blackbox_exporter的包wgethttps://github.com/prometheus/blackbox_exporter/releases/download/v0.19.0/blackbox_exporter-0.19.0.linux-amd64.tar.gz#解压tar-zxvfblackbox_exporter-0.19.0.linux-amd64.tar.gz#重命名mvblackbox_exporter-0.19.0。linux-amd64.tar.gzblackbox_exporter使用systemd管理blackbox_exportercat>/etc/systemd/system/blackbox_exporter.service<<"EOF"[Unit]Description=BlackboxExporterWants=network-online.targetAfter=network-online.target[Service]User=rootExecStart=/usr/local/blackbox_exporter/blackbox_exporter/blackbox_exporter--config.file=/usr/local/blackbox_exporter/blackbox_exporter/blackbox.ymlRestart=on-failure[Install]WantedBy=default.targetEOF对于blackbox_exporter的管理，有很多种方式。可以直接在后台用nohup启动，也可以通过supervisor启动。我习惯使用systemd进行管理，然后添加systemctldaemon-reloadsystemctlenableblackbox_exportersystemctlstartblackbox_exporter，因为没有指定日志输出，所以默认输出到消息日志的blackbox.yml只是最简单的配置，不能满足需求，所以添加一个modulemodules:xhj_login:#模块名，在配置blackbox.yml中prometheus配置文件中必须匹配prober:http#Protocoltimeout:30s#Timeouttimehttp:#模块获取协议方式：POST#http请求方式preferred_ip_protocol:"ip4"#Usedipv4protocolheaders:#配置post请求的header头Content-Type:application/jsonbody:''{"mobile":"13572801829","password":"ZWB123wyl"}''#后post请求参数配置好，保存blackbox.yml，启动blackbox_exportersystemctlstartblackbox_exportersystemctlstatusblackbox_exporterblackbox_exporter.service-BlackboxExporterLoaded:loaded(/etc/systemd/system/blackbox_exporter.service;enabled;vendorpreset:disabled)Active:active(running)sinceTue2022-01-0421:33:28CST;6sagoMainPID:24679(blackbox_export)Tasks:7Memory:1.9MCGroup:/system。slice/blackbox_exporter.service└─24679/data/prometheus/blackbox_exporter/blackbox_exporter/blackbox_exporter--config.file=/data/prometheus/blackbox_exporter/blackbox_exporter/blackbox....Jan0421:33:28systemd[1]:StartedBlackboxExporter.Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.173Zcaller=main.go:224msg="Startingblackbox_exporter"version="(v...33d1ed0)"Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.173Zcaller=main.go:225build_context="(go=go1.16.4,user=root@2b025...2:56:44)"Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.173Zcaller=main.go:237msg="Loadedconfigfile"Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.174Zcaller=main.go:385msg="Listeningonaddress"address=:9115Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.174Zcaller=tls_config.go:191msg="TLSisdisabled."http2=falseHint：有些行被省略了，use-ltoshowinfull。配置prometheus#blackbox-job_name:"blackbox"metrics_path:/probeparams:module:[xhj_login]static_configs:-targets:-https://xxx.aaa.com/api/pc/user/login/passwordrelabel_configs:-source_labels:[__address__]target_label:__param_target-source_labels:[_param_target]target_label:instance-target_label:__address__replacement:172.17.0.1:9115...Omit...Omit...配置完成后保存并热加载prometheus配置文件curl-XPOSThttp://localhost:9090/-/reload加载完成后，通过prometheus的UI查看targets。这时候其实可以通过被请求端的log看到blackbox_exporter发起的请求。我们也可以通过curl请求blackbox_exporter。看看#注意这里用&符号链接多个参数，需要转义curlhttp://172.17.0.1:9115/probe?target=https://xxx.aaa.com/api/pc/user/login/password\&module=xhj_login\&debug=true#查看返回结果Logsfortheprobe:ts=2022-01-04T14:10:32.979231489Zcaller=main.go:320module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Beginningprobe"probe=httptimeout_seconds=30ts=2022-01-04T14:10:32.979411891Zcaller=http.go:335module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="解析目标地址"ip_protocol=ip4ts=2022-01-04T14:10:32.986112778Zcaller=http.go:335module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Resolvedtargetaddress"ip=1.1.1.1ts=2022-01-04T14:10:32.986225541Zcaller=client.go:251module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="MakingHTTPrequest"url=https://1.1.1.1/api/pc/user/login/passwordhost=xxx.aaa.comts=2022-01-04T14:10:33.05701057Zcaller=main.go:130module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="ReceivedHTTPresponse"status_code=200ts=2022-01-04T14:10:33.057095911Zcaller=main.go:130module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Responsetimingsforroundtrip"roundtrip=0start=2022-01-04T22:10:32.986352765+08:00dnsDone=2022-01-04T22:10:32.986352765+08:00connectDone=2022-01-04T22:10:32.992658409+08:00got-03:2068273:00responseStart=2022-01-04T22:10:33.056951457+08:00tlsStart=2022-01-04T22:10:32.992701614+08:00tlsDone=2022-01-04T22:100-230:2302。01-04T22:10:33.057074505+08:00ts=2022-01-04T14:10:33.057187486Zcaller=main.go:320module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Probesucceeded"duration_seconds=0.077883376可以看到http_status_code200，probe_success为1，访问正常，添加告警，创建rulesgroups:-name:blackbox_networks_statsrules:-alert:PC登录界面失败expr:probe_success==0for:1mlabels:severity:criticalannotations:summary:"PC登录界面故障，无法正常登录，请及时检查！"description:"PC登录界面出现故障，无法正常登录，请及时检查！"创建完成后，热加载prometheus配置，然后查看prometheus在规则Grafana中添加监控图标，监控配置完成后，需要配置一个可视化dashboard，5345模板可以直接导入dashboard显示界面status，是否使用SSL，包括SSL证书过期时间、接口状态码、接口请求时间、DNS解析时间
