当前位置: 首页 > 科技观察

CentOS安装filebeat收集日志到kafka

时间:2023-03-12 02:25:33 科技观察

简介filebeat是一个日志数据收集器,可以代替logstash收集日志,部署起来更方便。一般日志量比较大,Kafka会先采集再消费。安装yum-yinstallhttps://mirrors.tuna.tsinghua.edu.cn/elasticstack/7.x/yum/7.14.0/filebeat-7.14.0-x86_64.rpm配置filebeatvim/etc/filebeat/filebeat.ymlfilebeat.inputs:-type:logpaths:-/var/log/*.log-/var/log/messagesfields:type:"systemlog"log_topic:"systemlog"fields_under_root:trueexclude_lines:["^DBG"]exclude_files:[".gz$"]name:192.168.122.100output.kafka:enabled:truehosts:["192.168.122.200:9092"]topic:"%{[log_topic]}"partition.round_robin:reachable_only:trueworker:2required_acks:1compression:gzipmax_message_bytes:1000000服务管理systemctlenablefilebeatsystemctlstartfilebeat总结filebeat是用Go写的,部署起来非常方便,无依赖,易于批量部署。