当前位置: 首页 > 后端技术 > Python

使用docker快速搭建kafka(qbit)

时间:2023-03-26 18:15:06 Python

本节搭建kafka技术栈OS:Ubuntu20.04LTSdocker:20.10.12docker-compose:1.25.0zookeeper:3.4.6kafka:2.13-2.8.1kafkadockerimage:https://hub.docker.com/r/wurs...kafkadockermirrorgithub:https://github.com/wurstmeisterSearchkafkamirror#dockersearchkafkaNAMEDESCRIPTIONSTARSOFFICIALAUTOMATEDwurstmeister/kafkaMulti-BrokerApacheKafkaImage1469[OK]spotify/kafka包含Kafka和Zo的简单docker镜像…414[OK]sheepkiller/kafka-managerkafka-manager214[OK]kafkamanager/kafka-manager用于Kafka管理器的Docker映像145ches/kafkaApacheKafka。标记的版本。JMX。Cluster-...117[OK]listtagwget-qhttps://registry.hub.docker.com/v1/repositories/wurstmeister/kafka/tags-O-|sed-e's/[][]//g'-e's/"//g'-e's///g'|tr'}''\n'|awk-F:'{print$3}'也可以使用podman命令列出dockerimagesearch--list-tagswurstmeister/kafka--limit1000编排文件docker-compose.ymlversion:'3.5'services:zookeeper:image:wurstmeister/zookeeper:3.4.6container_name:zookeeperports:-"2181:2181"restart:unless-stoppedkafka:image:wurstmeister/kafka:2.13-2.8.1container_name:kafkaports:-"9092:9092"环境:-KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181-KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.1.46:9092-KAFKA_LISTENERS=PLAINTEXT://:9092volumes:-/var/run/docker.runs/ock:/var/rundocker.sockrestart:unless-stoppedstartkafkaandzookeeperdocker-composeup-d查看kafka和zookeeper是否正常运行本节dockerstatskafkazookeeper可视化工具技术栈OS:Ubuntu20.04LTSdocker:20.10.12docker-compose:1.25.0kafka:2.8.1kafka-map:1.2.0试过LogiKM2.6.0,我对fsetExplorer2.2(kafka工具)不满意。怀念RabbitMQ自带的WEB界面。最后选择国内的kafka-mapdocker安装kafka-mapdockerrun-d\-p8888:8080\-v/opt/kafka-map/data:/usr/local/kafka-map/data\-eDEFAULT_USERNAME=admin\-eDEFAULT_PASSWORD=admin\--namekafka-map\--restartunless-stoppeddushixiang/kafka-map:安装后最新版本用浏览器打开http://192.168.1.46:8888,在web界面访问kafkaPython,读写kafka技术栈Python:3.8.2kafka-python:2.0.2producerproducer.py#encoding:utf-8#author:qbit#date:2022-01-27#summary:发消息给kafkaimporttimefromkafkaimportKafkaProducerdefproducer():producer=KafkaProducer(bootstrap_servers="192.168.21.46:9092")#发送三条消息给iinrange(0,3):msg=f"*{i}*{time.strftime('%Y-%m-%d%H:%M:%S')}".encode("utf8")生产者。send(topic="kafka_test",value=msg)print(f"send:{msg}")if__name__=="__main__":producer()consumer(consumer.py)#编码:utf-8#作者:qbit#date:2022-01-27#summary:从kafka读取从kafka获取消息importKafkaConsumerdefconsumer():consumer=KafkaConsumer("kafka_test",bootstrap_servers="192.168.21.46:9092")formsginconsumer:print(f"recv:{msg.value}")if__name__=="__main__":consumer()打开两个命令行窗口,先运行consumer,再运行producerpython3consumer.python3producer.pyoutput#python3producer.pysend:b'*0*2022-01-2710:58:04'send:b'*1*2022-01-2710:58:04'send:b'*2*2022-01-2710:58:04'#python3consumer.pyrecv:b'*0*2022-01-2710:58:04'recv:b'*1*2022-01-2710:58:04'recv:b'*2*2022-01-2710:58:04'这篇文章来自来自qbitsnap