浣滆€呭唹灏忛緳-鑵捐浜戜腑闂翠欢鍥㈤槦鐮斿彂宸ョ▼甯堬紝ApachePulsarcommitter锛孯oP浣滆€咃紝Maintainer锛孉pacheBookKeeper璐＄尞鑰咃紝浣滆€匒pachePulsarGo瀹㈡埛绔€丄pachePulsarGoFunctions銆丼treamNative/pulsarctl銆傛憳瑕佹棩鍓嶏紝鑵捐浜戜腑闂翠欢鍥㈤槦鍜孲treamNative鍥㈤槦姝ｅ紡鍙戝竷浜哛oP0.2.0鐗堟湰銆傝繖涓増鏈湪鏋舵瀯涓婃湁浜嗘柊鐨勫崌绾с€傜敤鎴峰彲浠ュ畬鍏ㄩ伩鍏嶆秷鎭涪澶便€佹秷鎭噸澶嶆秷璐癸紝鍙秷璐归儴鍒哖artition鏁版嵁銆傞棶棰樸€俁oP鐨勫畾涔夌被浼间簬KoP銆丮oP鍜孉oP锛孯oP鏄竴绉嶅彲鎻掓嫈鐨勫崗璁鐞嗘彃浠躲€傚湪鐜版湁鐨凱ulsar闆嗙兢涓坊鍔燫oP鍗忚澶勭悊鎻掍欢鍚庯紝鐢ㄦ埛鏃犻渶淇敼浠ｇ爜鍗冲彲灏嗙幇鏈夌殑RocketMQ搴旂敤鍜屾湇鍔¤縼绉诲埌Pulsar涓婏紝杩樺彲浠ヤ娇鐢≒ulsar寮哄ぇ鐨勭壒鎬э紝濡傦細璁＄畻瀛樺偍鍒嗙澶氱鎴疯法鍦板煙澶嶅埗鍒嗗眰鍒嗙墖杞婚噺绾ц绠楁鏋垛€斺€擯ulsarFunctions...鍙戝竷RoP0.2.02021骞?鏈?7鏃ワ紝鑵捐浜戜腑闂翠欢鍥㈤槦鍚戠ぞ鍖鸿础鐚簡RoP0.1.0beta鐗堬紝RoP锛圧ocketMQonPulsar)灏哛ocketMQ鍗忚澶勭悊鎻掍欢寮曞叆PulsarBroker锛屼娇Pulsar鍙互鏀寔鍘熺敓鐨凴ocketMQ鍗忚锛孯ocketMQ鐢ㄦ埛鍙互鏃犵紳杩佺Щ鍒癆pachePulsar銆備粖澶╋紝鎴戜滑鍙戝竷浜哛oP0.2.0锛岃繖鏄竴涓叏鏂扮殑鏋舵瀯鐗堟湰锛屽湪鍔熻兘鍜岀ǔ瀹氭€т笂鏈変簡寰堝ぇ鐨勬彁鍗囥€傛彁渚汚CL璁よ瘉楠岃瘉鍔熻兘锛屽彲浠ユ洿濂界殑淇濊瘉鐢ㄦ埛鏁版嵁鐨勫畨鍏ㄣ€傚悓鏃跺厑璁哥敤鎴锋墿灞昉artitionedTopic鐨勫閲忥紝鍙互鑾峰緱鏇村ソ鐨勫苟鍙戝啓鍏ヨ兘鍔涳紝瀹屽杽RocketMQ鍘熺敓鎺у埗鎺ュ彛銆傚彲浠ユ洿濂藉湴澶勭悊鍜岀洃鎺ф湇鍔°€傛渶鏂板姛鑳戒紭鍖栧湪0.2.0鐗堟湰涓紝鑵捐浜戜腑闂翠欢鍥㈤槦鍦?.1.0鏋舵瀯鐨勫熀纭€涓婅繘琛屼簡鍏ㄦ柊璁捐锛岄噸鏋勪簡MessageID鍜屾秷鎭矾鐢辨ā鍨嬶紝淇濊瘉浜嗕笉鍚屽満鏅笅RoP娑堟伅鐨勫噯纭€с€備富瑕佷紭鍖栨湁浠ヤ笅涓夌偣锛?.鏀寔RoPACL鍔熻兘ACL鏈哄埗鏄疪ocketMQ绀惧尯鑷缓鐨勮兘鍔涳紝鍙互寰堝ソ鐨勫鐢ㄦ埛鏁版嵁杩涜閴存潈璁よ瘉銆俁oP0.2.0鐗堟湰澶嶇敤浜哛ocketMQ鑷韩鐨凥ook瀹炵幇锛屽苟浣跨敤Pulsar鑷韩鐨勯壌鏉冩満鍒跺疄鐜颁簡瀵圭敤鎴锋暟鎹繘琛岄壌鏉冭璇佺殑鍔熻兘銆俁oPACL鐨勭敤娉曡繕鏄欢缁簡RocketMQ鐨勭敤娉曪紝鍙槸瀹氫箟浜咥CL_ACCESS_KEY鍜孉CL_SECRET_KEY瀛楁锛岀劧鍚庝娇鐢≧ocketMQ鐨凙CLRPCHook鍑芥暟鍔犺浇锛屽彲浠ヤ繚璇佺敤鎴峰敖鍙兘灏戞敼鍔ㄥ鎴风鐨勪笟鍔′唬鐮侀€昏緫銆傚叿浣撲唬鐮佺ず渚嬪涓嬶細privatestaticfinalStringACL_ACCESS_KEY="eyJrZXlJZCI6InJvY2tldG1xLW13bmI3bWFwMjhqZSIsImFsZyI6IkhTMjU2In0."+"eyJzdWIiOiJyb2NrZXRtcS1td25iN21hcDI4amVfdGVzdCJ9.BDOjqqY25a6apnZTMZCqg0I0pxVFcqz7fvZbaTqkf5U";//tokenprivatestaticfinalStringACL_SECRET_KEY="rop";publicstaticvoidproducer()throwsMQClientException{DefaultMQProducerproducer=newDefaultMQProducer("rocketmq-mwnb7map28je|nit","ProducerGroupName",getAclRPCHook());...}staticRPCHookgetAclRPCHook(){returnnewAclClientRPCHook(newSessionCredentials(ACL_ACCESS_KEY,ACL_SECRET_KEY));}ACL_ACCESS_KEY绾у埆锛氱敤鎴稵oken鍒涘缓鐨凬amespace銆侫CL_SECRET_KEY锛氬浐瀹氬€硷紝鍦≧oP鍐呴儴瑙ｆ瀽鏃朵笉浼氫娇鐢ㄨ瀛楁銆?銆侀噸鏋凪essageIDRocketMQ锛岀被浼间簬Kafka锛屼娇鐢?4浣嶇殑Offset鏉ュ敮涓€鏍囪瘑涓€鏉℃秷鎭紝鑰屽湪Pulsar涓紝浣跨敤64浣嶇殑LedgerID鍜?4浣嶇殑EntryID鏉ュ敮涓€鏍囪瘑涓€鏉℃秷鎭€備负浜嗚В鍐宠繖涓棶棰橈紝鍦≧oP0.1.0涓紝鎴戜滑浣跨敤濡備笅褰㈠紡鏋勯€燤essageID瀵硅薄锛歅artitionID锛?浣嶏紝涓€涓猼opic鏈€澶氬彲浠ュ垱寤?56涓狿artitionsLedgerID锛?2浣岴ntryID锛?4浣嶅彲鑳芥湁MessageID浣跨敤涓婇潰鐨勬柟娉曚細涓㈠け娑堟伅鐨勫噯纭€э紝骞朵笖鍦ㄧ郴缁熻繍琛屼竴娈垫椂闂村悗锛屾棤娉曠户缁垱寤烘柊鐨凩edgerID锛屽鑷存暣涓泦缇ょ殑鏈嶅姟涓嶅彲鐢ㄧ殑鎯呭喌澶栭潰鐨勪笘鐣屻€傝繖涓棶棰樺拰鏇存棭鐨凨oP鐗堟湰鎵€闈复鐨勫洶澧冩槸涓€鏍风殑锛屾墍浠ュ湪RoP0.2.0涓紝鎴戜滑閲囩敤浜嗗拰KoP涓€鏍风殑澶勭悊鏂瑰紡锛屼娇鐢≒IP70鐨勫鐞嗘€濊矾锛氬紩鍏ヨ交閲忕骇broker鍏ュ彛鍏冩暟鎹紝鍦˙roker鍗忚涓環eader锛屾坊鍔犱簡涓€涓?4浣嶇殑index/publish-time瀛楁锛岃繖鏍峰氨鍙互鍦ㄥ鎴风涓嶅垎鏋愬崗璁殑鎯呭喌涓嬶紝缁欐瘡鏉℃秷鎭坊鍔犱竴涓?4浣嶇殑瀛楁銆侾IP-70鏄彃浠跺姞杞界殑锛屾墍浠ュ湪鏈嶅姟鍚姩鐨勬椂鍊欙紝鎴戜滑闇€瑕佸仛濡備笅閰嶇疆锛歜rokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor娉細BrokerEntryMetadata鍦≒ulsar2.8.0鐗堟湰鏄彧鏀寔锛屾墍浠ラ渶瑕佷繚璇丳ulsarBroker鐨勭増鏈负2.8.0鍙婁互涓娿€傞渶瑕佹敞鎰忕殑鏄紝RocketMQ鍜孠afka鍦∣ffset鐨勪娇鐢ㄤ笂鏈夋墍涓嶅悓銆俁ocketMQ涓湁涓ょOffset锛屼竴绉嶆槸QueueOffset锛岀敤鏉ヨ〃绀烘秷鎭湪MessageQueue涓殑浣嶇疆銆侻essageQueue鏈川涓婃槸涓€涓暟缁勶紝褰撴湁娑堟伅杩涙潵鏃讹紝鏁扮粍涓嬫爣+1銆備竴绉嶆槸CommitLogOffset锛岀敤浜庢寚绀烘秷鎭湪CommitLog涓瓨鍌ㄧ殑浣嶇疆銆傛秷鎭瓨鍌ㄧ敱ConsumeQueue鍜孋ommitLog閰嶅悎瀹屾垚銆侰onsumeQueue鏄竴涓€昏緫闃熷垪锛孋ommitLog瀹為檯鏄瓨鏀炬秷鎭枃浠讹紝ConsumeQueue瀛樻斁鐨勬槸鎸囧悜鐗╃悊瀛樺偍鐨勫湴鍧€銆俆opic涓嬬殑姣忎釜MessageQueue閮芥湁瀵瑰簲鐨凜onsumeQueue鏂囦欢锛屽唴瀹逛篃浼氭寔涔呭寲鍒扮鐩樸€傚洜姝わ紝鍦∕essageID閲嶆瀯鐨勫疄鐜颁笂锛屼笉鍍廗afka锛屽彧鏈変竴涓叏灞€鐨凮ffset鏉ユ爣璇嗘秷鎭殑鍞竴鎬с€傚湪RoP涓紝杩欎袱绉峅ffset鎯呭喌闇€瑕佸垎鍒鐞嗭紝濡備笅锛歊ESERVED_BITS锛?瀛楄妭淇濈暀浣嶏紝閬垮厤棣栧瓧鑺備负璐熸暟瀵艰嚧Offset璁＄畻閿欒銆俁ETRY_TOPIC_TAG_BITS锛?瀛楄妭tag锛岀敤浜庢爣璇員opic鏄惁涓篟etry绫诲瀷TopicPULSAR_PARTITION_ID_BITS锛?0瀛楄妭PartitionNum锛岀敤浜庤褰曚竴涓狿artitionedTopic涓嬫湁澶氬皯涓狿artition锛屾渶澶氭敮鎸?024涓狿artition銆侽FFSET_BITS锛?2瀛楄妭锛岀敤浜庢爣璇嗘秷鎭殑Offset銆?銆佹秷鎭矾鐢辨ā鍨嬬殑閲嶆瀯鍦≧oP0.1.0鐗堟湰涓紝鍦ㄦ秷鎭矾鐢辩殑瀹炵幇涓紝RocketMQ鍜孭ulsar閮芥槸鍏堥€氳繃Topic鎼滅储鎿嶄綔鎵惧埌瀵瑰簲鐨凮wnerBroker鑺傜偣锛岀劧鍚庤繑鍥濨roker鐨勫湴鍧€銆備絾鏄湪杩欎釜鍔ㄤ綔涓紝蹇界暐浜嗕竴涓噸瑕佺殑闂锛岄偅灏辨槸RocketMQ涓嶅悓浜嶬afka鍜孭ulsar锛屽畠鐨凲ueue涓嶆槸鍏ㄥ眬鍞竴鐨勩€俁ocketMQ璺敱鍗忚涓昏鍖呮嫭涓ら儴鍒嗭細Broker鏈嶅姟鐨処P鍦板潃淇℃伅锛汢roker涓婂搴旂殑Topic鍒嗗尯鎬绘暟鍜屽垎鍖虹殑鍙鍐欎俊鎭€傚湪RocketMQ璺敱鍗忚涓紝娌℃湁鍏ㄥ眬鏍囪瘑Topic鍒嗗尯鐨勫敮涓€ID锛堟瘮濡傚湪Pulsar/Kafka涓紝partitionID鍦ㄩ泦缇や腑鏄敮涓€鐨勶級锛涜€屽湪RocketMQ涓紝鍒嗗尯璺敱淇℃伅鏄敱BrokerID鍔犱笂Broker涓婄殑ID鍐冲畾鐨勩€侷ndex浠?鈫扤鐨勯『搴忔爣璇嗕簡topic鐨勫垎鍖恒€傚洜姝わ紝鍦≧ocketMQ鍗忚涓紝瀹㈡埛绔彧闇€瑕佽幏鍙朤opic瀵瑰簲鐨凚roker涓婄殑鍒嗗尯鎬绘暟锛岀劧鍚庨€氳繃璁＄畻灏卞彲浠ュ緱鍒癇roker涓婄殑鍒嗗尯ID锛涙墍鏈夎姹傞兘鏄熀浜嶽Broker-Tag]+[Broker-Topic-Seq]鍞竴璺敱鏌ヨ鍘熻鏉ヨ姹傛湇鍔°€傜畝鍗曟潵璇达細RocketMQ鐨刾artition鏄湁鐘舵€佺殑锛屽畠鏄粦瀹氬埌鐗瑰畾鐨凚roker涓婄殑锛涗竴涓猵artition涓€鏃﹀垎閰嶇粰浜嗕竴涓狟roker锛屽氨鍜屽畠缁堢敓鐩稿叧锛屼笉鑳借縼绉汇€傚鎴风閫氳繃璁＄畻瑙ｆ瀽鍒嗗尯璺敱淇℃伅锛涗緥濡傦細涓€涓猅opicA鏈?涓垎鍖猴紝鍒嗗埆钀藉湪BrokerA鍜孊rokerB涓婏紝BrokerA鏈?涓垎鍖猴紝BrokerB鏈?涓垎鍖猴紱鍒欏崗璁褰曚负(BrokerA,3)(BrokerB,2)锛屽鎴风閫氳繃璁＄畻寰楀埌鎵€鏈夊垎鍖烘暟鎹細BrokerA-TopicA-0,BrokerA-TopicA-1,BrokerA-TopicA-2BrokerB-TopicA-0,BrokerB-TopicA-1鐢变簬涓婇潰鐨勮矾鐢卞叧绯伙紝鎵€浠ユ病鍔炴硶浣跨敤GET_ROUTEINTO_BY_TOPIC鍗忚璇锋眰涓嶱ulsar鐨勬悳绱㈠崗璁繘琛屾槧灏勩€傛湰璐ㄥ師鍥犳槸鍜孠afka/Pulsar涓€鏍凤紝瀹冪殑Partition淇℃伅鏄叏灞€鍞竴鐨勩€傛墽琛孴opic璺敱绛栫暐鍚庯紝鍙互鍑嗙‘杩斿洖涓€涓猅opicPartition瀵瑰簲鐨凮wnerBroker鏄皝銆備絾鏄疪ocketMQ鐨凾opic璺敱杩斿洖涓や釜瀛楁锛屼竴涓槸BrokerName锛屼竴涓槸Queue鐨勪釜鏁般€傚叿浣撶殑QueueID鏄浐瀹氱殑锛岀敱Client鏍规嵁Broker杩斿洖鐨勬暟瀛椾粠0寮€濮嬮€掑銆傚洜姝ゅ湪Topic鐨勮矾鐢辨槧灏勪腑锛孯ocketMQ鍜孭ulsar鑷韩鐨勮矾鐢卞崗璁笉鑳戒竴涓€瀵瑰簲銆備负浜嗚В鍐宠繖涓棶棰橈紝鍦≧oP0.2.0涓紝鎶借薄浜嗕竴灞侾roxy鏉ョ淮鎶opic鍜孊roker鐨勬槧灏勫叧绯汇€備负浜嗗疄鐜拌繖涓洰鏍囷紝涓昏鏈変互涓嬪嚑鐐归渶瑕佽€冭檻锛氳繖浜涙槧灏勫叧绯诲瓨鏀惧湪鍝噷锛熻矾鐢卞叧绯绘槸濡備綍鍒嗛厤鐨勶紵璺敱鍏崇郴鍙戠敓鍙樺寲鎬庝箞鍔烇紵瀵逛簬绗竴涓棶棰橈紝缁忚繃缁煎悎鑰冭檻锛屾垜浠€夋嫨灏嗚矾鐢辨槧灏勫叧绯诲瓨鍌ㄥ湪ZooKeeper闆嗙兢涓紝鍥犱负鐩墠鐨凴oP鏈嶅姟鏈韩涔熼渶瑕佷緷璧朲ooKeeper闆嗙兢锛屼笉浼氬紩鍏ユ柊鐨勭粍浠讹紱鍏舵锛孼ooKeeper鑷韩鐨勪竴鑷存€ц兘鍔涜兘澶熷緢濂界殑婊¤冻杩欎釜鍦烘櫙鐨勯渶姹傘€傚浜庣浜屼釜闂锛屾垜浠湪RoP鎺ュ彛涓婂垱寤哄垎鍖簍opic鐨勫悓鏃讹紝渚濇鎼滅储姣忎釜partition鎵€鍦ㄧ殑Broker鑺傜偣锛屾牴鎹垵濮媡opic鎵€鍦ㄨ妭鐐圭殑淇℃伅灏嗘槧灏勫叧绯诲啓鍏ooKeeper闆嗙兢涓?杩欐牱鍋氱殑濂藉鏄鐢ㄤ簡Pulsar鑷韩鐨勫垎鍖哄垎閰嶆満鍒惰绠楀嚭鏉ョ殑缁撴灉锛屽疄鐜扮畝鍗曘€傚垵濮嬪垎閰嶅悗锛岃櫄鎷熻妭鐐瑰拰鐗╃悊鑺傜偣鍦ㄤ竴涓妭鐐逛笂锛屾€ц兘鑹ソ銆傚鏋滅粨鍚堣矾鐢卞叧绯诲啀骞宠 鑳藉姏锛屽彲浠ュ疄鐜版渶浣虫€ц兘銆傚浜庣涓変釜闂锛屾垜浠彲浠ラ€氳繃鍔犲叆Master-Slave妯″紡鏉ラ檷浣庡崟鑺傜偣鏁呴殰瀵圭郴缁熺殑褰卞搷銆俍ooKeeper鍏冩暟鎹涓嬨€傚彧闇€瑕佹坊鍔燘roker鐩稿叧淇℃伅鍗冲彲瀹炵幇鍚勮妭鐐圭浉浜掍富浠庡叧绯汇€傚綋涓昏妭鐐逛笉鍙敤鏃讹紝浠庤妭鐐瑰彲浠ョ户缁彁渚涙湇鍔°€傜敱浜庡綋鍓嶇殑Offset淇℃伅瀛樺偍鍦–ompactTopic涓紝鎵€鏈夎妭鐐瑰悓鏃惰闃咃紝鍙互淇濊瘉鍚勪釜鑺傜偣鐨勫厓鏁版嵁涓€鑷达紝瀹炵幇涓讳粠鍒囨崲銆備互涓嬫槸鍦ㄦ祴璇曠幆澧冮儴缃睷oP闆嗙兢鏃剁殑璺敱鏄犲皠鍏崇郴锛氬洜姝わ紝涓轰簡淇濊瘉RoP闆嗙兢鑳藉鏈夋洿濂界殑瀹归敊鎬э紝寤鸿鍦ㄩ儴缃睷oP闆嗙兢鏃朵娇鐢ㄥ伓鏁颁釜鑺傜偣RoP闆嗙兢銆傞€氳繃浠ヤ笅鍙傛暟閰嶇疆鍙互纭畾褰撳墠Master鑺傜偣鏈夊灏戜釜Slave鑺傜偣浣滀负鍏跺浠借妭鐐癸細RoPBrokerReplicationNum=2鍋囪鏈?涓狟roker鑺傜偣锛孯oPBrokerReplicationNum=2锛屽垯琛ㄧず姝ゆ椂鍙湁3涓狹asterBroker鑺傜偣瀵瑰鎻愪緵鏈嶅姟鏃堕棿銆備絾鏄浜嶱ulsar锛孊roker鑺傜偣鏄钩绛夌殑銆傚垱寤轰富棰樻椂锛屽彲浠ュ皢鍏跺垎閰嶇粰浠讳綍鑺傜偣銆傚洜姝わ紝瀵逛簬涓嶅湪OwnerBroker鑺傜偣涓婄殑璇锋眰锛屼細鍦≧oPProxy灞傚仛涓€灞備唬鐞嗐€傚厛鎼滅储Topic锛岀劧鍚庡皢璇锋眰杞彂缁橭wnerBroker鑺傜偣杩斿洖銆傛湭鏉ヨ鍒掍负浜嗘洿濂界殑璺佃寮€婧愬崗浣溿€佸紑婧愬叡寤虹殑鐞嗗康锛屼互涓婂姛鑳藉潎宸插洖棣堢ぞ鍖恒€傚彟澶栵紝瀵逛簬RocketMQ鍟嗕笟鐗堢殑浠绘剰寤舵椂娑堟伅鍔熻兘锛岃吘璁簯涓棿浠跺洟闃熶篃鍩轰簬Pulsar鐨勫師鐢熺壒鎬у紑鍙戜簡鐩稿叧鎻掍欢鏉ユ敮鎸併€俁oP鐨勫欢杩熸秷鎭姛鑳介櫎浜嗘敮鎸佸绾у欢杩熸秷鎭锛岃繕鍏锋湁鏀寔浠绘剰寤惰繜娑堟伅鐨勮兘鍔涖€備箣鍚庯紝鑵捐浜戜腑闂翠欢鍥㈤槦鍦ㄤ繚璇丷oP椤圭洰绋冲畾鎬х殑鍚屾椂锛屽皢缁х画寮€鍙慠oP鐩稿叧鍔熻兘锛屽娑堟伅璺熻釜銆佹秷鎭煡璇笌鍥炴函銆佺洃鎺ц兘鍔涚瓑锛岃繘涓€姝ュ畬鍠凴oP鍙婂懆杈圭敓鎬佺殑鍔熻兘.RoP椤圭洰鍦板潃锛歨ttps://github.com/streamnati...鐗瑰埆鎰熻阿鑵捐浜戜腑闂翠欢鍥㈤槦闊╅摥娉姐€佸紶姘稿崕瀵规湰鏂囨彁渚涚殑鎶€鏈粏鑺傜殑楠岃瘉鍜屾敮鎸併€傚叧娉ㄥ叕浼楀彿銆孉pachePulsar銆嶏紝鑾峰彇骞茶揣鍜屾柊闂诲姞鍏pachePulsar涓枃浜ゆ祦缇ゐ煈囸煆荤偣鍑婚摼鎺ヤ笅杞芥渶鏂扮増RoP锛?/p>
