前端流媒体：MSE入门

鍓嶈█娴佸獟浣撳崗璁绉嶅鏍凤紝闊宠棰戠紪鐮佹牸寮忔洿鏄绉嶅鏍枫€傚湪娴忚鍣ㄤ腑姝ｅ父娴忚骞朵笉瀹规槗銆傞櫎浜哤ebRTC绛夋祻瑙堝櫒宸茬粡鏀寔鐨勫崗璁锛孒LS銆丗LV銆丷TSP銆丷TMP銆丏ASH绛夊崗璁兘闇€瑕佽繘琛岄澶勭悊锛屼絾娴佺▼澶ц嚧鐩稿悓锛氶€氳繃HTTP銆乄ebSocket绛夎幏鍙栨暟鎹紱澶勭悊鏁版嵁锛岃В鏋愬崗璁紝甯ц幏鍙栧獟浣撲俊鎭拰鏁版嵁锛涘皝瑁呮垚濯掍綋娈碉紝鎴栬В鐮佹垚甯э紱閫氳繃瑙嗛鎴栫敾甯?WebGL)鎾斁銆傜洰鍓嶅競闈笂鏈変竴浜涘墠绔В鐮佹柟妗堬紝姣斿璋冪敤鍏锋湁WASM楂樻€ц兘鐨刢瑙ｇ爜搴擄紝鎴栬€呯洿鎺ヤ娇鐢ㄦ祻瑙堝櫒鐨刉ebCodecsAPI杩涜缂栬В鐮佲€︹€︿絾鏄兘鏈夊眬闄愭€э紝WebCodecs浠嶇劧鏄竴涓疄楠屽姛鑳斤紱鑰學ASM鏂规铏界劧绐佺牬浜嗘祻瑙堝櫒娌欑洅闄愬埗锛堝彲浠ユ挱鏀綡265绛夋祻瑙堝櫒涓嶆敮鎸佺殑缂栫爜鏍煎紡锛夛紝浣嗕笌娴忚鍣ㄥ師瑙ｇ爜銆佸閫氶亾鎬ц兘绛夋柟闈粛鏈夊樊璺濅篃寰堝樊锛屽洜涓哄彧鑳界敤杞В鐮併€傚彈涓嶄簡銆傛墍浠ュ競闈笂鏇村鐨勬槸閲囩敤鍙︿竴绉嶆柟寮忥紝瑙ｅ喅鍗忚+灏佽+鏈枃鐨勪富瑙扢ediaSourceExtensions锛堜互涓嬬畝绉癕SE锛夈€侶TML5瑙勮寖鍏佽鎴戜滑鐩存帴鍦ㄧ綉椤典腑宓屽叆瑙嗛锛?videosrc="demo.mp4">锛屼絾鏄痵rc鎸囧畾鐨勮祫婧愬湴鍧€蹇呴』鏄竴涓畬鏁寸殑濯掍綋鏂囦欢锛屽浣曞疄鐜版祦濯掍綋璧勬簮鍦ㄧ綉娓革紵MSE鎻愪緵浜嗚繖鏍风殑鍙兘鎬э紝鍏堢湅鐪婱DN瀵瑰畠鐨勬弿杩帮細MediaSourceExtensionAPI(MSE)providestheabilitytoimplementplug-in-freeandweb-basedstreamingmedia銆備娇鐢∕SE锛屽彲浠ヤ娇鐢?audio>鍜?video>鍏冪礌鍒涘缓鍜屾挱鏀惧獟浣撴祦銆備笂闈㈡彁鍒癕SE鍏佽鎴戜滑閫氳繃JS鍒涘缓濯掍綋璧勬簮锛屼娇鐢ㄨ捣鏉ラ潪甯告柟渚匡細constmediaSource=newMediaSource();constvideo=document.querySelector('video');video.src=URL.createObjectURL(濯掍綋婧愶級锛涘垱寤哄獟浣撹祫婧愬璞″悗锛屼笅涓€姝ユ槸鍚戝叾鎻愪緵瑙嗛鏁版嵁锛堢墖娈碉級锛屼唬鐮佸涓嬶細mediaSource.addEventListener('sourceopen',()=>{constmime='video/mp4;codecs="avc1.42E01E,mp4a.40.2"';constsourceBuffer=mediaSource.addSourceBuffer(mime);constdata=newArrayBuffer([...]);//瑙嗛鏁版嵁sourceBuffer.appendBuffer(data);});姝ゆ椂锛岃棰戝皢姝ｅ父鎾斁銆傛兂瑕佸疄鐜版祦寮忔挱鏀撅紝鍙渶瑕佷笉鏂皟鐢╝ppendBuffer锛屾妸闊宠棰戞暟鎹杺杩涘幓灏卞ソ浜嗐€俢odecs="avc1.42E01E,mp4a.40.2"'杩欎釜娈靛瓧绗︿覆鏄粈涔堟剰鎬濓紵闊宠棰戞暟鎹粠浣曡€屾潵锛燄煠擬IMETYPE//webmMIME-type'video/webm;codecs="vp8,vorbis"'//fmp4MIME-type'video/mp4;codecs="avc1.42E01E,mp4a.40.2"'杩欎釜瀛楃涓叉弿杩扮浉鍏冲弬鏁拌棰戠殑灏佽鏍煎紡銆侀煶瑙嗛缂栫爜鏍煎紡绛夐噸瑕佷俊鎭€備互涓婇潰鐨刴p4閮ㄥ垎涓轰緥锛屽垎涓轰袱閮ㄥ垎锛氬墠鍗婇儴鍒嗙殑video/mp4琛ㄧず杩欐槸涓€涓猰p4鏍煎紡鐨勮棰戯紱鍚庡崐閮ㄥ垎鐨刢odecs鎻忚堪浜嗚棰戠殑缂栫爜淇℃伅锛屽畠鐢变竴涓垨澶氫釜鐢憋紝鍒嗛殧鐨勫€肩粍鎴愶紝姣忎釜鍊肩敱涓€涓垨澶氫釜鍏冪礌缁勬垚锛岀敱.鍒嗛殧锛歛vc1琛ㄧず瑙嗛鏄疉VC锛堝嵆H264锛夌紪鐮侊紱42E01E鐢变笁涓瓧鑺傜粍鎴愶紙鐢?6杩涘埗琛ㄧず锛夛紝鎻忚堪Video鐩稿叧淇℃伅锛?x42锛圓VCProfileIndication锛夎〃绀鸿棰戠殑profile锛屽父瑙佺殑鏈塀aseline/Extended/Main/Highprofile绛夛紱0xE0锛坧rofile_compatibility锛夎〃绀虹紪鐮佺骇鍒殑绾︽潫锛?x1E(AVCLevlIndication)琛ㄧずH264鐨勭骇鍒紝琛ㄧず鏈€澶ф敮鎸佸垎杈ㄧ巼銆佸抚鐜囥€佺爜鐜囩瓑锛沵p4a琛ㄧず鏌愮MPEG-4闊抽锛?0鏄疢P4娉ㄥ唽鏈烘瀯瑙勫畾鐨凮bjectTypeIndication(OTI)锛?x40瀵瑰簲AudioISO/IEC14496-3(d)鏍囧噯锛?琛ㄧず鏌愰煶棰慜TI锛宮p4a.40.2琛ㄧずAACLC銆備絾鏄紝鏈夊绉嶉煶棰戝拰瑙嗛鏍煎紡銆傚墠绔湁娌℃湁鍔炴硶鐩存帴鑾峰彇瑙嗛鐨凪IMETYPE锛焟p4鏍煎紡鍙互浣跨敤锛氿煂燄煂燄煂焟p4box馃専馃専馃専锛岃幏鍙栨柟娉曞涓嬶細//utils.ts//addlibrary//yarnaddmp4boximportMP4Boxfrom'mp4box';exportfunctiongetMimeType(buffer:ArrayBuffer){returnnewPromise((resolve,reject)=>{constmp4boxfile=MP4Box.createFile();mp4boxfile.onReady=(info:any)=>resolve(info.mime)澶嶅埗浠ｇ爜;mp4boxfile.onError=()=>reject();(bufferasany).fileStart=0;mp4boxfile.appendBuffer(buffer);});}鑾峰彇MIMETYPE鍚庯紝鍙互鏌ョ湅褰撳墠娴忚鍣ㄦ槸鍚︽敮鎸佸獟浣撴牸寮忛€氳繃MSE鐨勯潤鎬佹柟娉昅ediaSource.isTypeSupported()銆俰mport{getMimeType}from'./utils';...constmime=awaitgetMimeType(buffer);if(!MediaSource.isTypeSupported(mime)){thrownewError('mimetypenotsupported');}MediaSegmentSourceBuffer.appendBuffer(source)鏃ㄥ湪灏嗗獟浣撶墖娈垫暟鎹簮娣诲姞鍒癝ourceBuffer瀵硅薄涓紝鍙傝MDN涓妔ource鐨勬弿杩帮細涓€涓狟ufferSource(en-US)瀵硅薄锛圓rrayBufferView鎴朅rrayBuffer锛夛紝瀹冨瓨鍌ㄤ簡浣犺娣诲姞鍒癝ourceBuffer涓殑濯掍綋鐗囨鏁版嵁銆傞偅涔坰ource鏄竴涓蹭簩杩涘埗鏁版嵁锛屽綋鐒朵笉浠呬粎鏄竴涓瓧绗︿覆锛岄偅涔坢edia娈靛簲璇ユ弧瓒充粈涔堟潯浠跺憿锛熺鍚圡SEByteStreamFormatRegistry瑕佹眰鐨凪IME绫诲瀷灞炰簬InitializationSegment鎴朚ediaSegment涔嬩竴銆傚浜庣涓€涓潯浠讹紝MSE鏀寔鐨勫獟浣撴牸寮忓拰闊宠棰戞牸寮忚緝灏戯紝甯歌鐨勪竴鑸湁fmp4(h264+aac)鍜寃ebm(vp8/vorbis)绛夈€備粈涔堟槸fmp4锛熶粈涔堟槸webm锛熷ぇ瀹跺彲浠ョ偣鍑讳簡瑙ｏ紝鏈枃涓嶅睍寮€璁ㄨ銆傚浜庣浜岀鎯呭喌锛孖nitializationSegment琛ㄧず鍒濆鍖栨锛屽寘鍚玀ediaSegment瑙ｇ爜鎵€闇€鐨勫垵濮嬪寲淇℃伅锛屽濯掍綋鍒嗚鲸鐜囥€佹椂闀裤€佹瘮鐗圭巼绛変俊鎭紱MediaSegment鏄甫鏈夋椂闂存埑鐨勯煶棰戝拰瑙嗛鐗囨锛屽苟涓斾笌鏈€杩戞坊鍔犵殑InitializationSegment鐩稿叧鑱斻€傞€氬父锛屽湪闄勫姞鍒濆鍖栫墖娈靛悗闄勫姞澶氫釜濯掍綋鐗囨銆傚浜巉mp4锛屽垵濮嬪寲娈靛拰濯掍綋娈靛叾瀹為兘鏄疢P4鐩掑瓙锛屽彧鏄被鍨嬩笉鍚岋紙浜嗚В鏇村锛夛紱瀵逛簬webm锛屽垵濮嬪寲娈垫槸EBMLHeader鍜孋luster鍏冪礌涔嬪墠鐨勫唴瀹癸紙鍖呮嫭涓€浜沵edia锛宼rack绛夛級锛宮ediafragment鏄竴涓狢luster鍏冪礌锛堜簡瑙ｆ洿澶氾級銆傞亾鐞嗘垜閮芥噦锛屼絾鏄疄璺靛憿锛熷浣曚粠鐜版湁濯掍綋鏂囦欢鐢熸垚涓婅堪濯掍綋鐗囨锛熻繖閲屾垜浠娇鐢ㄧ殑鏄煂燄煂燄煂烣Fmpeg馃専馃専馃専锛屼娇鐢ㄨ捣鏉ラ潪甯告柟渚匡紝鍙渶瑕佷竴琛屽懡浠わ細ffmpeg-ixxx-ccopy-fdashindex.mpdxxx鏄綘鏈湴鐨勫獟浣撴枃浠讹紝鎴戣繖閲岀敤鐨勬槸lol.mp4鍜宐ig-buck-bunny.webm涓や釜鏂囦欢鐢ㄤ簬娴嬭瘯锛氿煈塮fmpeg-ilol.mp4-ccopy-fdashindex.mpd馃憞馃憠ffmpeg-ibig-buck-bunny.webm-ccopy-fdashindex.mpd馃憞浠庢祴璇曠粨鏋滃彲浠ョ湅鍑猴紝init-xxx.xx鍜宑hunk-xxx-xxx.xx鐨勬枃浠堕兘鐢熸垚浜嗐€傚緢鏄庢樉init-xxx.xx浠ｈ〃鍒濆鍖栫墖娈碉紝chunk-xxx-xxx.xx浠ｈ〃Media鐗囨锛屽叾涓璼tream0鍜宻tream1鍒嗗埆浠ｈ〃瑙嗛鍜岄煶棰戦€氶亾銆傚€熷姪鍦ㄧ嚎mp4box瑙ｆ瀽宸ュ叿锛屾煡鐪媐mp4鍒濆鍖栫墖娈靛拰media鐗囨鐨勫唴閮ㄧ粨鏋勶細涓嶪SOBMFF鎻忚堪涓€鑷达紝鍒濆鍖栫墖娈电敱ftypbox+moovbox缁勬垚锛沵ediafragmentsstypbox,sidxbox,moofbox,mdatbox锛屾兂鐭ラ亾鍚勭妗嗙殑鎰忔€濆彲浠ュ幓瀛︿範MP4锛岃繖鏍风洿鎾璧锋潵鏇村帀瀹炽€傜ず渚嬸煈囸煐ュ湪绾挎紨绀吼煂梆煍梘ithub馃専