Webrtc Video Simulcast & Bandwidth bitrate allocation

title WebRTC Video Simulcast

Client->PeerConnection:SetLocalDescription
PeerConnection->SdpOfferAnswerHandler:SetLocalDescription
SdpOfferAnswerHandler->SdpOfferAnswerHandler:SetLocalDescription
SdpOfferAnswerHandler->SdpOfferAnswerHandler:ApplyLocalDescription
SdpOfferAnswerHandler->SdpOfferAnswerHandler:UpdateSessionState
SdpOfferAnswerHandler->SdpOfferAnswerHandler:PushdownMediaDescription

note right of SdpOfferAnswerHandler: async call BaseChannel:SetLocalContent

SdpOfferAnswerHandler->BaseChannel:SetLocalContent
BaseChannel->VideoChannel:SetLocalContent_w
VideoChannel->BaseChannel:UpdateLocalStreams_w
BaseChannel->WebRtcVideoChannel:AddSendStream(const StreamParams& sp)
WebRtcVideoChannel->WebRtcVideoSendStream:WebRtcVideoSendStream
WebRtcVideoSendStream->WebRtcVideoSendStream:CreateRtpParametersWithEncodings

 

 

worker_thread 0x0x7f978932f250 (83)#0 0x00000001027c068e in cricket::CreateRtpParametersWithEncodings(cricket::StreamParams) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/media/base/media_engine.cc:51
#1 0x0000000102cea7d4 in cricket::WebRtcVideoChannel::WebRtcVideoSendStream::WebRtcVideoSendStream(webrtc::Call*, cricket::StreamParams const&, webrtc::VideoSendStream::Config, cricket::VideoOptions const&, bool, int, absl::optional<cricket::WebRtcVideoChannel::VideoCodecSettings> const&, absl::optional<std::__1::vector<webrtc::RtpExtension, std::__1::allocator<webrtc::RtpExtension> > > const&, cricket::VideoSendParameters const&) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/media/engine/webrtc_video_engine.cc:2183
#2 0x0000000102cdb3c3 in cricket::WebRtcVideoChannel::WebRtcVideoSendStream::WebRtcVideoSendStream(webrtc::Call*, cricket::StreamParams const&, webrtc::VideoSendStream::Config, cricket::VideoOptions const&, bool, int, absl::optional<cricket::WebRtcVideoChannel::VideoCodecSettings> const&, absl::optional<std::__1::vector<webrtc::RtpExtension, std::__1::allocator<webrtc::RtpExtension> > > const&, cricket::VideoSendParameters const&) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/media/engine/webrtc_video_engine.cc:2192
#3 0x0000000102cd9d1f in cricket::WebRtcVideoChannel::AddSendStream(cricket::StreamParams const&) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/media/engine/webrtc_video_engine.cc:1440
#4 0x00000001023e419e in cricket::BaseChannel::UpdateLocalStreams_w(std::__1::vector<cricket::StreamParams, std::__1::allocator<cricket::StreamParams> > const&, webrtc::SdpType, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/channel.cc:699
#5 0x00000001023edce5 in cricket::VideoChannel::SetLocalContent_w(cricket::MediaContentDescription const*, webrtc::SdpType, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/channel.cc:1164
#6 0x00000001023f7ef8 in cricket::BaseChannel::SetLocalContent(cricket::MediaContentDescription const*, webrtc::SdpType, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*)::$_4::operator()() const at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/channel.cc:283
#7 0x00000001023f7c65 in bool rtc::FunctionView<bool ()>::CallVoidPtr<cricket::BaseChannel::SetLocalContent(cricket::MediaContentDescription const*, webrtc::SdpType, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*)::$_4>(rtc::FunctionView<bool ()>::VoidUnion) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/api/function_view.h:109
#8 0x0000000101669c6b in rtc::FunctionView<bool ()>::operator()() const at /Users/rcadmin/.conan/data/webrtc/m91.22.3.20-08121555.a47a736c16/rooms/stable/package/105876dec92f9a83153bf5fcecc0fc2571d7ba2f/include/api/function_view.h:95
#9 0x0000000101669bb9 in bool rtc::Thread::Invoke<bool, void>(rtc::Location const&, rtc::FunctionView<bool ()>)::'lambda'()::operator()() const at /Users/rcadmin/.conan/data/webrtc/m91.22.3.20-08121555.a47a736c16/rooms/stable/package/105876dec92f9a83153bf5fcecc0fc2571d7ba2f/include/rtc_base/thread.h:384
#10 0x0000000101669b95 in void rtc::FunctionView<void ()>::CallVoidPtr<bool rtc::Thread::Invoke<bool, void>(rtc::Location const&, rtc::FunctionView<bool ()>)::'lambda'()>(rtc::FunctionView<void ()>::VoidUnion) at /Users/rcadmin/.conan/data/webrtc/m91.22.3.20-08121555.a47a736c16/rooms/stable/package/105876dec92f9a83153bf5fcecc0fc2571d7ba2f/include/api/function_view.h:109
#11 0x0000000102770609 in rtc::FunctionView<void ()>::operator()() const at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/api/function_view.h:95
#12 0x000000010277054d in rtc::Thread::InvokeInternal(rtc::Location const&, rtc::FunctionView<void ()>)::FunctorMessageHandler::OnMessage(rtc::Message*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1061
#13 0x000000010277f5b0 in rtc::Thread::Send(rtc::Location const&, rtc::MessageHandler*, unsigned int, rtc::MessageData*)::$_1::operator()() at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1011
#14 0x000000010277f419 in webrtc::webrtc_new_closure_impl::ClosureTask<rtc::Thread::Send(rtc::Location const&, rtc::MessageHandler*, unsigned int, rtc::MessageData*)::$_1>::Run() at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/task_utils/to_queued_task.h:32
#15 0x000000010276c553 in rtc::Thread::QueuedTaskHandler::OnMessage(rtc::Message*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1091
#16 0x000000010276921b in rtc::Thread::Dispatch(rtc::Message*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:755
#17 0x00000001027651ef in rtc::Thread::ProcessMessages(int) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1217
#18 0x000000010276b23a in rtc::Thread::Run() at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:951
#19 0x000000010276adaa in rtc::Thread::PreRun(void*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:940
#20 0x0000000143bc2cb0 in _pthread_start ()
#21 0x0000000143bcacdf in thread_start ()
signaling_thread 0x0x7f978932f3e0 (84)#0 0x00007ff806390506 in __psynch_cvwait ()
#1 0x0000000143bc6060 in _pthread_cond_wait ()
#2 0x00000001026cdbe0 in rtc::Event::Wait(int, int)::$_0::operator()(absl::optional<timespec>) const at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/event.cc:165
#3 0x00000001026cd896 in rtc::Event::Wait(int, int) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/event.cc:181
#4 0x000000010274d03b in rtc::NullSocketServer::Wait(int, bool) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/null_socket_server.cc:24
#5 0x000000010276bb10 in rtc::Thread::Send(rtc::Location const&, rtc::MessageHandler*, unsigned int, rtc::MessageData*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1027
#6 0x000000010276c28a in rtc::Thread::InvokeInternal(rtc::Location const&, rtc::FunctionView<void ()>) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1067
#7 0x0000000101663db2 in bool rtc::Thread::Invoke<bool, void>(rtc::Location const&, rtc::FunctionView<bool ()>) at /Users/rcadmin/.conan/data/webrtc/m91.22.3.20-08121555.a47a736c16/rooms/stable/package/105876dec92f9a83153bf5fcecc0fc2571d7ba2f/include/rtc_base/thread.h:384
#8 0x00000001023de04f in bool cricket::BaseChannel::InvokeOnWorker<bool>(rtc::Location const&, rtc::FunctionView<bool ()>) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/channel.h:299
#9 0x00000001023ddfbc in cricket::BaseChannel::SetLocalContent(cricket::MediaContentDescription const*, webrtc::SdpType, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/channel.cc:281
#10 0x0000000102306cb1 in webrtc::SdpOfferAnswerHandler::PushdownMediaDescription(webrtc::SdpType, cricket::ContentSource) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/sdp_offer_answer.cc:4279
#11 0x00000001022e8079 in webrtc::SdpOfferAnswerHandler::UpdateSessionState(webrtc::SdpType, cricket::ContentSource, cricket::SessionDescription const*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/sdp_offer_answer.cc:2526
#12 0x00000001022e20b3 in webrtc::SdpOfferAnswerHandler::ApplyLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/sdp_offer_answer.cc:1384
#13 0x00000001022f1a53 in webrtc::SdpOfferAnswerHandler::DoSetLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/sdp_offer_answer.cc:1943
#14 0x0000000102335ebf in webrtc::SdpOfferAnswerHandler::SetLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>)::$_5::operator()(std::__1::function<void ()>) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/sdp_offer_answer.cc:1186
#15 0x0000000102335c45 in rtc::rtc_operations_chain_internal::OperationWithFunctor<webrtc::SdpOfferAnswerHandler::SetLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>)::$_5>::Run() at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/operations_chain.h:71
#16 0x00000001022e04c0 in void rtc::OperationsChain::ChainOperation<webrtc::SdpOfferAnswerHandler::SetLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>)::$_5>(webrtc::SdpOfferAnswerHandler::SetLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>)::$_5&&) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/operations_chain.h:154
#17 0x00000001022e0141 in webrtc::SdpOfferAnswerHandler::SetLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/sdp_offer_answer.cc:1174
#18 0x00000001021e123c in webrtc::PeerConnection::SetLocalDescription(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/pc/peer_connection.cc:1353
#19 0x0000000102274afe in void webrtc::ReturnType<void>::Invoke<webrtc::PeerConnectionInterface, void (webrtc::PeerConnectionInterface::*)(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>), std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface> >(webrtc::PeerConnectionInterface*, void (webrtc::PeerConnectionInterface::*)(std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>), std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >&&, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface>&&) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/api/proxy.h:102
#20 0x0000000102274a29 in void webrtc::MethodCall<webrtc::PeerConnectionInterface, void, std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface> >::Invoke<0ul, 1ul>(std::__1::integer_sequence<unsigned long, 0ul, 1ul>) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/api/proxy.h:140
#21 0x00000001022747e9 in webrtc::MethodCall<webrtc::PeerConnectionInterface, void, std::__1::unique_ptr<webrtc::SessionDescriptionInterface, std::__1::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetLocalDescriptionObserverInterface> >::Run() at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/api/proxy.h:133
#22 0x000000010276c553 in rtc::Thread::QueuedTaskHandler::OnMessage(rtc::Message*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1091
#23 0x000000010276921b in rtc::Thread::Dispatch(rtc::Message*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:755
#24 0x00000001027651ef in rtc::Thread::ProcessMessages(int) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:1217
#25 0x000000010276b23a in rtc::Thread::Run() at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:951
#26 0x000000010276adaa in rtc::Thread::PreRun(void*) at /Users/andy.d.hu/git/webrtc-build/apple/macos/src/rtc_base/thread.cc:940
#27 0x0000000143bc2cb0 in _pthread_start ()
#28 0x0000000143bcacdf in thread_start ()

 

 

 

2. Bandwidth bitrate allocation

Video Encoder Bitrate Allocation

BWE->BitrateAllocator:OnNetworkEstimateChanged(TargetTransferRate)
BitrateAllocator->VideoStreamEncoder:OnBitrateUpdated(BitrateAllocationUpdate)
VideoStreamEncoder->VideoStreamEncoder:SetEncoderRates(UpdateBitrateAllocation)
VideoStreamEncoder->VideoStreamEncoder:UpdateBitrateAllocation()

note right of VideoBitrateAllocator: new_allocation
VideoStreamEncoder->VideoBitrateAllocator:Allocate()

note right of VideoBitrateAllocator: adjusted_allocation
VideoStreamEncoder->EncoderBitrateAdjuster:AdjustRateAllocation()