UE 像素流通信

再UE中是可以像我前后单一样进行通信的

准备

先随便建一个空白项目，按照官方文档设置UE引擎

https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/PixelStreamingIntro/

找到在虚幻引擎安装文件夹中，在 Samples/PixelStreaming/WebServers/SignallingWebServer 中找到信令服务器的路径

例如我的就是

C:\Program Files\Epic Games\UE_4.27\Samples\PixelStreaming\WebServers\SignallingWebServer

打开可以看到是一个Node 项目，既然是node 项目那其他的先不说，先安装一下依赖

在这里不做赘述，参考

https://www.cnblogs.com/makalochen/p/13768288.html

也可以直接项目跟目录执行

npm install

前提是node环境已经配好了

关于联机调试

上面官方文档是打包进行调试，但其实还有不用打包的，下面这样设置

image-20231101134944704

创建自定义UI

参考：

https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/#使用emituiinteraction函数

我们在项目根目录创建

image-20231101143004119

myUI.html

<!DOCTYPE html>
<html>

<head>
    <!-- 该文件处理浏览器和虚幻引擎应用间的通信，接受并显示来自服务器的媒体流。在非必要的情况下，请勿修改此JavaScript文件 -->
    <script type="text/javascript" src="../scripts/webRtcPlayer.js"></script>
    <!-- 此文件将设置处理键盘、鼠标和触摸事件的事件监听器 -->
    <script type="text/javascript" src="../scripts/app.js"></script>


    <link rel="shortcut icon" href="/images/favicon.ico" type="image/x-icon">
	<link rel="icon" type="image/png" sizes="96x96" href="/images/favicon-96x96.png">
	<link rel="icon" type="image/png" sizes="32x32" href="/images/favicon-32x32.png">
	<link rel="icon" type="image/png" sizes="16x16" href="/images/favicon-16x16.png">
    <link type="text/css" rel="stylesheet" href="player.css">
    <title>myUI</title>
</head>

<body onload="load()">
    <div id="playerUI">
		<div id="player"></div>
		<div id="overlay" class="overlay text-light bg-dark">
			<div>
				<div id="qualityStatus" class="greyStatus">&#9679</div>
				<div id="overlayButton">+</div>
			</div>
			<div id="overlaySettings">
				<div id="kickOthers">
					<div class="settings-text">Kick all other players</div>
					<label class="btn-overlay">
						<input type="button" id="kick-other-players-button" class="overlay-button btn-flat" value="Kick">
					</label>
				</div>
				<div id="fillWindow">
					<div class="settings-text">Enlarge Display to Fill Window</div>
					<label class="tgl-switch">
						<input type="checkbox" id="enlarge-display-to-fill-window-tgl" class="tgl tgl-flat" checked>
						<div class="tgl-slider"></div>
					</label>
				</div>
				<div id="qualityControlOwnership">
					<div class="settings-text">Quality control ownership</div>
					<label class="tgl-switch">
						<input type="checkbox" id="quality-control-ownership-tgl" class="tgl tgl-flat">
						<div class="tgl-slider"></div>
					</label>
				</div>
				<br>

                <!-- 编码设置 -->
				<section id="encoderSettings">
					<div class="settings-text">Encoder Settings</div>
					<div id="encoderParamsContainer" class="collapse">
						<div class="form-group">
							<label for="encoder-rate-control" class="settings-text">Rate Control</label>
							<select id="encoder-rate-control">
								<option value="CBR" selected>CBR</option>
								<option value="VBR">VBR</option>
								<option value="ConstQP">ConstQP</option>
							</select>
							<label for="encoder-target-bitrate-text">Target Bitrate (kbps)</label>
							<input type="number" class="form-control" id="encoder-target-bitrate-text" value="0" min="0" max="100000" />
							<label for="encoder-max-bitrate-text">Max Bitrate (kbps)</label>
							<input type="number" class="form-control" id="encoder-max-bitrate-text" value="0" min="0" max="100000" />
							<label for="encoder-min-qp-text">Min QP</label>
							<input type="number" class="form-control" id="encoder-min-qp-text" value="0" min="0" max="999" />
							<label for="encoder-max-qp-text">Max QP</label>
							<input type="number" class="form-control" id="encoder-max-qp-text" value="0" min="0" max="999" />
							<label for="encoder-multipass" class="settings-text">Multipass</label>
							<select id="encoder-multipass">
								<option value="DISABLED" selected>DISABLED</option>
								<option value="QUARTER">QUARTER</option>
								<option value="FULL">FULL</option>
							</select>
							<div class="settings-text">Filler Data</div>
							<label class="tgl-switch">
								<input type="checkbox" id="encoder-filler-data-tgl" class="tgl tgl-flat">
								<div class="tgl-slider"></div>
							</label>
						</div>
						<input id="encoder-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
					</div>
					<br>
				</section>

                <!-- webRTC 设置 -->
				<section id="webRTCSettings">
					<div class="settings-text">WebRTC Settings</div>
					<div id="webrtcParamsContainer" class="collapse">
						<div class="form-group">
							<label for="webrtc-degradation-pref">Degradation Pref</label>
							<select id="webrtc-degradation-pref">
								<option value="BALANCED">BALANCED</option>
								<option value="MAINTAIN_FRAMERATE">MAINTAIN_FRAMERATE</option>
								<option value="MAINTAIN_RESOLUTION">MAINTAIN_RESOLUTION</option>
							</select>
							<label for="webrtc-max-fps-text">Max FPS</label>
							<input type="number" class="form-control" id="webrtc-max-fps-text" value="1" min="1" max="999" />
							<label for="webrtc-min-bitrate-text">Min Bitrate (kbps)</label>
							<input type="number" class="form-control" id="webrtc-min-bitrate-text" value="0" min="0" max="100000" />
							<label for="webrtc-max-bitrate-text">Max Bitrate (kbps)</label>
							<input type="number" class="form-control" id="webrtc-max-bitrate-text" value="0" min="0" max="100000" />
							<label for="webrtc-low-qp-text">Low QP Threshold</label>
							<input type="number" class="form-control" id="webrtc-low-qp-text" value="0" min="0" max="999" />
							<label for="webrtc-high-qp-text">High QP Threshold</label>
							<input type="number" class="form-control" id="webrtc-high-qp-text" value="0" min="0" max="999" />
						</div>
						<input id="webrtc-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
					</div>
				</section>
				<br>

                <!-- 显示FPS -->
				<div id="showFPS">
					<div class="settings-text">Show FPS</div>
					<label class="btn-overlay">
						<input type="button" id="show-fps-button" class="overlay-button btn-flat" value="Toggle">
					</label>
				</div>
				<div id="matchViewportResolution">
					<div class="settings-text">Match Viewport Resolution</div>
					<label class="tgl-switch">
						<input type="checkbox" id="match-viewport-res-tgl" class="tgl tgl-flat">
						<div class="tgl-slider"></div>
					</label>
				</div>

				<div id="statsPanel">
					<div class="settings-text">Show Stats</div>
					<label class="tgl-switch">
						<input type="checkbox" id="show-stats-tgl" class="tgl tgl-flat" checked>
						<div class="tgl-slider"></div>
					</label>
					<div id="statsContainer" class="statsContainer">
						<div id="stats" class="stats"></div>
					</div>
					<br>
				</div>
				
				<div id="latencyTest">
					<div class="settings-text">Latency Stats</div>
					<label class="btn-overlay">
						<input type="button" id="test-latency-button" class="overlay-button btn-flat" value="Test Latency">
					</label>
					<div id="latencyStatsContainer" class="statsContainer">
						<div id=LatencyStats class="stats">No stats yet...</div>
					</div>
				</div>
			</div>
		</div>
	</div>
</body>
</html>

player.css

/*Copyright Epic Games, Inc. All Rights Reserved.*/

:root {
	/*Using colour scheme https://color.adobe.com/TD-Colors---Option-3-color-theme-10394433/*/
	--colour1:#2B3A42;
	--colour2:#3F5765;
	--colour3:#BDD4DE;
	--colour4:#EFEFEF;
	--colour5:#FF5035;
	
	--buttonFont:Helvetica;
	--inputFont:Helvetica;
}

body{
    margin: 0px;
    background-color: black;
}	

#playerUI {
	width: 100%;
	position: absolute;
	/*top: 0;
	left: 0;*/
	z-index: 10;
}

.statsContainer {
	background-color: rgba(0,0,0,0.8);
	text-align: left;
	display: block;
	margin-top: 5px;
}

.stats {
	font-size: 14px;
	font-weight: bold;
	padding: 6px;
	color: lime;
}

canvas{
    image-rendering: crisp-edges;
    position: absolute;
}

video{
	position: absolute;
	width: 100%;
	height: 100%;
}

#player{
	width: 1280px;
	height: 720px;
	position: relative;
	background-color: #000;
}

#overlay{
	-moz-border-radius-bottomright: 5px;
	-moz-border-radius-bottomleft: 5px;
	-webkit-border-bottom-right-radius: 5px;
	-webkit-border-bottom-left-radius: 5px;
	border-bottom-right-radius: 5px; /* future proofing */
	border-bottom-left-radius: 5px; /* future proofing */
	-khtml-border-bottom-right-radius: 5px; /* for old Konqueror browsers */
	-khtml-border-bottom-left-radius: 5px; /* for old Konqueror browsers */
	
	-webkit-touch-callout: none; /* iOS Safari */
    -webkit-user-select: none; /* Safari */
     -khtml-user-select: none; /* Konqueror HTML */
       -moz-user-select: none; /* Firefox */
        -ms-user-select: none; /* Internet Explorer/Edge */
            user-select: none; /* Non-prefixed version, currently
                                  supported by Chrome and Opera */
	
	position: absolute;
	padding: 4px;
	top: 0;
	right: 2%;
	z-index: 100;
	border: 2px solid var(--colour4);
	border-top-width: 0px;
}

.overlay {
    background-color: var(--colour2);
	font-family: var(--buttonFont);
	font-weight: lighter;
	color: var(--colour4);
}

.overlay-shown > #overlaySettings {
	padding: 50px 4px 4px 4px;
	display: block;
}

.overlay-shown > div > #overlayButton {
	transform: rotate(-135deg);
	-webkit-transform: rotate(-135deg); /* Safari */
	-moz-transform: rotate(-135deg); /* Firefox */
	-ms-transform: rotate(-135deg); /* IE */
	-o-transform: rotate(-135deg); /* Opera */
}

#overlayButton:hover{
	cursor: pointer;
}

#overlayButton{
	transition-duration: 250ms;
	float: right;
	text-align: right;
	font-size: 40px;
}

#qualityStatus{
	float: left;
	font-size: 37px;
	padding-right: 4px;
}

#overlaySettings{
	width: 400px;
	display: none;
}

.greyStatus {
	color: grey;
}

.limeStatus {
	color: lime;
}

.orangeStatus {
	color: orange;
}

.redStatus {
	color: red;
}

#videoMessageOverlay{
	z-index: 20;
	color: var(--colour4);;
	font-size: 1.8em;
	position: absolute;
	margin: auto;
	font-family: var(--inputFont);;
	width: 100%;
}

#videoPlayOverlay{
	z-index: 30;
	position: absolute;
	color: var(--colour4);
	font-size: 1.8em;
	font-family: var(--inputFont);
	width: 100%;
	height: 100%;
	background-color: rgba(100, 100, 100, 0.7);
}

/* State for element to be clickable */
.clickableState{
	align-items: center;
	justify-content: center;
	display: flex;
	cursor: pointer;
}

/* State for element to show text, this is for informational use*/
.textDisplayState{
	display: flex;
}

/* State to hide overlay, WebRTC communication is in progress and or is playing */
.hiddenState{
	display: none;
}

#playButton{
	display: inline-block;
	height: auto;
}

img#playButton{
	max-width: 241px;
	width: 10%;
}

#UIInteraction{
	position: fixed;
}

#UIInteractionButtonBoundary{
	padding: 2px;
}

#UIInteractionButton{
	cursor: pointer;
}

#hiddenInput{
	position: absolute;
	left: -10%;   /* Although invisible, push off-screen to prevent user interaction. */
	width: 0px;
	opacity: 0;
}

#editTextButton{
	position: absolute;
	height: 40px;
	width: 40px;
}

.settings-text{
	color: var(--colour4);
	vertical-align: middle;
	font-size: 18px;
	font-weight: normal;
	display: inline-block;
}

.overlay-button{
	line-height: 1.1;
	padding: 1px 6px;
}

.btn-overlay{
	float: right;
	vertical-align: middle;
	display: inline-block;
}

.btn-flat{
	background: var(--colour4);
	border: 2px solid var(--colour5);
	font-weight: bold;
	cursor: pointer;
	font-family: var(--buttonFont);
	font-size: 10px;
	color: var(--colour5);
	border-radius: 5px;
	height: 20px;
}

.btn-flat:disabled{
	background: var(--colour4);
	border-color: var(--colour3);
	color: var(--colour3);
	cursor: default;
}

.btn-flat:active{
	border-color: var(--colour2);
	color: var(--colour2);
}

.btn-flat:focus{
	outline: none;
}
/*** Toggle Switch styles ***/
.tgl-switch {
  float: right;
  vertical-align: middle;
  display: inline-block;
}

.tgl-switch .tgl {
	display:none;
}

.tgl, .tgl:after, .tgl:before, .tgl *, .tgl *:after, .tgl *:before, .tgl + .tgl-slider {
  -webkit-box-sizing: border-box;
          box-sizing: border-box;
}
.tgl::-moz-selection, .tgl:after::-moz-selection, .tgl:before::-moz-selection, .tgl *::-moz-selection, .tgl *:after::-moz-selection, .tgl *:before::-moz-selection, .tgl + .tgl-slider::-moz-selection {
  background: none;
}
.tgl::selection, .tgl:after::selection, .tgl:before::selection, .tgl *::selection, .tgl *:after::selection, .tgl *:before::selection, .tgl + .tgl-slider::selection {
  background: none;
}

.tgl-slider {
	float:right;
}

.tgl + .tgl-slider {
  outline: 0;
  display: block;
  width: 40px;
  height: 18px;
  position: relative;
  cursor: pointer;
  -webkit-user-select: none;
     -moz-user-select: none;
      -ms-user-select: none;
          user-select: none;
}

.tgl + .tgl-slider:after, .tgl + .tgl-slider:before {
  position: relative;
  display: block;
  content: "";
  width: 50%;
  height: 100%;
}
.tgl + .tgl-slider:after {
  left: 0;
}
.tgl + .tgl-slider:before {
  display: none;
}

.tgl-flat + .tgl-slider {
  padding: 2px;
  -webkit-transition: all .2s ease;
  transition: all .2s ease;
  background: #fff;
  border: 3px solid var(--colour4);
  border-radius: 2em;
}

.tgl-flat + .tgl-slider:after {
  -webkit-transition: all .2s ease;
  transition: all .2s ease;
  background: var(--colour4);
  content: "";
  border-radius: 1em;
}

.tgl-flat:checked + .tgl-slider {
  border: 3px solid var(--colour5);
}

.tgl-flat:checked + .tgl-slider:after {
  left: 50%;
  background: var(--colour5);
}
/*** Toggle Switch styles ***/

#encoderSettings, #webRTCSettings {
	margin: 10px 0px;
}

#encoderParamsContainer, #webrtcParamsContainer {
	padding-left: 5%;
}

#encoder-params-submit {
	float: right;
}

#webrtc-params-submit {
	float: right;
}

select {
	text-align: right;
}

input {
	text-align: right;
}

.form-group {
	display: grid;
	grid-template-columns: 50% 50%;
}

.form-group label {
	color: var(--colour4);
	vertical-align: middle;
	font-size: 18px;
	font-weight: normal;
}

#latencyTest {
	display: block;
}

#latencyTest button {
	margin: 30px 0px;
}

#freezeFrameOverlay {
	background-color: transparent;
}

.freezeframeBackground {
	background-color: #000 !important;
}

更改项目首页

找到项目目录下的 config.json ,将项目首页更改成自己的，然后重新启动

image-20231101134104682

可以看到

image-20231101134254698

demo 测试

直接在浏览器输入

http://127.0.0.1/

可以看到

image-20231101134323304

点击一下，就可以看到跟UE引擎一样的，像素流

image-20231101134400128

通信

新建游戏模式

右键新建 游戏基础模式

image-20231101144349443

新建默认pawn类，为 Camera

image-20231101150741875

双击打开，拖入一个Camera 摄像机

image-20231101150904514

设置参数

image-20231101150925591

设置游戏模式

image-20231101151015577

测试启动

image-20231101151105303

启动之后，你会发送什么都动不了，这是正常的，没有写移动的逻辑

html前端发送

参考：https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/#使用emituiinteraction函数

我们在定义UI的html 页面修改一点东西,下面的代码很简单，就是添加一个按钮，调用发送函数，并且调用回调

<div>
	<button onclick="testSend()">发送消息</button>
</div>

<script>
    function api_send(proto, data , callback){
        let jsonData = { command: 'event', func_name: proto, args: data};
        emitUIInteraction(jsonData);
        callback(jsonData);
    }
    function testSend(){
        console.log('testSend...........');
        api_send("onClickUEsend", "testSend......", (info) => {
            console.log(info);
        });
    }
</script>

image-20231101160501864

UE接收蓝图编辑

双击GM 游戏模式，回到编辑界面，添加 PixelStreamerInput 事件

image-20231101151423005

image-20231101151459717

添加输入事件

image-20231101151609140

添加打印

image-20231101151700346

运行一下看下

image-20231101151746501

发送消息Demo消息测试

输入

http://127.0.0.1/

运行点击开始播放之后

image-20231101160834786

可以看到，ue 这边已经接收到了

ue 返回

前面我们已经实现了，从前端 到 ue，但是前面的那个回调函数是我们自己调用，那我们现在改造一下，这边我们发给ue 的是什么，就返回什么

前端修改

<script>
    // 发送到ue
    function api_send(proto, data , callback){
        let jsonData = { command: 'event', func_name: proto, args: data};
        // 发送
        emitUIInteraction(jsonData);
        // 调用 app.js 提供的 addResponseEventListener 函数来注册监听器函数，注册ue response 监听函数
        addResponseEventListener("handle_responses", callback)
    }
    function testSend(){
        console.log('testSend...........');
        api_send("onClickUEsend", "testSend......", (info) => {
            console.log(info);
        });
    }
</script>

image-20231101161842732

ue修改

主要是 Send Pixel Streaming Response 这个函数，下面的逻辑就是，接收前端的消息 并打印到ue 的像素流中，并且返回给前端

image-20231101162011042

我们也可以，在接收到的字符串里加点信息，代表是ue 返回的

如

image-20231101163140356

通信demo测试

将前端和ue 重新跑下，可以看到

image-20231101163304502

参考

vue + ue4通信：

https://www.bilibili.com/video/BV1Ly4y1u7nk/?spm_id_from=333.337.search-card.all.click&vd_source=e7d69fe6f27a4fce66589e592196cbbd

https://blog.csdn.net/xujl0429/article/details/125768807

ue 参考：

https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/

https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/

https://docs.unrealengine.com/4.27/zh-CN/Resources/Showcases/PixelStreamingShowcase/?

posted @ 2023-11-01 16:34  makalo  阅读(342)  评论(0)  编辑  收藏  举报