【分享】以H.265 1080p 运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display
【分享】以H.265 1080p 运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display
运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display，可以测试HDMI输入输出，和VCU的低延时编码。Xilinx wiki的文章MPSoC VCU TRD 2019.2 - Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display以H.264和4K分辨率为例。
下面记录H.265和1080p分辨率的运行命令。
1. 设置HDMI输入分辨率
首先设置HDMI输入，保证输入分辨率是1080p。我使用电脑Dell 5580作为ZCU106单板的HDMI输入源。电脑Dell 5580上看到两个显示器，一个是真实显示器，一个是ZCU106单板的HDMI输入。虽然我在电脑Dell 5580设置ZCU106单板的HDMI输入为1080p，但是在ZCU106单板使用命令“xmedia-ctl -p -d /dev/media1”查看，HDMI输入的分辨率一直是4K。后来把电脑Dell 5580的输出模式从扩展(Extend)改为复制(Duplicate)后，在ZCU106单板才看到HDMI输入的分辨率是1080p。这一点很莫名其妙。
错误的分辨率信息
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/3840x2160 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps: progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 3840x2160p30 (4400x2250) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
正确的分辨率信息
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps: progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
2. 设置VPSS分辨率和格式
接下来设置ZCU106的VPSS的分辨率和格式。设置后，再使用命令“xmedia-ctl -p -d /dev/media1”查看，所有pad的分辨率信息都是1080p了。
root@vcu_llp2_xv20:~# xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\":0 [fmt:RBG888_1X24/1920x1080 field:none]"
root@vcu_llp2_xv20:~# xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\":1 [fmt:UYVY10_1X20/1920x1080 field:none]"
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:RBG888_1X24/1920x1080 field:none]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:UYVY10_1X20/1920x1080 field:none]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
3. 设置HDMI输出分辨率
使用命令“modetest -D a00c0000.v_mix -s 35:1920x1080-30@BG24”，设置设置HDMI输出分辨率。设置时，会产生异常，可以暂时忽略。设置后，HDMI显示器会显示蓝屏。
root@vcu_llp2_xv20:~# modetest -D a00c0000.v_mix -s 35:1920x1080-30@BG24
setting mode 1920x1080-30Hz@BG24 on connectors 35, crtc 33
[ 254.364908] ------------[ cut here ]------------
[ 254.369527] driver forgot to call drm_crtc_vblank_off()
[ 254.374783] WARNING: CPU: 2 PID: 2878 at drivers/gpu/drm/drm_atomic_helper.c:1000 drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.386941] Modules linked in: dmaproxy(O) al5d(O) al5e(O) allegro(O) xlnx_vcu xlnx_vcu_clk xilinx_hdmi_tx(O) xilinx_hdmi_rx(O) xlnx_vcu_core xilinx_vphy(O) dp159(O) mali(O) uio_pdrv_genirq
[ 254.403812] CPU: 2 PID: 2878 Comm: modetest Tainted: G O 4.19.0-xilinx-v2019.2 #1
[ 254.412584] Hardware name: ZynqMP ZCU106 RevA (DT)
[ 254.417361] pstate: 60000005 (nZCv daif -PAN -UAO)
[ 254.422135] pc : drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.428646] lr : drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.435154] sp : ffffff8014feba00
[ 254.438453] x29: ffffff8014feba00 x28: ffffffc87b2d2400
[ 254.443757] x27: ffffffc85fc14800 x26: 0000000000000000
[ 254.449061] x25: 0000000000000038 x24: ffffff80091c725b
[ 254.454365] x23: ffffff8008d74e98 x22: 0000000000000000
[ 254.459669] x21: ffffffc85f29c800 x20: ffffffc85fc53890
[ 254.464973] x19: ffffffc87ae49980 x18: 0000000000000010
[ 254.470277] x17: 0000000000000000 x16: 0000000000000000
[ 254.475581] x15: ffffffffffffffff x14: ffffff8009148648
[ 254.480885] x13: ffffff80891e7f0f x12: ffffff80091e7f17
[ 254.486189] x11: ffffff800915a000 x10: ffffff8014feb6e0
[ 254.491493] x9 : 00000000ffffffd0 x8 : ffffff8008543da8
[ 254.496797] x7 : 6c6163206f742074 x6 : 00000000000001d7
[ 254.502101] x5 : 0000000000000003 x4 : 0000000000000000
[ 254.507396] x3 : 0000000000000000 x2 : ffffffffffffffff
[ 254.512700] x1 : 6b059e08b5ce2000 x0 : 0000000000000000
[ 254.518004] Call trace:
[ 254.520438] drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.526609] drm_atomic_helper_commit_tail+0x20/0x78
[ 254.531556] commit_tail+0x74/0x78
[ 254.534942] drm_atomic_helper_commit+0xc8/0x140
[ 254.539544] drm_atomic_commit+0x48/0x58
[ 254.543458] drm_atomic_helper_set_config+0xa0/0xb0
[ 254.548321] drm_mode_setcrtc+0x144/0x5d8
[ 254.552322] drm_ioctl_kernel+0xb4/0x100
[ 254.556228] drm_ioctl+0x204/0x3a8
[ 254.559616] do_vfs_ioctl+0xb8/0x8a0
[ 254.563182] ksys_ioctl+0x44/0x90
[ 254.566481] __arm64_sys_ioctl+0x1c/0x28
[ 254.570388] el0_svc_common+0x84/0xd8
[ 254.574041] el0_svc_handler+0x68/0x80
[ 254.577774] el0_svc+0x8/0xc
[ 254.580637] ---[ end trace 44a2c3f75cacf5a7 ]---
4. 本板运行
复制文件/media/card/config/input.cfg为/media/card/config/input-1080p.cfg，修改其中的分辨率为1080p。再以命令"vcu_gst_app /media/card/config/input-1080p.cfg"运行，可以看到本板的HDMI输入显示到了HDMI输出上。
root@vcu_llp2_xv20:/media/card/config# cat input-1080p.cfg
Common Configuration : START
Num Of Input : 1
Output : HDMI
Out Type : Display
Frame Rate : 60
Exit
Input Configuration : START
Input Num : 1
Input Type : hdmi_1
Raw : FALSE
Width : 1920
Height : 1080
Format : XV20
Enable LLP2 : FALSE
Exit
Encoder Configuration : START
Encoder Num : 1
Encoder Name : AVC
Profile : High
Rate Control : Low_Latency
Filler Data : False
QP : Auto
L2 Cache : TRUE
Latency Mode : Sub_Frame
Low Bandwidth : FALSE
Gop Mode : Basic
Bitrate : 25000
B Frames : 0
Slice : 8
GoP Length : 60
Preset : Custom
Exit
Streaming Configuration : START
Streaming Num : 1
Host IP : 192.168.25.89
Port : 5004
Exit
Trace Configuration : START
FPS Info : TRUE
APM Info : TRUE
Pipeline Info : TRUE
Exit
root@vcu_llp2_xv20:/media/card/config# vcu_gst_app /media/card/config/input-1080p.cfg
/*************************Pipeline Information Start*************************/
Pipeline Info : On
Fps Info : On
APM Info : On
Output goes on : HDMI
Frame rate : 60
Number of Source is : 1
Use case is to : Display
B Frames : 0
Bitrate : 25000
Enable L2Cache : True
Enc Name : AVC
Gop Len : 60
Profile : High
Qp Mode : Auto
Rc Mode : Low_Latency
Num Slice : 8
GoP Mode : Basic
Filler Data : False
Low Bandwidth : False
Latency Mode : Sub_Frame
Device Type : HDMI
Format : XV20
Width : 1920
Height : 1080
Raw : False
Accelerator flag : False
Enable_scd flag : False
Enable_roi flag : False
Enable_llp2 flag : False
Src Type : Live Src
/*************************Pipeline Information End*************************/
!! The specified Level is too low and will be adjusted !!
Encoder Memory Bandwidth (1.04 Gbps)
Decoder Memory Bandwidth (0.00 Gbps)
Pipeline [1] Fps[0]
Encoder Memory Bandwidth (8.72 Gbps)
Decoder Memory Bandwidth (7.99 Gbps)
Pipeline [1] Fps[20]
Encoder Memory Bandwidth (8.78 Gbps)
Decoder Memory Bandwidth (7.12 Gbps)
Pipeline [1] Fps[22]
Encoder Memory Bandwidth (8.76 Gbps)
Decoder Memory Bandwidth (7.18 Gbps)
Pipeline [1] Fps[22]
Encoder Memory Bandwidth (8.72 Gbps)
Decoder Memory Bandwidth (7.17 Gbps)
Pipeline [1] Fps[22]
^CHit Ctrl-C
Quitting the app now
Quitting the loop
playback count 1
pipeline stopped successfully
root@vcu_llp2_xv20:/media/card/config# ^C
5. 网络运行
5.1. 设置网络
在HDMI输入单板上执行命令“ifconfig eth0 192.168.25.10 up”，配置HDMI输入单板的IP为192.168.25.10。
在HDMI输出单板上执行命令“ifconfig eth0 192.168.25.89 up”，配置HDMI输出单板的IP为192.168.25.89。再执行命令，“ping 192.168.25.10”，检查网络是否通畅。
5.2. 发送视频流
root@vcu_llp2_xv20:~# ifconfig eth0 192.168.25.10 up
root@vcu_llp2_xv20:/media/card/config# gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=1920, height=1080, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=low-delay-p gop-length=60 b-frames=0 target-bitrate=20000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
!! Warning : Adapting profile to support bitdepth and chroma mode
!! The specified Level is too low and will be adjusted !!
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:21.804704005
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
root@vcu_llp2_xv20:/media/card/config# gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=1920, height=1080, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=low-delay-p gop-length=60 b-frames=0 target-bitrate=20000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
!! Warning : Adapting profile to support bitdepth and chroma mode
!! The specified Level is too low and will be adjusted !!
5.3. 接收视频流
root@vcu_llp2_xv20:~# ifconfig eth0 192.168.25.89 up
root@vcu_llp2_xv20:~#
root@vcu_llp2_xv20:~# ping 192.168.25.10
PING 192.168.25.10 (192.168.25.10): 56 data bytes
64 bytes from 192.168.25.10: seq=0 ttl=64 time=0.229 ms
64 bytes from 192.168.25.10: seq=1 ttl=64 time=0.071 ms
^C
--- 192.168.25.10 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.071/0.150/0.229 ms
root@vcu_llp2_xv20:~# gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, media=video, clock-rate=90000, payload=96, encoding-name=H265" ! rtpjitterbuffer latency=5 ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30" sync=true -v
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: display-width = 1920
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: display-height = 1080
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: sync = true
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstRtpH265Depay:rtph265depay0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstRtpH265Depay:rtph265depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
Redistribute latency...
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
Redistribute latency...
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:1:7:1, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: sync = true
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 14, dropped: 15, fps: 27.85, drop rate: 29.84
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 29, dropped: 23, fps: 30.00, drop rate: 16.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 45, dropped: 31, fps: 30.00, drop rate: 15.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 60, dropped: 39, fps: 30.00, drop rate: 16.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 75, dropped: 44, fps: 29.99, drop rate: 10.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 90, dropped: 49, fps: 29.97, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 105, dropped: 54, fps: 29.98, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 120, dropped: 59, fps: 29.97, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 136, dropped: 64, fps: 30.09, drop rate: 9.40
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 150, dropped: 70, fps: 26.25, drop rate: 11.25
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 162, dropped: 76, fps: 23.85, drop rate: 11.92
6. 技巧
Xilinx wiki的文章的命令中，大约每秒发送一个I帧，造成视频闪烁。如果去掉“periodicity-idr=60”，就不会有视频闪烁了。
以网络运行demo时，先执行接收端Gstreamer命令，再执行发送端Gstreamer命令。如果顺序反了，接收端收不到包，不能显示图像。
7. 已知问题
使用Yavta不能成功捕获视频帧。
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:RBG888_1X24/1920x1080 field:none]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:UYVY10_1X20/1920x1080 field:none]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f UYVY -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes)[63898.159475] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
device.
Video format set: UYVY (59565955) 1920x1080 field none[63898.173868] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
, 1 planes:
* Stride 3840, buffer size 4147200
Video format:[63898.188189] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
UYVY (59565955) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7fa84b0000.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fa80bb000.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7fa7cc6000.
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f YUYV -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device.
Video format set: YUYV (56595559) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
Video format: YUYV (56595559) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7f8ad60000.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7f8a96b000.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7f8a576000.
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f Y10 -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes)[63965.043523] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
device.
Video format set: Y10 (20303159) 1920x1080 field none,[63965.057886] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
1 planes:
* Stride 3840, buffer size 4147200
Video format: [63965.072212] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
Y10 (20303159) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7fbdb14000.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fbd71f000.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7fbd32a000.
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~# v4l2-ctl --list-devices
vcap_hdmi output 0 (platform:vcap_hdmi:0):
/dev/video0
vcap_hdmi_2 output 0 (platform:vcap_hdmi_2:0):
/dev/video1
root@vcu_llp2_xv20:~# ls /sys/class/video4linux/video0/device/ | grep -i media
media1
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:RBG888_1X24/1920x1080 field:none]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
root@vcu_llp2_xv20:~# media-ctl -d /dev/media1 -V '"a0040000.v_proc_ss":1 [fmt:VYYUYY8_1X24/1920x10e]'field:none
Unable to setup formats: Invalid argument (22)
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842522/Xilinx+V4L2+MIPI+CSI+driver
root@vcu_llp2_xv20:~# media-ctl -d /dev/media1 -V '"a0040000.v_proc_ss":1 [fmt:UYVY/1920x1080 colorspace:srgb field:none]'
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f UYVY -s 1920x1080 --skip 7 -F /dev/video1
Device /dev/video1 opened.
Device `vcap_hdmi_2 output 0' on `platform:vcap_hdmi_2:0' is a video output (without mplan[64761.083514] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
es) device.
Video format set: UYVY (59565955) 1920x1080 field n[64761.097896] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
one, 1 planes:
* Stride 3840, buffer size 4147200
Video form[64761.112561] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
at: UYVY (59565955) 1920x1080 field none, 1 planes:
* Stride [64761.125320] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
3840, buffer size 4147200
3 buffers requested.
length: 1 offse[64761.139647] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
t: 4059898224 timestamp type/source: mono/EoF
Buffer 0/0 mapped[64761.154311] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
at address 0x7fb4223000.
length: 1 offset: 4059898224 timestam[64761.167077] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
p type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fb3e2e0[64761.181400] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
00.
length: 1 offset: 4059898224 timestamp type/source: mono/Eo[64761.196071] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
F
Buffer 2/0 mapped at address 0x7fb3a39000.
[64761.265216] WARNING: CPU: 2 PID: 18896 at drivers/media/common/videobuf2/videobuf2-core.c:1371 vb2_start_streaming+0xe4/0x140
[64761.276502] Modules linked in: dmaproxy(O) al5e(O) al5d(O) allegro(O) xlnx_vcu_clk xlnx_vcu xilinx_hdmi_rx(O) xilinx_hdmi_tx(O) dp159(O) xlnx_vcu_core xilinx_vphy(O) mali(O) uio_pdrv_genirq
[64761.293373] CPU: 2 PID: 18896 Comm: yavta Tainted: G W O 4.19.0-xilinx-v2019.2 #1
[64761.301972] Hardware name: ZynqMP ZCU106 RevA (DT)
[64761.306748] pstate: 20000005 (nzCv daif -PAN -UAO)
[64761.311523] pc : vb2_start_streaming+0xe4/0x140
[64761.316036] lr : vb2_start_streaming+0x64/0x140
[64761.320549] sp : ffffff8009663b20
[64761.323847] x29: ffffff8009663b20 x28: ffffffc87924bc00
[64761.329151] x27: ffffff800919e5c8 x26: ffffff8008c02d60
[64761.334455] x25: 0000000000000000 x24: ffffffc87bb757a8
[64761.339759] x23: 0000000000000000 x22: ffffff8009148648
[64761.345063] x21: ffffffc87bb75a90 x20: ffffffc87bb75aa8
[64761.350367] x19: 00000000ffffffea x18: 0000000000000400
[64761.355671] x17: 0000000000000000 x16: 0000000000000000
[64761.360975] x15: 0000000000000400 x14: 000000000000011a
[64761.366279] x13: 0000000000000000 x12: 0000000000000001
[64761.371583] x11: 0000000000000000 x10: 00000000000007f0
[64761.376879] x9 : ffffff80096638e0 x8 : ffffffc8600aeb50
[64761.382183] x7 : ffffffc87ff86a00 x6 : ffffffc8600ae3c0
[64761.387487] x5 : 0000000000000005 x4 : 0000000000004000
[64761.392782] x3 : ffffffc87bb75b08 x2 : 6b059e08b5ce2000
[64761.398086] x1 : 0000000000000000 x0 : ffffffc85fb611b8
[64761.403390] Call trace:
[64761.405823] vb2_start_streaming+0xe4/0x140
[64761.409990] vb2_core_streamon+0x88/0x180
[64761.413983] vb2_streamon+0x18/0x60
[64761.417455] vb2_ioctl_streamon+0x48/0x58
[64761.421449] v4l_streamon+0x20/0x28
[64761.424929] __video_do_ioctl+0x23c/0x498
[64761.428922] video_usercopy+0x144/0x518
[64761.432741] video_ioctl2+0x14/0x1c
[64761.436214] v4l2_ioctl+0x3c/0x58
[64761.439515] do_vfs_ioctl+0xb8/0x8a0
[64761.443081] ksys_ioctl+0x44/0x90
[64761.446380] __arm64_sys_ioctl+0x1c/0x28
[64761.450288] el0_svc_common+0x84/0xd8
[64761.453941] el0_svc_handler+0x68/0x80
[64761.457674] el0_svc+0x8/0xc
[64761.460537] ---[ end trace 091bf6f1324177c3 ]---
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~#
7.1. 参考文章
Zynq UltraScale+ MPSoC VCU TRD - Debugging - HDMI Rx Capture Pipeline
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/174784624/Zynq+UltraScale+MPSoC+VCU+TRD+-+Debugging+-+HDMI+Rx+Capture+Pipeline
Cannot capture image by yavta tool
https://forums.xilinx.com/t5/Embedded-Linux/Cannot-capture-image-by-yavta-tool/m-p/1064135#M39452
Xilinx V4L2 MIPI CSI driver
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842522/Xilinx+V4L2+MIPI+CSI+driver
付汉杰 hankf@amd.com
