Messages in this thread | | | Date | Wed, 6 Apr 2022 14:50:42 +0200 | Subject | Re: [PATCH v4 00/15] Move HEVC stateless controls out of staging | From | Benjamin Gaignard <> |
| |
Le 06/04/2022 à 14:46, Adam Ford a écrit : > On Wed, Apr 6, 2022 at 7:41 AM Benjamin Gaignard > <benjamin.gaignard@collabora.com> wrote: >> >> Le 06/04/2022 à 14:28, Adam Ford a écrit : >>> On Wed, Apr 6, 2022 at 1:56 AM Benjamin Gaignard >>> <benjamin.gaignard@collabora.com> wrote: >>>> Le 05/04/2022 à 23:27, Adam Ford a écrit : >>>>> On Mon, Apr 4, 2022 at 10:56 AM Benjamin Gaignard >>>>> <benjamin.gaignard@collabora.com> wrote: >>>>>> Le 02/04/2022 à 18:59, Adam Ford a écrit : >>>>>>> On Sat, Apr 2, 2022 at 11:22 AM Adam Ford <aford173@gmail.com> wrote: >>>>>>>> On Fri, Apr 1, 2022 at 8:18 AM Benjamin Gaignard >>>>>>>> <benjamin.gaignard@collabora.com> wrote: >>>>>>>>> Le 31/03/2022 à 08:53, Benjamin Gaignard a écrit : >>>>>>>>>> Le 30/03/2022 à 20:52, Adam Ford a écrit : >>>>>>>>>>> On Wed, Mar 30, 2022 at 2:53 AM Benjamin Gaignard >>>>>>>>>>> <benjamin.gaignard@collabora.com> wrote: >>>>>>>>>>>> Le 28/02/2022 à 15:08, Benjamin Gaignard a écrit : >>>>>>>>>>>>> This series aims to make HEVC uapi stable and usable for hardware >>>>>>>>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro) >>>>>>>>>>>>> and 2 out of the tree drivers (rkvdec and RPI). >>>>>>>>>>>>> >>>>>>>>>>>>> After the remarks done on version 2, I have completely reworked to >>>>>>>>>>>>> patches >>>>>>>>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the >>>>>>>>>>>>> titles. >>>>>>>>>>>>> >>>>>>>>>>>>> Version 4: >>>>>>>>>>>>> - Add num_entry_point_offsets field in struct >>>>>>>>>>>>> v4l2_ctrl_hevc_slice_params >>>>>>>>>>>>> - Fix V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS name >>>>>>>>>>>>> - Initialize control V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS >>>>>>>>>>>>> - Fix space/tab issue in kernel-doc >>>>>>>>>>>>> - Add patch to change data_bit_offset definition >>>>>>>>>>>>> - Fix hantro-media SPDX license >>>>>>>>>>>>> - put controls under stateless section in v4l2-ctrls-defs.c >>>>>>>>>>>>> >>>>>>>>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec. >>>>>>>>>>>> Dear reviewers, >>>>>>>>>>>> >>>>>>>>>>>> This series is waiting for your feedback, >>>>>>>>>>> I tried several times with the suggested repos for both the kernel and >>>>>>>>>>> g-streamer without success getting Fluster to pass any tests on the >>>>>>>>>>> imx8mq. I can try again but I likely won't get to it until this >>>>>>>>>>> weekend. If I can get it working, I'll test both the 8mq and 8mm. >>>>>>>>>> Thanks a lot for that. >>>>>>>>>> >>>>>>>>>> Benjamin >>>>>>>>> Adam, >>>>>>>>> >>>>>>>>> You may need to check if h265parse and v4l2slh265dec are available on your board. >>>>>>>> I ran gst-inspect to see what showed up with 265 in the name. >>>>>>>> >>>>>>>> # gst-inspect-1.0 |grep 265 >>>>>>>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder >>>>>>>> rtp: rtph265depay: RTP H265 depayloader >>>>>>>> rtp: rtph265pay: RTP H265 payloader >>>>>>>> typefindfunctions: video/x-h265: h265, x265, 265 >>>>>>>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder >>>>>>>> videoparsersbad: h265parse: H.265 parser >>>>>>>> >>>>>>>> It appears I have both h265parse and v4l2slh265dec. >>>>>>>> >>>>>>>>> fluster check if v4l2slh265dec is working fine with this command line: >>>>>>>>> >>>>>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink >>>>>>>>> >>>>>>>>> so if one of them is missing it won't work. >>>>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink >>>>>>>> Setting pipeline to PAUSED ... >>>>>>>> 0:00:00.098389938 526 0xaaaaf9d86ac0 ERROR v4l2codecs-decoder >>>>>>>> gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> >>>>>>>> VIDIOC_G_EXT_CTRLS failed: Invalid argument >>>>>>>> ERROR: from element >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not >>>>>>>> report framing and start code method. >>>>>>>> Additional debug info: >>>>>>>> ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): >>>>>>>> gst_v4l2_codec_h265_dec_open (): >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: >>>>>>>> gst_v4l2_decoder_get_controls() failed: Invalid argument >>>>>>>> ERROR: pipeline doesn't want to preroll. >>>>>>>> ERROR: from element >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not >>>>>>>> initialize supporting library. >>>>>>>> Additional debug info: >>>>>>>> ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): >>>>>>>> gst_video_decoder_change_state (): >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: >>>>>>>> Failed to open decoder >>>>>>>> ERROR: pipeline doesn't want to preroll. >>>>>>>> Failed to set pipeline to PAUSED. >>>>>>>> Setting pipeline to NULL ... >>>>>>>> Freeing pipeline ... >>>>>>>> >>>>>>>> Does this mean I have a wrong version of the kernel and/or incomplete patches? >>>>>>> I double checked the branches. >>>>>>> >>>>>>> Kernel: >>>>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git >>>>>>> branch: origin/HEVC_UAPI_V4 >>>>>>> >>>>>>> Gstreamer: >>>>>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git >>>>>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 >>>>>>> >>>>>>> >>>>>>> I am still not able to run h.265/HEVC tests. >>>>>> Hello Adam, >>>>>> >>>>>> I have updated the following branches with the versions I have used today: >>>>>> >>>>>> Kernel: >>>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git >>>>>> branch: origin/HEVC_UAPI_V5 only one change in documentation vs version 4 but rebased in v5.18-rc1 >>>>>> >>>>>> Gstreamer: >>>>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git >>>>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 updated on the latest GST main branch >>>>>> >>>>>> I hope this will work fine this time. >>>>> I wish I had better news for you: >>>>> >>>>> dmesg shows the hantro driver is being loaded: >>>>> >>>>> [ 38.612243] hantro-vpu 38300000.video-codec: registered >>>>> nxp,imx8mq-vpu-g1-dec as /dev/video0 >>>>> [ 38.612618] hantro-vpu 38310000.video-codec: registered >>>>> nxp,imx8mq-vpu-g2-dec as /dev/video1 >>>>> >>>>> # gst-inspect-1.0 |grep 265 >>>>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder >>>>> rtp: rtph265depay: RTP H265 depayloader >>>>> rtp: rtph265pay: RTP H265 payloader >>>>> typefindfunctions: video/x-h265: h265, x265, 265 >>>>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder >>>>> videoparsersbad: h265parse: H.265 parser >>>>> >>>>> Fluster reports: >>>>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for >>>>> GStreamer 1.0... ❌ >>>> Still the same error about non supported control ? >>> # gst-lagst-launch-1.0 appsrc num-buffers=0 ! h265parse ! >>> v4l2slh265dec ! fakesink >>> Setting pipeline to PAUSED ... >>> 0:00:01.704385508 420 0xaaaaec0a76c0 ERROR v4l2codecs-decoder >>> gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> >>> VIDIOC_G_EXT_CTRLS failed: Invalid argument >>> ERROR: from element >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not >>> report framing and start code method. >>> Additional debug info: >>> ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): >>> gst_v4l2_codec_h265_dec_open (): >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: >>> gst_v4l2_decoder_get_controls() failed: Invalid argument >>> ERROR: pipeline doesn't want to preroll. >>> ERROR: from element >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not >>> initialize supporting library. >>> Additional debug info: >>> ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): >>> gst_video_decoder_change_state (): >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: >>> Failed to open decoder >>> ERROR: pipeline doesn't want to preroll. >>> Failed to set pipeline to PAUSED. >>> Setting pipeline to NULL ... >>> Freeing pipeline ... >> What is the result of v4l2-ctl ? >> Here I have: >> v4l2-ctl --all -d 1 >> Driver Info: >> Driver name : hantro-vpu >> Card type : nxp,imx8mq-vpu-g2-dec >> Bus info : platform: hantro-vpu >> Driver version : 5.18.0 >> Capabilities : 0x84204000 >> Video Memory-to-Memory Multiplanar >> Streaming >> Extended Pix Format >> Device Capabilities >> Device Caps : 0x04204000 >> Video Memory-to-Memory Multiplanar >> Streaming >> Extended Pix Format >> Media Driver Info: >> Driver name : hantro-vpu >> Model : hantro-vpu >> Serial : >> Bus info : platform: hantro-vpu >> Media version : 5.18.0 >> Hardware revision: 0x00000000 (0) >> Driver version : 5.18.0 >> Interface Info: >> ID : 0x0300000c >> Type : V4L Video >> Entity Info: >> ID : 0x00000001 (1) >> Name : nxp,imx8mq-vpu-g2-dec-source >> Function : V4L2 I/O >> Pad 0x01000002 : 0: Source >> Link 0x02000008: to remote pad 0x1000004 of entity 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable >> Priority: 2 >> Format Video Capture Multiplanar: >> Width/Height : 48/48 >> Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear)) >> Field : None >> Number of planes : 1 >> Flags : >> Colorspace : JPEG >> Transfer Function : Default >> YCbCr/HSV Encoding: Default >> Quantization : Default >> Plane 0 : >> Bytes per Line : 48 >> Size Image : 3600 >> Format Video Output Multiplanar: >> Width/Height : 48/48 >> Pixel Format : 'S265' (HEVC Parsed Slice Data) >> Field : None >> Number of planes : 1 >> Flags : >> Colorspace : JPEG >> Transfer Function : Default >> YCbCr/HSV Encoding: Default >> Quantization : Default >> Plane 0 : >> Bytes per Line : 0 >> Size Image : 4608 >> >> User Controls >> >> hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 step=1 default=0 value=0 >> >> Codec Controls >> >> hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0 >> 0: Main >> 1: Main Still Picture >> 2: Main 10 >> hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0 >> 0: 1 >> 1: 2 >> 2: 2.1 >> 3: 3 >> 4: 3.1 >> 5: 4 >> 6: 4.1 >> 7: 5 >> 8: 5.1 >> >> Stateless Codec Controls >> >> vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload >> vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload >> hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload >> hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload >> hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload >> hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload >> hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 default=1 value=1 >> 1: Frame-Based >> hevc_start_code 0x00a40a96 (menu) : min=1 max=1 default=1 value=1 >> 1: Annex B Start Code >> > Mine looks like: > > # v4l2-ctl --all -d 1 > Driver Info: > Driver name : hantro-vpu > Card type : nxp,imx8mq-vpu-g2-dec > Bus info : platform: hantro-vpu > Driver version : 5.18.0 > Capabilities : 0x84204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : hantro-vpu > Model : hantro-vpu > Serial : > Bus info : platform: hantro-vpu > Media version : 5.18.0 > Hardware revision: 0x00000000 (0) > Driver version : 5.18.0 > Interface Info: > ID : 0x0300000c > Type : V4L Video > Entity Info: > ID : 0x00000001 (1) > Name : nxp,imx8mq-vpu-g2-dec-source > Function : V4L2 I/O > Pad 0x01000002 : 0: Source > Link 0x02000008: to remote pad 0x1000004 of entity > 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable > Priority: 2 > Format Video Capture Multiplanar: > Width/Height : 48/48 > Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear)) > Field : None > Number of planes : 1 > Flags : > Colorspace : JPEG > Transfer Function : Default > YCbCr/HSV Encoding: Default > Quantization : Default > Plane 0 : > Bytes per Line : 48 > Size Image : 3600 > Format Video Output Multiplanar: > Width/Height : 48/48 > Pixel Format : 'S265' (HEVC Parsed Slice Data) > Field : None > Number of planes : 1 > Flags : > Colorspace : JPEG > Transfer Function : Default > YCbCr/HSV Encoding: Default > Quantization : Default > Plane 0 : > Bytes per Line : 0 > Size Image : 4608 > > User Controls > > hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 > step=1 default=0 value=0 > > Codec Controls > > hevc_profile 0x00990b67 (menu) : min=0 max=2 > default=0 value=0 > 0: Main > 1: Main Still Picture > 2: Main 10 > hevc_level 0x00990b68 (menu) : min=0 max=8 > default=0 value=0 > 0: 1 > 1: 2 > 2: 2.1 > 3: 3 > 4: 3.1 > 5: 4 > 6: 4.1 > 7: 5 > 8: 5.1 > > Stateless Codec Controls > > vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload > vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload > hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload > hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload > hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload > hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload > hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 > default=1 value=1 > 1: Frame-Based > hevc_start_code 0x00a40a96 (menu) : min=1 max=1 > default=1 value=1 > 1: Annex B Start Code
It is the same so that suggest the issue is coming from GStreamer plugin.
> >>>> Benjamin >>>> >>>>> adam >>>>>> Benjamin >>>>>> >>>>>>> adam >>>>>>>> adam >>>>>>>>> Regards, >>>>>>>>> Benjamin >>>>>>>>> >>>>>>>>>>> adam >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Benjamin >>>>>>>>>>>> >>>>>>>>>>>>> Benjamin >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Benjamin Gaignard (12): >>>>>>>>>>>>> media: uapi: HEVC: Add missing fields in HEVC controls >>>>>>>>>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS >>>>>>>>>>>>> prefix >>>>>>>>>>>>> media: uapi: HEVC: Add document uAPI structure >>>>>>>>>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS >>>>>>>>>>>>> as a >>>>>>>>>>>>> dynamic array >>>>>>>>>>>>> media: uapi: Move parsed HEVC pixel format out of staging >>>>>>>>>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS >>>>>>>>>>>>> control >>>>>>>>>>>>> media: uapi: Move the HEVC stateless control type out of staging >>>>>>>>>>>>> media: controls: Log HEVC stateless control in .std_log >>>>>>>>>>>>> media: uapi: Create a dedicated header for Hantro control >>>>>>>>>>>>> media: uapi: HEVC: fix padding in v4l2 control structures >>>>>>>>>>>>> media: uapi: Change data_bit_offset definition >>>>>>>>>>>>> media: uapi: move HEVC stateless controls out of staging >>>>>>>>>>>>> >>>>>>>>>>>>> Hans Verkuil (3): >>>>>>>>>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY >>>>>>>>>>>>> v4l2-ctrls: add support for dynamically allocated arrays. >>>>>>>>>>>>> vivid: add dynamic array test control >>>>>>>>>>>>> >>>>>>>>>>>>> .../userspace-api/media/drivers/hantro.rst | 5 - >>>>>>>>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 833 >>>>>>>>>>>>> ++++++++++++++++++ >>>>>>>>>>>>> .../media/v4l/ext-ctrls-codec.rst | 780 >>>>>>>>>>>>> ---------------- >>>>>>>>>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +- >>>>>>>>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 + >>>>>>>>>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 + >>>>>>>>>>>>> .../media/videodev2.h.rst.exceptions | 5 + >>>>>>>>>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 + >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++- >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++- >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 37 +- >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +- >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +- >>>>>>>>>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +- >>>>>>>>>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +- >>>>>>>>>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +- >>>>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +- >>>>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 2 +- >>>>>>>>>>>>> include/media/hevc-ctrls.h | 250 ------ >>>>>>>>>>>>> include/media/v4l2-ctrls.h | 48 +- >>>>>>>>>>>>> include/uapi/linux/hantro-media.h | 19 + >>>>>>>>>>>>> include/uapi/linux/v4l2-controls.h | 439 +++++++++ >>>>>>>>>>>>> include/uapi/linux/videodev2.h | 13 + >>>>>>>>>>>>> 23 files changed, 1697 insertions(+), 1170 deletions(-) >>>>>>>>>>>>> delete mode 100644 include/media/hevc-ctrls.h >>>>>>>>>>>>> create mode 100644 include/uapi/linux/hantro-media.h >>>>>>>>>>>>>
| |