lkml.org 
[lkml]   [2022]   [Apr]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [PATCH v4 00/15] Move HEVC stateless controls out of staging
    On Wed, Apr 6, 2022 at 7:41 AM Benjamin Gaignard
    <benjamin.gaignard@collabora.com> wrote:
    >
    >
    > Le 06/04/2022 à 14:28, Adam Ford a écrit :
    > > On Wed, Apr 6, 2022 at 1:56 AM Benjamin Gaignard
    > > <benjamin.gaignard@collabora.com> wrote:
    > >>
    > >> Le 05/04/2022 à 23:27, Adam Ford a écrit :
    > >>> On Mon, Apr 4, 2022 at 10:56 AM Benjamin Gaignard
    > >>> <benjamin.gaignard@collabora.com> wrote:
    > >>>> Le 02/04/2022 à 18:59, Adam Ford a écrit :
    > >>>>> On Sat, Apr 2, 2022 at 11:22 AM Adam Ford <aford173@gmail.com> wrote:
    > >>>>>> On Fri, Apr 1, 2022 at 8:18 AM Benjamin Gaignard
    > >>>>>> <benjamin.gaignard@collabora.com> wrote:
    > >>>>>>> Le 31/03/2022 à 08:53, Benjamin Gaignard a écrit :
    > >>>>>>>> Le 30/03/2022 à 20:52, Adam Ford a écrit :
    > >>>>>>>>> On Wed, Mar 30, 2022 at 2:53 AM Benjamin Gaignard
    > >>>>>>>>> <benjamin.gaignard@collabora.com> wrote:
    > >>>>>>>>>> Le 28/02/2022 à 15:08, Benjamin Gaignard a écrit :
    > >>>>>>>>>>> This series aims to make HEVC uapi stable and usable for hardware
    > >>>>>>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
    > >>>>>>>>>>> and 2 out of the tree drivers (rkvdec and RPI).
    > >>>>>>>>>>>
    > >>>>>>>>>>> After the remarks done on version 2, I have completely reworked to
    > >>>>>>>>>>> patches
    > >>>>>>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the
    > >>>>>>>>>>> titles.
    > >>>>>>>>>>>
    > >>>>>>>>>>> Version 4:
    > >>>>>>>>>>> - Add num_entry_point_offsets field in struct
    > >>>>>>>>>>> v4l2_ctrl_hevc_slice_params
    > >>>>>>>>>>> - Fix V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS name
    > >>>>>>>>>>> - Initialize control V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS
    > >>>>>>>>>>> - Fix space/tab issue in kernel-doc
    > >>>>>>>>>>> - Add patch to change data_bit_offset definition
    > >>>>>>>>>>> - Fix hantro-media SPDX license
    > >>>>>>>>>>> - put controls under stateless section in v4l2-ctrls-defs.c
    > >>>>>>>>>>>
    > >>>>>>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
    > >>>>>>>>>> Dear reviewers,
    > >>>>>>>>>>
    > >>>>>>>>>> This series is waiting for your feedback,
    > >>>>>>>>> I tried several times with the suggested repos for both the kernel and
    > >>>>>>>>> g-streamer without success getting Fluster to pass any tests on the
    > >>>>>>>>> imx8mq. I can try again but I likely won't get to it until this
    > >>>>>>>>> weekend. If I can get it working, I'll test both the 8mq and 8mm.
    > >>>>>>>> Thanks a lot for that.
    > >>>>>>>>
    > >>>>>>>> Benjamin
    > >>>>>>> Adam,
    > >>>>>>>
    > >>>>>>> You may need to check if h265parse and v4l2slh265dec are available on your board.
    > >>>>>> I ran gst-inspect to see what showed up with 265 in the name.
    > >>>>>>
    > >>>>>> # gst-inspect-1.0 |grep 265
    > >>>>>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder
    > >>>>>> rtp: rtph265depay: RTP H265 depayloader
    > >>>>>> rtp: rtph265pay: RTP H265 payloader
    > >>>>>> typefindfunctions: video/x-h265: h265, x265, 265
    > >>>>>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder
    > >>>>>> videoparsersbad: h265parse: H.265 parser
    > >>>>>>
    > >>>>>> It appears I have both h265parse and v4l2slh265dec.
    > >>>>>>
    > >>>>>>> fluster check if v4l2slh265dec is working fine with this command line:
    > >>>>>>>
    > >>>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink
    > >>>>>>>
    > >>>>>>> so if one of them is missing it won't work.
    > >>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink
    > >>>>>> Setting pipeline to PAUSED ...
    > >>>>>> 0:00:00.098389938 526 0xaaaaf9d86ac0 ERROR v4l2codecs-decoder
    > >>>>>> gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2>
    > >>>>>> VIDIOC_G_EXT_CTRLS failed: Invalid argument
    > >>>>>> ERROR: from element
    > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not
    > >>>>>> report framing and start code method.
    > >>>>>> Additional debug info:
    > >>>>>> ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155):
    > >>>>>> gst_v4l2_codec_h265_dec_open ():
    > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0:
    > >>>>>> gst_v4l2_decoder_get_controls() failed: Invalid argument
    > >>>>>> ERROR: pipeline doesn't want to preroll.
    > >>>>>> ERROR: from element
    > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not
    > >>>>>> initialize supporting library.
    > >>>>>> Additional debug info:
    > >>>>>> ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909):
    > >>>>>> gst_video_decoder_change_state ():
    > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0:
    > >>>>>> Failed to open decoder
    > >>>>>> ERROR: pipeline doesn't want to preroll.
    > >>>>>> Failed to set pipeline to PAUSED.
    > >>>>>> Setting pipeline to NULL ...
    > >>>>>> Freeing pipeline ...
    > >>>>>>
    > >>>>>> Does this mean I have a wrong version of the kernel and/or incomplete patches?
    > >>>>> I double checked the branches.
    > >>>>>
    > >>>>> Kernel:
    > >>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git
    > >>>>> branch: origin/HEVC_UAPI_V4
    > >>>>>
    > >>>>> Gstreamer:
    > >>>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git
    > >>>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15
    > >>>>>
    > >>>>>
    > >>>>> I am still not able to run h.265/HEVC tests.
    > >>>> Hello Adam,
    > >>>>
    > >>>> I have updated the following branches with the versions I have used today:
    > >>>>
    > >>>> Kernel:
    > >>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git
    > >>>> branch: origin/HEVC_UAPI_V5 only one change in documentation vs version 4 but rebased in v5.18-rc1
    > >>>>
    > >>>> Gstreamer:
    > >>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git
    > >>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 updated on the latest GST main branch
    > >>>>
    > >>>> I hope this will work fine this time.
    > >>> I wish I had better news for you:
    > >>>
    > >>> dmesg shows the hantro driver is being loaded:
    > >>>
    > >>> [ 38.612243] hantro-vpu 38300000.video-codec: registered
    > >>> nxp,imx8mq-vpu-g1-dec as /dev/video0
    > >>> [ 38.612618] hantro-vpu 38310000.video-codec: registered
    > >>> nxp,imx8mq-vpu-g2-dec as /dev/video1
    > >>>
    > >>> # gst-inspect-1.0 |grep 265
    > >>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder
    > >>> rtp: rtph265depay: RTP H265 depayloader
    > >>> rtp: rtph265pay: RTP H265 payloader
    > >>> typefindfunctions: video/x-h265: h265, x265, 265
    > >>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder
    > >>> videoparsersbad: h265parse: H.265 parser
    > >>>
    > >>> Fluster reports:
    > >>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
    > >>> GStreamer 1.0... ❌
    > >> Still the same error about non supported control ?
    > > # gst-lagst-launch-1.0 appsrc num-buffers=0 ! h265parse !
    > > v4l2slh265dec ! fakesink
    > > Setting pipeline to PAUSED ...
    > > 0:00:01.704385508 420 0xaaaaec0a76c0 ERROR v4l2codecs-decoder
    > > gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2>
    > > VIDIOC_G_EXT_CTRLS failed: Invalid argument
    > > ERROR: from element
    > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not
    > > report framing and start code method.
    > > Additional debug info:
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155):
    > > gst_v4l2_codec_h265_dec_open ():
    > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0:
    > > gst_v4l2_decoder_get_controls() failed: Invalid argument
    > > ERROR: pipeline doesn't want to preroll.
    > > ERROR: from element
    > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not
    > > initialize supporting library.
    > > Additional debug info:
    > > ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909):
    > > gst_video_decoder_change_state ():
    > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0:
    > > Failed to open decoder
    > > ERROR: pipeline doesn't want to preroll.
    > > Failed to set pipeline to PAUSED.
    > > Setting pipeline to NULL ...
    > > Freeing pipeline ...
    >
    > What is the result of v4l2-ctl ?
    > Here I have:
    > v4l2-ctl --all -d 1
    > Driver Info:
    > Driver name : hantro-vpu
    > Card type : nxp,imx8mq-vpu-g2-dec
    > Bus info : platform: hantro-vpu
    > Driver version : 5.18.0
    > Capabilities : 0x84204000
    > Video Memory-to-Memory Multiplanar
    > Streaming
    > Extended Pix Format
    > Device Capabilities
    > Device Caps : 0x04204000
    > Video Memory-to-Memory Multiplanar
    > Streaming
    > Extended Pix Format
    > Media Driver Info:
    > Driver name : hantro-vpu
    > Model : hantro-vpu
    > Serial :
    > Bus info : platform: hantro-vpu
    > Media version : 5.18.0
    > Hardware revision: 0x00000000 (0)
    > Driver version : 5.18.0
    > Interface Info:
    > ID : 0x0300000c
    > Type : V4L Video
    > Entity Info:
    > ID : 0x00000001 (1)
    > Name : nxp,imx8mq-vpu-g2-dec-source
    > Function : V4L2 I/O
    > Pad 0x01000002 : 0: Source
    > Link 0x02000008: to remote pad 0x1000004 of entity 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable
    > Priority: 2
    > Format Video Capture Multiplanar:
    > Width/Height : 48/48
    > Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear))
    > Field : None
    > Number of planes : 1
    > Flags :
    > Colorspace : JPEG
    > Transfer Function : Default
    > YCbCr/HSV Encoding: Default
    > Quantization : Default
    > Plane 0 :
    > Bytes per Line : 48
    > Size Image : 3600
    > Format Video Output Multiplanar:
    > Width/Height : 48/48
    > Pixel Format : 'S265' (HEVC Parsed Slice Data)
    > Field : None
    > Number of planes : 1
    > Flags :
    > Colorspace : JPEG
    > Transfer Function : Default
    > YCbCr/HSV Encoding: Default
    > Quantization : Default
    > Plane 0 :
    > Bytes per Line : 0
    > Size Image : 4608
    >
    > User Controls
    >
    > hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 step=1 default=0 value=0
    >
    > Codec Controls
    >
    > hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0
    > 0: Main
    > 1: Main Still Picture
    > 2: Main 10
    > hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0
    > 0: 1
    > 1: 2
    > 2: 2.1
    > 3: 3
    > 4: 3.1
    > 5: 4
    > 6: 4.1
    > 7: 5
    > 8: 5.1
    >
    > Stateless Codec Controls
    >
    > vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload
    > vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload
    > hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload
    > hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload
    > hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload
    > hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload
    > hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 default=1 value=1
    > 1: Frame-Based
    > hevc_start_code 0x00a40a96 (menu) : min=1 max=1 default=1 value=1
    > 1: Annex B Start Code
    >

    Mine looks like:

    # v4l2-ctl --all -d 1
    Driver Info:
    Driver name : hantro-vpu
    Card type : nxp,imx8mq-vpu-g2-dec
    Bus info : platform: hantro-vpu
    Driver version : 5.18.0
    Capabilities : 0x84204000
    Video Memory-to-Memory Multiplanar
    Streaming
    Extended Pix Format
    Device Capabilities
    Device Caps : 0x04204000
    Video Memory-to-Memory Multiplanar
    Streaming
    Extended Pix Format
    Media Driver Info:
    Driver name : hantro-vpu
    Model : hantro-vpu
    Serial :
    Bus info : platform: hantro-vpu
    Media version : 5.18.0
    Hardware revision: 0x00000000 (0)
    Driver version : 5.18.0
    Interface Info:
    ID : 0x0300000c
    Type : V4L Video
    Entity Info:
    ID : 0x00000001 (1)
    Name : nxp,imx8mq-vpu-g2-dec-source
    Function : V4L2 I/O
    Pad 0x01000002 : 0: Source
    Link 0x02000008: to remote pad 0x1000004 of entity
    'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable
    Priority: 2
    Format Video Capture Multiplanar:
    Width/Height : 48/48
    Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear))
    Field : None
    Number of planes : 1
    Flags :
    Colorspace : JPEG
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization : Default
    Plane 0 :
    Bytes per Line : 48
    Size Image : 3600
    Format Video Output Multiplanar:
    Width/Height : 48/48
    Pixel Format : 'S265' (HEVC Parsed Slice Data)
    Field : None
    Number of planes : 1
    Flags :
    Colorspace : JPEG
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization : Default
    Plane 0 :
    Bytes per Line : 0
    Size Image : 4608

    User Controls

    hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256
    step=1 default=0 value=0

    Codec Controls

    hevc_profile 0x00990b67 (menu) : min=0 max=2
    default=0 value=0
    0: Main
    1: Main Still Picture
    2: Main 10
    hevc_level 0x00990b68 (menu) : min=0 max=8
    default=0 value=0
    0: 1
    1: 2
    2: 2.1
    3: 3
    4: 3.1
    5: 4
    6: 4.1
    7: 5
    8: 5.1

    Stateless Codec Controls

    vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload
    vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload
    hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload
    hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload
    hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload
    hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload
    hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1
    default=1 value=1
    1: Frame-Based
    hevc_start_code 0x00a40a96 (menu) : min=1 max=1
    default=1 value=1
    1: Annex B Start Code

    > >
    > >> Benjamin
    > >>
    > >>> adam
    > >>>> Benjamin
    > >>>>
    > >>>>> adam
    > >>>>>> adam
    > >>>>>>> Regards,
    > >>>>>>> Benjamin
    > >>>>>>>
    > >>>>>>>>> adam
    > >>>>>>>>>> Thanks,
    > >>>>>>>>>> Benjamin
    > >>>>>>>>>>
    > >>>>>>>>>>> Benjamin
    > >>>>>>>>>>>
    > >>>>>>>>>>>
    > >>>>>>>>>>> Benjamin Gaignard (12):
    > >>>>>>>>>>> media: uapi: HEVC: Add missing fields in HEVC controls
    > >>>>>>>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
    > >>>>>>>>>>> prefix
    > >>>>>>>>>>> media: uapi: HEVC: Add document uAPI structure
    > >>>>>>>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS
    > >>>>>>>>>>> as a
    > >>>>>>>>>>> dynamic array
    > >>>>>>>>>>> media: uapi: Move parsed HEVC pixel format out of staging
    > >>>>>>>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS
    > >>>>>>>>>>> control
    > >>>>>>>>>>> media: uapi: Move the HEVC stateless control type out of staging
    > >>>>>>>>>>> media: controls: Log HEVC stateless control in .std_log
    > >>>>>>>>>>> media: uapi: Create a dedicated header for Hantro control
    > >>>>>>>>>>> media: uapi: HEVC: fix padding in v4l2 control structures
    > >>>>>>>>>>> media: uapi: Change data_bit_offset definition
    > >>>>>>>>>>> media: uapi: move HEVC stateless controls out of staging
    > >>>>>>>>>>>
    > >>>>>>>>>>> Hans Verkuil (3):
    > >>>>>>>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
    > >>>>>>>>>>> v4l2-ctrls: add support for dynamically allocated arrays.
    > >>>>>>>>>>> vivid: add dynamic array test control
    > >>>>>>>>>>>
    > >>>>>>>>>>> .../userspace-api/media/drivers/hantro.rst | 5 -
    > >>>>>>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 833
    > >>>>>>>>>>> ++++++++++++++++++
    > >>>>>>>>>>> .../media/v4l/ext-ctrls-codec.rst | 780
    > >>>>>>>>>>> ----------------
    > >>>>>>>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +-
    > >>>>>>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
    > >>>>>>>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 +
    > >>>>>>>>>>> .../media/videodev2.h.rst.exceptions | 5 +
    > >>>>>>>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
    > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
    > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
    > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 37 +-
    > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
    > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
    > >>>>>>>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
    > >>>>>>>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
    > >>>>>>>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
    > >>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
    > >>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 2 +-
    > >>>>>>>>>>> include/media/hevc-ctrls.h | 250 ------
    > >>>>>>>>>>> include/media/v4l2-ctrls.h | 48 +-
    > >>>>>>>>>>> include/uapi/linux/hantro-media.h | 19 +
    > >>>>>>>>>>> include/uapi/linux/v4l2-controls.h | 439 +++++++++
    > >>>>>>>>>>> include/uapi/linux/videodev2.h | 13 +
    > >>>>>>>>>>> 23 files changed, 1697 insertions(+), 1170 deletions(-)
    > >>>>>>>>>>> delete mode 100644 include/media/hevc-ctrls.h
    > >>>>>>>>>>> create mode 100644 include/uapi/linux/hantro-media.h
    > >>>>>>>>>>>

    \
     
     \ /
      Last update: 2022-04-06 17:42    [W:3.397 / U:0.024 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site