Xu,

 I am able to play that file properly using ffplay, when i use screenrecorder command to capture the screen to a h264 file ,

I tried configuring the MediaCodec encoder in java (https://developer.android.com/reference/android/media/MediaCodec) with similar parameters used in screenrecorder encoder configuration.

The video pipeline is, I am capturing YUYV422 ( Camera is returning only YUYV422 ) frames from Camera using v4l2 , executing MSDK VPP processing to convert the buffered frame to I420 and submitting it to MediaCodec encoder. 

I am writing the output h264 frame of the MediaCodec encoder to a file and also submitting it to MediaCodec Decoder.

The ffplay of the h264 file generated using the MediaCodec is not so clear and it is pixalated when compared to the h264 file generated using "screenrecorder".

As screenrecorder is in cpp , I am wondering , whether we have a java based screenrecorder,  that uses MediaCodec encoder configuration for comparison purposes.

Thanks,
Sathish

On Mon, Dec 17, 2018 at 9:45 AM Xu, Guangxin <guangxin.xu@intel.com> wrote:

Hi Sathish,

Could you share your screen recorder patch to us. So we can use your patch to reproduce the issue?

 

thanks

 

From: Sathish [mailto:satish.chevuru@gmail.com]
Sent: Saturday, December 15, 2018 1:09 AM
To: Xu, Guangxin <guangxin.xu@intel.com>
Cc: celadon@lists.01.org
Subject: Re: [01.org Celadon] Encoded Frame is not proper , Using Hardware Encoder through Media Codec

 

Hi,

 

I am using the below configurations for encoder and decoder and verified with screenrecord.cpp

 

format = MediaFormat.createVideoFormat(param.codec, param.width, param.height);

                                                          format.setInteger(MediaFormat.KEY_WIDTH, param.width);

                                                          format.setInteger(MediaFormat.KEY_HEIGHT, param.height);

                                                          format.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible);

                                                          format.setInteger(MediaFormat.KEY_BIT_RATE, 14000 * 1000); //trying with different bit rates

                                                          format.setInteger(MediaFormat.KEY_FRAME_RATE, 30);

                                                          format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 0);

                                                          format.setInteger(MediaFormat.KEY_PROFILE, CodecProfileLevel.AVCProfileHigh);

                                                          format.setInteger(MediaFormat.KEY_LEVEL, CodecProfileLevel.AVCLevel31);

                                                          encoderInstance.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);

 

12-11 22:29:04.508  3154  4229 I mediasdk_omx: mfx_omx_venc_component: Created OMX.Intel.hw_ve.h264

12-11 22:29:04.509  3154  4229 E OMXNodeInstance: setParameter(0xf6320cc0:Intel.hw_ve.h264, OMX.google.android.index.enableAndroidNativeBuffers(0x6ff00004): Output:1 en=0) ERROR: UnsupportedIndex(0x8000101a)

12-11 22:29:04.509  3154  4229 E OMXNodeInstance: setParameter(0xf6320cc0:Intel.hw_ve.h264, OMX.google.android.index.allocateNativeHandle(0x6ff00002): Output:1 en=0) ERROR: UnsupportedIndex(0x8000101a)

12-11 22:29:04.509  4431  7009 I ACodec  : [OMX.Intel.hw_ve.h264] using color format 0x15 in place of 0x7f420888

12-11 22:29:04.510  4431  7009 I ACodec  : setupAVCEncoderParameters with [profile: High] [level: Level31]

12-11 22:29:04.510  4431  7009 I ACodec  : [OMX.Intel.hw_ve.h264] cannot encode color aspects. Ignoring.

12-11 22:29:04.510  4431  7009 I ACodec  : [OMX.Intel.hw_ve.h264] cannot encode HDR static metadata. Ignoring.

12-11 22:29:04.510  4431  7009 I ACodec  : setupVideoEncoder succeeded

12-11 22:29:04.510  3154  4229 E OMXNodeInstance: getConfig(0xf6320cc0:Intel.hw_ve.h264, ConfigLatency(0x6f800005)) ERROR: UnsupportedIndex(0x8000101a)

12-11 22:29:04.510  3154  4229 E OMXNodeInstance: getConfig(0xf6320cc0:Intel.hw_ve.h264, ConfigAndroidVendorExtension(0x6f100004)) ERROR: UnsupportedIndex(0x8000101a)

12-11 22:29:04.511  4431  4431 I System.out: NUCRAW .. Initializing and configuring encoder stop

                                                decoderInstance = MediaCodec.createDecoderByType(codec);

                                                                      format = MediaFormat.createVideoFormat(codec, decoderWidth, decoderHeight);

                                                                      format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, decoderWidth * decoderHeight); 

                                                          format.setInteger("dohflip", doHflip ? 1 : 0);

                                                          format.setInteger("dovflip", doVflip ? 1 : 0);

                                                          decoderInstance.setVideoScalingMode(videoScalingMode); //MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT;

                                                          decoderInstance.configure(format, renderSurface, null, 0);

 

12-11 22:29:05.410  3154  6663 I mediasdk_omx: mfx_omx_vdec_component: Created OMX.Intel.hw_vd.h264

12-11 22:29:05.410  4431  4431 D TelyDecoder: Decoder Format: {mime=video/avc, width=1280, height=720}

12-11 22:29:05.411  4431  4431 D TelyDecoder: StartDecoder with Surface: Surface(name=null)/@0x62909d9 - 103352793

12-11 22:29:05.411  4431  7061 D SurfaceUtils: connecting to surface 0x79ef92dd2010, reason connectToSurface

12-11 22:29:05.411  4431  7061 I MediaCodec: [OMX.Intel.hw_vd.h264] setting surface generation to 4537345

12-11 22:29:05.411  4431  7061 D SurfaceUtils: disconnecting from surface 0x79ef92dd2010, reason connectToSurface(reconnect)

12-11 22:29:05.411  4431  7061 D SurfaceUtils: connecting to surface 0x79ef92dd2010, reason connectToSurface(reconnect)

12-11 22:29:05.411  3154  6663 E OMXNodeInstance: getConfig(0xf5e2b360:Intel.hw_vd.h264, ConfigAndroidVendorExtension(0x6f100004)) ERROR: UnsupportedIndex(0x8000101a)

 

But still having the same issue, do we see any issues with the config logs above ?

 

Thanks,

Sathish

 

On Mon, Dec 10, 2018 at 10:28 AM Xu, Guangxin <guangxin.xu@intel.com> wrote:

Hi Sathish,

Could you  refer to google’s screen recorder for your param?

Or could you reproduce your issue using screen recorder?

It will speedup our debug process.

thanks

 

http://androidxref.com/9.0.0_r3/xref/frameworks/av/cmds/screenrecord/screenrecord.cpp

 

From: Celadon [mailto:celadon-bounces@lists.01.org] On Behalf Of Sathish
Sent: Sunday, December 9, 2018 6:27 AM
To: celadon@lists.01.org
Subject: [01.org Celadon] Encoded Frame is not proper , Using Hardware Encoder through Media Codec

 

Hi,

 

As part of a video app , I am using  Hardware Encoder/Decoder through Media Codec.

 

The camera is generating YUYV422 Raw frame , I am converting it to YUV420 using Media SDK VPP Processing.

and submitting the YUV420 frame to the Media Codec Encoder.

 

The Encoder is configured as H264,  1080p , COLOR_FormatYUV420Flexible, 1500Kbps, gop 0, Baseline , AVCLevel31, and fps 30.

 

I am submitting the encoded frame to Decoder , But this is not being rendered on the surface.

 

I tried playing the encoded frame using ffplay , but the quality is very poor but no errors. The YUYV422 and YUV420 frames looks good when played through ffplay.

 

It seems the NALReference IDC value is 01 instead of 11 indicating of a B frame.

 

I see the below codes in the generated H264 file , 00 00 00  01 25 being repeated most of the time.

00 00 00 01  27

00 00  00 01 28

00 00 00  01 25

 

could you please let me know if there are any configuration change's that I am missing in  Encoder ? Is there any configuration to avoid generating B frames ?

 

Appreciate any inputs on this .

 

Thanks,
Sathish