Hello,

I am currently working with an NVIDIA Jetson Orin Nano and have installed JetPack 6. I would like to know if the real-time kernel is available with JetPack 6 by default, or if I need to follow the custom kernel installation guide as outlined in the documentation to enable real-time kernel support.

Additionally, I have connected a CSI camera and am using GStreamer with OpenCV to capture and display live video at 60 fps. However, I have observed that the frames are not arriving at consistent 16ms intervals as expected. Instead, I am seeing significant variations, with frames arriving at intervals such as 3ms or 21ms.

I have tested this behavior using both Python and C, and in both cases, the frame intervals remain inconsistent. I am trying to determine whether this issue is related to the hardware, such as the CSI camera or the Jetson platform itself, or if it is a software issue that might be addressed by using a real-time kernel or other optimizations.

Could you provide guidance on whether the real-time kernel would help achieve consistent frame timing, or if there are other steps I should take to diagnose and resolve this issue?

Thank you for your assistance!

Hi,
If you use developer kit, you can install the rt-kernel deb. Please check

Software Packages and the Update Mechanism — NVIDIA Jetson Linux Developer Guide 1 documentation

For checking performance of camera source, you can try gst-launch-1.0 with fpsdisplaysink:

$ gst-launch-1.0 -v nvarguscamerasrc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0

To check if the source generates frames in steady frame rate.

1 Like

Hi, @DaneLLL

Your solution was very helpful; thank you! I confirmed that the GStreamer was receiving the camera frames at an average of about 60.3fps when set to 60fps.

I installed the RT kernel as per the guide, but I didn’t notice a significant improvement in real-time performance compared to the regular kernel. I’m not sure in which areas the RT kernel is supposed to be more effective.

Additionally, I have one more question:

# MIT License
# Copyright (c) 2019-2022 JetsonHacks

# Using a CSI camera (such as the Raspberry Pi Version 2) connected to a
# NVIDIA Jetson Nano Developer Kit using OpenCV
# Drivers for the camera and OpenCV are included in the base image

import cv2
import time

"""
gstreamer_pipeline returns a GStreamer pipeline for capturing from the CSI camera
Flip the image by setting the flip_method (most common values: 0 and 2)
display_width and display_height determine the size of each camera pane in the window on the screen
Default 1920x1080 displayd in a 1/4 size window
"""

def gstreamer_pipeline(
   sensor_id=0,
   capture_width=1280,
   capture_height=720,
   display_width=640,
   display_height=360,
   framerate=60,
   flip_method=2,
):
   return (
       "nvarguscamerasrc sensor-id=%d ! "
       "video/x-raw(memory:NVMM), width=(int)%d, height=(int)%d, framerate=(fraction)%d/1 ! "
       "nvvidconv flip-method=%d ! "
       "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
       "videoconvert ! "
       "video/x-raw, format=(string)BGR ! appsink"
       % (
           sensor_id,
           capture_width,
           capture_height,
           framerate,
           flip_method,
           display_width,
           display_height,
       )
   )

def show_camera():
   window_title = "CSI Camera"
   prev_time = 0

   # To flip the image, modify the flip_method parameter (0 and 2 are the most common)
   print(gstreamer_pipeline(flip_method=2))
   video_capture = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER)
   if video_capture.isOpened():
       try:
           window_handle = cv2.namedWindow(window_title, cv2.WINDOW_AUTOSIZE)
           while True:
               ret_val, frame = video_capture.read()

               current_time = time.perf_counter()
               sec = current_time - prev_time
               prev_time = current_time
               print(1/sec)

               if cv2.getWindowProperty(window_title, cv2.WND_PROP_AUTOSIZE) >= 0:
                   cv2.imshow(window_title, frame)
               else:
                   break
               keyCode = cv2.waitKey(16) & 0xFF
               # Stop the program on the ESC key or 'q'
               if keyCode == 27 or keyCode == ord('q'):
                   break
       finally:
           video_capture.release()
           cv2.destroyAllWindows()
   else:
       print("Error: Unable to open camera")

if __name__ == "__main__":
   show_camera()

I’m testing this code in Python. cv2.imshow() is affected by waitKey and updates the display every 16ms, right? I used the time.perf_counter() function to measure the time for the cv2.imshow part. When I converted this time to fps, I got an average of about 55fps. Why do you think this is happening?

I need to accurately receive real-time video data at 60fps. Can you help me with this?

Hi,
Please run sudo jetson_clocks. OpenCV uses frame data in BGR format so the CPU capability dominates the performance. Please disable dynamic frequency scaling to run the cores at maximum clock. For optimal performance, we would suggest not to use OpenCV and have frame data in NVMM buffer(NvBufSurface) from source to sink.

Not sure but OpenCV application may not have performance deviation between Kernel and RT Kernel. This is our experience and may not be precise. Would see if other users can share experience.

Hi, @DaneLLL

I followed all your suggestions, including running sudo jetson_clocks and disabling dynamic frequency scaling, but I’m still not achieving the desired 16ms per frame for 60fps. Without using OpenCV, I’m not sure how to save the video frames. I’m currently opening the camera and capturing the stopwatch screen to check the time difference for each frame. How can I resolve this issue?

Hi,
Please check if you can achieve target frame rate in running gst-launch-1.0 command. To ensure the camera source can achieve the performance.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.