Integrating MIPI sensors and cameras on SCAILX

The SCAILX AI platform is a modular compact vision AI Edge solution with various interface boards for connecting image sensors/cameras to the NXP IMX8M Plus processor. The interface for the sensors/cameras is via the 2 MIPI interfaces.

In this post, we’ll describe the principles and benefits of the MIPI interface and the challenges of integrating MIPI sensors/cameras on SCAILX. MIPI stands for Mobile Industry Processor Interface. It is a set of standards developed by the MIPI Alliance, a global organization that creates interface specifications for mobile and mobile-influenced devices. These standards are widely used in smartphones, tablets, IoT devices, automotive systems, and other embedded systems.

The MIPI CSI-2 specification is the most commonly used in embedded camera solutions. The CSI (Camera Serial Interface) connects sensors and cameras to processors. The interface supports high-speed data transmission for high-resolution video and image capture.

The MIPI standard is widely used due to its:

• low power consumption
• high data throughput
• compact design

MIPI cameras are widely used in smartphones, automotive applications, drones and robotics, wearables, and many more.

Major characteristics of MIPI CSI-2

• High data rates of up to typically 1.5 Gbps per lane (data wire pair);
• Scalable by using multiple lanes between sensor/camera and processor;
• Minimum of 2 wire pairs between sensor/camera and processor, typically 4+1 lanes;
• I²C communication for software control of sensor/camera;

• Compact design, thanks to its roots in the mobile phone market

• Low latency due to high speed direct connection;
• MIPI supports a wide range of video formats, including RAW, RGB, YUV, JPEG, etc

MIPI hardware and software requirements for integration on an embedded system

The embedded systems, like SCAILX, can support cameras with different interface standards, for example:

• USB cameras: UVC (Universal Video Class) drivers;
• IP cameras: Rtsp or other standard computer protocols.

However, a MIPI sensor/camera needs a dedicated driver for video capture, streaming and control. A MIPI sensor/camera is typically used in an embedded audio/video system, based on Linux. These devices are generally supported by a software framework called: Video-for-Linux (aka ‘v4l’). Video-4-Linux is standardized and is used by developers to communicate with audio/video devices. MIPI sensors/cameras rely on the Video-4-Linux framework to be integrated into an Embedded system. One downside to the MIPI standard is that is does not have a standard driver for communication with all major sensors, cameras or capture devices to stream the video/audio data to an application for processing. The MIPI sensor/camera needs the following parts to be integrated into a system:

• MIPI CSI-2 port, usually a connector on the processor board
• foil cable with a maximum length restriction, usually less than 40 cm
• a dedicated v4l-driver which is responsible for receiving the MIPI high-speed data packages, converting the packages into video frames in the processor, video buffering and control of the sensor/camera via I2C.
  
  

Connector interfaces of the SCAiLX SoM Board

RAW versus ISP-based MIPI sensors/cameras

In general, the MIPI standard supports various video formats. Unlike a USB and IP camera, the RAW MIPI video formats cannot be directly supported by the v4l framework. A hardware or software ISP (Image Signal Processor) is needed for all basic steps to convert the RAW pixel data into a high-quality image for viewing and analysis.

To effectively support a MIPI image sensor with RAW data output, an Image Signal Processor (ISP) is essential. This can be achieved through a dedicated hardware ISP, such as the one integrated into the SCAILX-2GS234-xY board camera, or by utilizing an ISP on the processor board.

In the case of a sensor module with an integrated ISP, like Omnivision OV5640-AF, or a sensor directly supported by Video-4-Linux, like Omnivison OS08A20, the integration may be significantly easier. Only the v4l driver will be required in the embedded system.

Both solutions depend heavily on software development of the ISP implementation.

SCAILX support for MIPI sensors/cameras

The SCAILX Edge AI platform is based on the NXP IMX8M Plus processor with AI. Capabilities include a modular embedded video platform with two MIPI CSI 4-lane ports, I2C communication, power, and GPIO (General Purpose Input/Output) pins.

The SCAiLX plate form supports the SCAILX-2GS234-xY board camera with Onsemi AR0234 Global Shutter sensor, onboard ISP, and a dedicated SCAILX ‘v4l’-driver. The MIPI-based FCB-EV9500M zoom block camera from Sony. In addition to direct MIPI cameras, the SCAiLX platform included an LVDS-2-MIPI converter board to support various zoom block cameras from Videology, Sony and Tamron.

From specification to integration, Videology has you covered

MIPI CSI-2 is an ideal interface when prioritizing high-speed, low power consumption, compact design, high-resolution, and real-time video.

While integration challenges can be complex, Videology’s expertise simplifies the process. Let us help you develop a robust, tailored solution for your edge AI camera needs. Contact us today to turn your vision into reality!