aboutsummaryrefslogtreecommitdiff
path: root/Documentation/userspace-api/media/v4l/dev-subdev.rst
diff options
context:
space:
mode:
authorLibravatar Linus Torvalds <torvalds@linux-foundation.org>2023-02-21 18:24:12 -0800
committerLibravatar Linus Torvalds <torvalds@linux-foundation.org>2023-02-21 18:24:12 -0800
commit5b7c4cabbb65f5c469464da6c5f614cbd7f730f2 (patch)
treecc5c2d0a898769fd59549594fedb3ee6f84e59a0 /Documentation/userspace-api/media/v4l/dev-subdev.rst
downloadlinux-5b7c4cabbb65f5c469464da6c5f614cbd7f730f2.tar.gz
linux-5b7c4cabbb65f5c469464da6c5f614cbd7f730f2.zip
Merge tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-nextgrafted
Pull networking updates from Jakub Kicinski: "Core: - Add dedicated kmem_cache for typical/small skb->head, avoid having to access struct page at kfree time, and improve memory use. - Introduce sysctl to set default RPS configuration for new netdevs. - Define Netlink protocol specification format which can be used to describe messages used by each family and auto-generate parsers. Add tools for generating kernel data structures and uAPI headers. - Expose all net/core sysctls inside netns. - Remove 4s sleep in netpoll if carrier is instantly detected on boot. - Add configurable limit of MDB entries per port, and port-vlan. - Continue populating drop reasons throughout the stack. - Retire a handful of legacy Qdiscs and classifiers. Protocols: - Support IPv4 big TCP (TSO frames larger than 64kB). - Add IP_LOCAL_PORT_RANGE socket option, to control local port range on socket by socket basis. - Track and report in procfs number of MPTCP sockets used. - Support mixing IPv4 and IPv6 flows in the in-kernel MPTCP path manager. - IPv6: don't check net.ipv6.route.max_size and rely on garbage collection to free memory (similarly to IPv4). - Support Penultimate Segment Pop (PSP) flavor in SRv6 (RFC8986). - ICMP: add per-rate limit counters. - Add support for user scanning requests in ieee802154. - Remove static WEP support. - Support minimal Wi-Fi 7 Extremely High Throughput (EHT) rate reporting. - WiFi 7 EHT channel puncturing support (client & AP). BPF: - Add a rbtree data structure following the "next-gen data structure" precedent set by recently added linked list, that is, by using kfunc + kptr instead of adding a new BPF map type. - Expose XDP hints via kfuncs with initial support for RX hash and timestamp metadata. - Add BPF_F_NO_TUNNEL_KEY extension to bpf_skb_set_tunnel_key to better support decap on GRE tunnel devices not operating in collect metadata. - Improve x86 JIT's codegen for PROBE_MEM runtime error checks. - Remove the need for trace_printk_lock for bpf_trace_printk and bpf_trace_vprintk helpers. - Extend libbpf's bpf_tracing.h support for tracing arguments of kprobes/uprobes and syscall as a special case. - Significantly reduce the search time for module symbols by livepatch and BPF. - Enable cpumasks to be used as kptrs, which is useful for tracing programs tracking which tasks end up running on which CPUs in different time intervals. - Add support for BPF trampoline on s390x and riscv64. - Add capability to export the XDP features supported by the NIC. - Add __bpf_kfunc tag for marking kernel functions as kfuncs. - Add cgroup.memory=nobpf kernel parameter option to disable BPF memory accounting for container environments. Netfilter: - Remove the CLUSTERIP target. It has been marked as obsolete for years, and we still have WARN splats wrt races of the out-of-band /proc interface installed by this target. - Add 'destroy' commands to nf_tables. They are identical to the existing 'delete' commands, but do not return an error if the referenced object (set, chain, rule...) did not exist. Driver API: - Improve cpumask_local_spread() locality to help NICs set the right IRQ affinity on AMD platforms. - Separate C22 and C45 MDIO bus transactions more clearly. - Introduce new DCB table to control DSCP rewrite on egress. - Support configuration of Physical Layer Collision Avoidance (PLCA) Reconciliation Sublayer (RS) (802.3cg-2019). Modern version of shared medium Ethernet. - Support for MAC Merge layer (IEEE 802.3-2018 clause 99). Allowing preemption of low priority frames by high priority frames. - Add support for controlling MACSec offload using netlink SET. - Rework devlink instance refcounts to allow registration and de-registration under the instance lock. Split the code into multiple files, drop some of the unnecessarily granular locks and factor out common parts of netlink operation handling. - Add TX frame aggregation parameters (for USB drivers). - Add a new attr TCA_EXT_WARN_MSG to report TC (offload) warning messages with notifications for debug. - Allow offloading of UDP NEW connections via act_ct. - Add support for per action HW stats in TC. - Support hardware miss to TC action (continue processing in SW from a specific point in the action chain). - Warn if old Wireless Extension user space interface is used with modern cfg80211/mac80211 drivers. Do not support Wireless Extensions for Wi-Fi 7 devices at all. Everyone should switch to using nl80211 interface instead. - Improve the CAN bit timing configuration. Use extack to return error messages directly to user space, update the SJW handling, including the definition of a new default value that will benefit CAN-FD controllers, by increasing their oscillator tolerance. New hardware / drivers: - Ethernet: - nVidia BlueField-3 support (control traffic driver) - Ethernet support for imx93 SoCs - Motorcomm yt8531 gigabit Ethernet PHY - onsemi NCN26000 10BASE-T1S PHY (with support for PLCA) - Microchip LAN8841 PHY (incl. cable diagnostics and PTP) - Amlogic gxl MDIO mux - WiFi: - RealTek RTL8188EU (rtl8xxxu) - Qualcomm Wi-Fi 7 devices (ath12k) - CAN: - Renesas R-Car V4H Drivers: - Bluetooth: - Set Per Platform Antenna Gain (PPAG) for Intel controllers. - Ethernet NICs: - Intel (1G, igc): - support TSN / Qbv / packet scheduling features of i226 model - Intel (100G, ice): - use GNSS subsystem instead of TTY - multi-buffer XDP support - extend support for GPIO pins to E823 devices - nVidia/Mellanox: - update the shared buffer configuration on PFC commands - implement PTP adjphase function for HW offset control - TC support for Geneve and GRE with VF tunnel offload - more efficient crypto key management method - multi-port eswitch support - Netronome/Corigine: - add DCB IEEE support - support IPsec offloading for NFP3800 - Freescale/NXP (enetc): - support XDP_REDIRECT for XDP non-linear buffers - improve reconfig, avoid link flap and waiting for idle - support MAC Merge layer - Other NICs: - sfc/ef100: add basic devlink support for ef100 - ionic: rx_push mode operation (writing descriptors via MMIO) - bnxt: use the auxiliary bus abstraction for RDMA - r8169: disable ASPM and reset bus in case of tx timeout - cpsw: support QSGMII mode for J721e CPSW9G - cpts: support pulse-per-second output - ngbe: add an mdio bus driver - usbnet: optimize usbnet_bh() by avoiding unnecessary queuing - r8152: handle devices with FW with NCM support - amd-xgbe: support 10Mbps, 2.5GbE speeds and rx-adaptation - virtio-net: support multi buffer XDP - virtio/vsock: replace virtio_vsock_pkt with sk_buff - tsnep: XDP support - Ethernet high-speed switches: - nVidia/Mellanox (mlxsw): - add support for latency TLV (in FW control messages) - Microchip (sparx5): - separate explicit and implicit traffic forwarding rules, make the implicit rules always active - add support for egress DSCP rewrite - IS0 VCAP support (Ingress Classification) - IS2 VCAP filters (protos, L3 addrs, L4 ports, flags, ToS etc.) - ES2 VCAP support (Egress Access Control) - support for Per-Stream Filtering and Policing (802.1Q, 8.6.5.1) - Ethernet embedded switches: - Marvell (mv88e6xxx): - add MAB (port auth) offload support - enable PTP receive for mv88e6390 - NXP (ocelot): - support MAC Merge layer - support for the the vsc7512 internal copper phys - Microchip: - lan9303: convert to PHYLINK - lan966x: support TC flower filter statistics - lan937x: PTP support for KSZ9563/KSZ8563 and LAN937x - lan937x: support Credit Based Shaper configuration - ksz9477: support Energy Efficient Ethernet - other: - qca8k: convert to regmap read/write API, use bulk operations - rswitch: Improve TX timestamp accuracy - Intel WiFi (iwlwifi): - EHT (Wi-Fi 7) rate reporting - STEP equalizer support: transfer some STEP (connection to radio on platforms with integrated wifi) related parameters from the BIOS to the firmware. - Qualcomm 802.11ax WiFi (ath11k): - IPQ5018 support - Fine Timing Measurement (FTM) responder role support - channel 177 support - MediaTek WiFi (mt76): - per-PHY LED support - mt7996: EHT (Wi-Fi 7) support - Wireless Ethernet Dispatch (WED) reset support - switch to using page pool allocator - RealTek WiFi (rtw89): - support new version of Bluetooth co-existance - Mobile: - rmnet: support TX aggregation" * tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1872 commits) page_pool: add a comment explaining the fragment counter usage net: ethtool: fix __ethtool_dev_mm_supported() implementation ethtool: pse-pd: Fix double word in comments xsk: add linux/vmalloc.h to xsk.c sefltests: netdevsim: wait for devlink instance after netns removal selftest: fib_tests: Always cleanup before exit net/mlx5e: Align IPsec ASO result memory to be as required by hardware net/mlx5e: TC, Set CT miss to the specific ct action instance net/mlx5e: Rename CHAIN_TO_REG to MAPPED_OBJ_TO_REG net/mlx5: Refactor tc miss handling to a single function net/mlx5: Kconfig: Make tc offload depend on tc skb extension net/sched: flower: Support hardware miss to tc action net/sched: flower: Move filter handle initialization earlier net/sched: cls_api: Support hardware miss to tc action net/sched: Rename user cookie and act cookie sfc: fix builds without CONFIG_RTC_LIB sfc: clean up some inconsistent indentings net/mlx4_en: Introduce flexible array to silence overflow warning net: lan966x: Fix possible deadlock inside PTP net/ulp: Remove redundant ->clone() test in inet_clone_ulp(). ...
Diffstat (limited to 'Documentation/userspace-api/media/v4l/dev-subdev.rst')
-rw-r--r--Documentation/userspace-api/media/v4l/dev-subdev.rst503
1 files changed, 503 insertions, 0 deletions
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
new file mode 100644
index 000000000..fd1de0a73
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -0,0 +1,503 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _subdev:
+
+********************
+Sub-device Interface
+********************
+
+The complex nature of V4L2 devices, where hardware is often made of
+several integrated circuits that need to interact with each other in a
+controlled way, leads to complex V4L2 drivers. The drivers usually
+reflect the hardware model in software, and model the different hardware
+components as software blocks called sub-devices.
+
+V4L2 sub-devices are usually kernel-only objects. If the V4L2 driver
+implements the media device API, they will automatically inherit from
+media entities. Applications will be able to enumerate the sub-devices
+and discover the hardware topology using the media entities, pads and
+links enumeration API.
+
+In addition to make sub-devices discoverable, drivers can also choose to
+make them directly configurable by applications. When both the
+sub-device driver and the V4L2 device driver support this, sub-devices
+will feature a character device node on which ioctls can be called to
+
+- query, read and write sub-devices controls
+
+- subscribe and unsubscribe to events and retrieve them
+
+- negotiate image formats on individual pads
+
+Sub-device character device nodes, conventionally named
+``/dev/v4l-subdev*``, use major number 81.
+
+Drivers may opt to limit the sub-device character devices to only expose
+operations that do not modify the device state. In such a case the sub-devices
+are referred to as ``read-only`` in the rest of this documentation, and the
+related restrictions are documented in individual ioctls.
+
+
+Controls
+========
+
+Most V4L2 controls are implemented by sub-device hardware. Drivers
+usually merge all controls and expose them through video device nodes.
+Applications can control all sub-devices through a single interface.
+
+Complex devices sometimes implement the same control in different pieces
+of hardware. This situation is common in embedded platforms, where both
+sensors and image processing hardware implement identical functions,
+such as contrast adjustment, white balance or faulty pixels correction.
+As the V4L2 controls API doesn't support several identical controls in a
+single device, all but one of the identical controls are hidden.
+
+Applications can access those hidden controls through the sub-device
+node with the V4L2 control API described in :ref:`control`. The ioctls
+behave identically as when issued on V4L2 device nodes, with the
+exception that they deal only with controls implemented in the
+sub-device.
+
+Depending on the driver, those controls might also be exposed through
+one (or several) V4L2 device nodes.
+
+
+Events
+======
+
+V4L2 sub-devices can notify applications of events as described in
+:ref:`event`. The API behaves identically as when used on V4L2 device
+nodes, with the exception that it only deals with events generated by
+the sub-device. Depending on the driver, those events might also be
+reported on one (or several) V4L2 device nodes.
+
+
+.. _pad-level-formats:
+
+Pad-level Formats
+=================
+
+.. warning::
+
+ Pad-level formats are only applicable to very complex devices that
+ need to expose low-level format configuration to user space. Generic
+ V4L2 applications do *not* need to use the API described in this
+ section.
+
+.. note::
+
+ For the purpose of this section, the term *format* means the
+ combination of media bus data format, frame width and frame height.
+
+Image formats are typically negotiated on video capture and output
+devices using the format and
+:ref:`selection <VIDIOC_SUBDEV_G_SELECTION>` ioctls. The driver is
+responsible for configuring every block in the video pipeline according
+to the requested format at the pipeline input and/or output.
+
+For complex devices, such as often found in embedded systems, identical
+image sizes at the output of a pipeline can be achieved using different
+hardware configurations. One such example is shown on
+:ref:`pipeline-scaling`, where image scaling can be performed on both
+the video sensor and the host image processing hardware.
+
+
+.. _pipeline-scaling:
+
+.. kernel-figure:: pipeline.dot
+ :alt: pipeline.dot
+ :align: center
+
+ Image Format Negotiation on Pipelines
+
+ High quality and high speed pipeline configuration
+
+
+
+The sensor scaler is usually of less quality than the host scaler, but
+scaling on the sensor is required to achieve higher frame rates.
+Depending on the use case (quality vs. speed), the pipeline must be
+configured differently. Applications need to configure the formats at
+every point in the pipeline explicitly.
+
+Drivers that implement the :ref:`media API <media-controller-intro>`
+can expose pad-level image format configuration to applications. When
+they do, applications can use the
+:ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` and
+:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls. to
+negotiate formats on a per-pad basis.
+
+Applications are responsible for configuring coherent parameters on the
+whole pipeline and making sure that connected pads have compatible
+formats. The pipeline is checked for formats mismatch at
+:ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>` time, and an ``EPIPE`` error
+code is then returned if the configuration is invalid.
+
+Pad-level image format configuration support can be tested by calling
+the :ref:`VIDIOC_SUBDEV_G_FMT` ioctl on pad
+0. If the driver returns an ``EINVAL`` error code pad-level format
+configuration is not supported by the sub-device.
+
+
+Format Negotiation
+------------------
+
+Acceptable formats on pads can (and usually do) depend on a number of
+external parameters, such as formats on other pads, active links, or
+even controls. Finding a combination of formats on all pads in a video
+pipeline, acceptable to both application and driver, can't rely on
+formats enumeration only. A format negotiation mechanism is required.
+
+Central to the format negotiation mechanism are the get/set format
+operations. When called with the ``which`` argument set to
+:ref:`V4L2_SUBDEV_FORMAT_TRY <VIDIOC_SUBDEV_G_FMT>`, the
+:ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` and
+:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls operate on
+a set of formats parameters that are not connected to the hardware
+configuration. Modifying those 'try' formats leaves the device state
+untouched (this applies to both the software state stored in the driver
+and the hardware state stored in the device itself).
+
+While not kept as part of the device state, try formats are stored in
+the sub-device file handles. A
+:ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` call will return
+the last try format set *on the same sub-device file handle*. Several
+applications querying the same sub-device at the same time will thus not
+interact with each other.
+
+To find out whether a particular format is supported by the device,
+applications use the
+:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctl. Drivers
+verify and, if needed, change the requested ``format`` based on device
+requirements and return the possibly modified value. Applications can
+then choose to try a different format or accept the returned value and
+continue.
+
+Formats returned by the driver during a negotiation iteration are
+guaranteed to be supported by the device. In particular, drivers
+guarantee that a returned format will not be further changed if passed
+to an :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` call as-is
+(as long as external parameters, such as formats on other pads or links'
+configuration are not changed).
+
+Drivers automatically propagate formats inside sub-devices. When a try
+or active format is set on a pad, corresponding formats on other pads of
+the same sub-device can be modified by the driver. Drivers are free to
+modify formats as required by the device. However, they should comply
+with the following rules when possible:
+
+- Formats should be propagated from sink pads to source pads. Modifying
+ a format on a source pad should not modify the format on any sink
+ pad.
+
+- Sub-devices that scale frames using variable scaling factors should
+ reset the scale factors to default values when sink pads formats are
+ modified. If the 1:1 scaling ratio is supported, this means that
+ source pads formats should be reset to the sink pads formats.
+
+Formats are not propagated across links, as that would involve
+propagating them from one sub-device file handle to another.
+Applications must then take care to configure both ends of every link
+explicitly with compatible formats. Identical formats on the two ends of
+a link are guaranteed to be compatible. Drivers are free to accept
+different formats matching device requirements as being compatible.
+
+:ref:`sample-pipeline-config` shows a sample configuration sequence
+for the pipeline described in :ref:`pipeline-scaling` (table columns
+list entity names and pad numbers).
+
+
+.. raw:: latex
+
+ \begingroup
+ \scriptsize
+ \setlength{\tabcolsep}{2pt}
+
+.. tabularcolumns:: |p{2.0cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|
+
+.. _sample-pipeline-config:
+
+.. flat-table:: Sample Pipeline Configuration
+ :header-rows: 1
+ :stub-columns: 0
+ :widths: 5 5 5 5 5 5 5
+
+ * -
+ - Sensor/0
+
+ format
+ - Frontend/0
+
+ format
+ - Frontend/1
+
+ format
+ - Scaler/0
+
+ format
+ - Scaler/0
+
+ compose selection rectangle
+ - Scaler/1
+
+ format
+ * - Initial state
+ - 2048x1536
+
+ SGRBG8_1X8
+ - (default)
+ - (default)
+ - (default)
+ - (default)
+ - (default)
+ * - Configure frontend sink format
+ - 2048x1536
+
+ SGRBG8_1X8
+ - *2048x1536*
+
+ *SGRBG8_1X8*
+ - *2046x1534*
+
+ *SGRBG8_1X8*
+ - (default)
+ - (default)
+ - (default)
+ * - Configure scaler sink format
+ - 2048x1536
+
+ SGRBG8_1X8
+ - 2048x1536
+
+ SGRBG8_1X8
+ - 2046x1534
+
+ SGRBG8_1X8
+ - *2046x1534*
+
+ *SGRBG8_1X8*
+ - *0,0/2046x1534*
+ - *2046x1534*
+
+ *SGRBG8_1X8*
+ * - Configure scaler sink compose selection
+ - 2048x1536
+
+ SGRBG8_1X8
+ - 2048x1536
+
+ SGRBG8_1X8
+ - 2046x1534
+
+ SGRBG8_1X8
+ - 2046x1534
+
+ SGRBG8_1X8
+ - *0,0/1280x960*
+ - *1280x960*
+
+ *SGRBG8_1X8*
+
+.. raw:: latex
+
+ \endgroup
+
+1. Initial state. The sensor source pad format is set to its native 3MP
+ size and V4L2_MBUS_FMT_SGRBG8_1X8 media bus code. Formats on the
+ host frontend and scaler sink and source pads have the default
+ values, as well as the compose rectangle on the scaler's sink pad.
+
+2. The application configures the frontend sink pad format's size to
+ 2048x1536 and its media bus code to V4L2_MBUS_FMT_SGRBG_1X8. The
+ driver propagates the format to the frontend source pad.
+
+3. The application configures the scaler sink pad format's size to
+ 2046x1534 and the media bus code to V4L2_MBUS_FMT_SGRBG_1X8 to
+ match the frontend source size and media bus code. The media bus code
+ on the sink pad is set to V4L2_MBUS_FMT_SGRBG_1X8. The driver
+ propagates the size to the compose selection rectangle on the
+ scaler's sink pad, and the format to the scaler source pad.
+
+4. The application configures the size of the compose selection
+ rectangle of the scaler's sink pad 1280x960. The driver propagates
+ the size to the scaler's source pad format.
+
+When satisfied with the try results, applications can set the active
+formats by setting the ``which`` argument to
+``V4L2_SUBDEV_FORMAT_ACTIVE``. Active formats are changed exactly as try
+formats by drivers. To avoid modifying the hardware state during format
+negotiation, applications should negotiate try formats first and then
+modify the active settings using the try formats returned during the
+last negotiation iteration. This guarantees that the active format will
+be applied as-is by the driver without being modified.
+
+
+.. _v4l2-subdev-selections:
+
+Selections: cropping, scaling and composition
+---------------------------------------------
+
+Many sub-devices support cropping frames on their input or output pads
+(or possible even on both). Cropping is used to select the area of
+interest in an image, typically on an image sensor or a video decoder.
+It can also be used as part of digital zoom implementations to select
+the area of the image that will be scaled up.
+
+Crop settings are defined by a crop rectangle and represented in a
+struct :c:type:`v4l2_rect` by the coordinates of the top
+left corner and the rectangle size. Both the coordinates and sizes are
+expressed in pixels.
+
+As for pad formats, drivers store try and active rectangles for the
+selection targets :ref:`v4l2-selections-common`.
+
+On sink pads, cropping is applied relative to the current pad format.
+The pad format represents the image size as received by the sub-device
+from the previous block in the pipeline, and the crop rectangle
+represents the sub-image that will be transmitted further inside the
+sub-device for processing.
+
+The scaling operation changes the size of the image by scaling it to new
+dimensions. The scaling ratio isn't specified explicitly, but is implied
+from the original and scaled image sizes. Both sizes are represented by
+struct :c:type:`v4l2_rect`.
+
+Scaling support is optional. When supported by a subdev, the crop
+rectangle on the subdev's sink pad is scaled to the size configured
+using the
+:ref:`VIDIOC_SUBDEV_S_SELECTION <VIDIOC_SUBDEV_G_SELECTION>` IOCTL
+using ``V4L2_SEL_TGT_COMPOSE`` selection target on the same pad. If the
+subdev supports scaling but not composing, the top and left values are
+not used and must always be set to zero.
+
+On source pads, cropping is similar to sink pads, with the exception
+that the source size from which the cropping is performed, is the
+COMPOSE rectangle on the sink pad. In both sink and source pads, the
+crop rectangle must be entirely contained inside the source image size
+for the crop operation.
+
+The drivers should always use the closest possible rectangle the user
+requests on all selection targets, unless specifically told otherwise.
+``V4L2_SEL_FLAG_GE`` and ``V4L2_SEL_FLAG_LE`` flags may be used to round
+the image size either up or down. :ref:`v4l2-selection-flags`
+
+
+Types of selection targets
+--------------------------
+
+
+Actual targets
+^^^^^^^^^^^^^^
+
+Actual targets (without a postfix) reflect the actual hardware
+configuration at any point of time. There is a BOUNDS target
+corresponding to every actual target.
+
+
+BOUNDS targets
+^^^^^^^^^^^^^^
+
+BOUNDS targets is the smallest rectangle that contains all valid actual
+rectangles. It may not be possible to set the actual rectangle as large
+as the BOUNDS rectangle, however. This may be because e.g. a sensor's
+pixel array is not rectangular but cross-shaped or round. The maximum
+size may also be smaller than the BOUNDS rectangle.
+
+
+Order of configuration and format propagation
+---------------------------------------------
+
+Inside subdevs, the order of image processing steps will always be from
+the sink pad towards the source pad. This is also reflected in the order
+in which the configuration must be performed by the user: the changes
+made will be propagated to any subsequent stages. If this behaviour is
+not desired, the user must set ``V4L2_SEL_FLAG_KEEP_CONFIG`` flag. This
+flag causes no propagation of the changes are allowed in any
+circumstances. This may also cause the accessed rectangle to be adjusted
+by the driver, depending on the properties of the underlying hardware.
+
+The coordinates to a step always refer to the actual size of the
+previous step. The exception to this rule is the sink compose
+rectangle, which refers to the sink compose bounds rectangle --- if it
+is supported by the hardware.
+
+1. Sink pad format. The user configures the sink pad format. This format
+ defines the parameters of the image the entity receives through the
+ pad for further processing.
+
+2. Sink pad actual crop selection. The sink pad crop defines the crop
+ performed to the sink pad format.
+
+3. Sink pad actual compose selection. The size of the sink pad compose
+ rectangle defines the scaling ratio compared to the size of the sink
+ pad crop rectangle. The location of the compose rectangle specifies
+ the location of the actual sink compose rectangle in the sink compose
+ bounds rectangle.
+
+4. Source pad actual crop selection. Crop on the source pad defines crop
+ performed to the image in the sink compose bounds rectangle.
+
+5. Source pad format. The source pad format defines the output pixel
+ format of the subdev, as well as the other parameters with the
+ exception of the image width and height. Width and height are defined
+ by the size of the source pad actual crop selection.
+
+Accessing any of the above rectangles not supported by the subdev will
+return ``EINVAL``. Any rectangle referring to a previous unsupported
+rectangle coordinates will instead refer to the previous supported
+rectangle. For example, if sink crop is not supported, the compose
+selection will refer to the sink pad format dimensions instead.
+
+
+.. _subdev-image-processing-crop:
+
+.. kernel-figure:: subdev-image-processing-crop.svg
+ :alt: subdev-image-processing-crop.svg
+ :align: center
+
+ **Figure 4.5. Image processing in subdevs: simple crop example**
+
+In the above example, the subdev supports cropping on its sink pad. To
+configure it, the user sets the media bus format on the subdev's sink
+pad. Now the actual crop rectangle can be set on the sink pad --- the
+location and size of this rectangle reflect the location and size of a
+rectangle to be cropped from the sink format. The size of the sink crop
+rectangle will also be the size of the format of the subdev's source
+pad.
+
+
+.. _subdev-image-processing-scaling-multi-source:
+
+.. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
+ :alt: subdev-image-processing-scaling-multi-source.svg
+ :align: center
+
+ **Figure 4.6. Image processing in subdevs: scaling with multiple sources**
+
+In this example, the subdev is capable of first cropping, then scaling
+and finally cropping for two source pads individually from the resulting
+scaled image. The location of the scaled image in the cropped image is
+ignored in sink compose target. Both of the locations of the source crop
+rectangles refer to the sink scaling rectangle, independently cropping
+an area at location specified by the source crop rectangle from it.
+
+
+.. _subdev-image-processing-full:
+
+.. kernel-figure:: subdev-image-processing-full.svg
+ :alt: subdev-image-processing-full.svg
+ :align: center
+
+ **Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
+
+The subdev driver supports two sink pads and two source pads. The images
+from both of the sink pads are individually cropped, then scaled and
+further composed on the composition bounds rectangle. From that, two
+independent streams are cropped and sent out of the subdev from the
+source pads.
+
+
+.. toctree::
+ :maxdepth: 1
+
+ subdev-formats