Merge tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-nextgrafted

Pull networking updates from Jakub Kicinski:
 "Core:

   - Add dedicated kmem_cache for typical/small skb->head, avoid having
     to access struct page at kfree time, and improve memory use.

   - Introduce sysctl to set default RPS configuration for new netdevs.

   - Define Netlink protocol specification format which can be used to
     describe messages used by each family and auto-generate parsers.
     Add tools for generating kernel data structures and uAPI headers.

   - Expose all net/core sysctls inside netns.

   - Remove 4s sleep in netpoll if carrier is instantly detected on
     boot.

   - Add configurable limit of MDB entries per port, and port-vlan.

   - Continue populating drop reasons throughout the stack.

   - Retire a handful of legacy Qdiscs and classifiers.

  Protocols:

   - Support IPv4 big TCP (TSO frames larger than 64kB).

   - Add IP_LOCAL_PORT_RANGE socket option, to control local port range
     on socket by socket basis.

   - Track and report in procfs number of MPTCP sockets used.

   - Support mixing IPv4 and IPv6 flows in the in-kernel MPTCP path
     manager.

   - IPv6: don't check net.ipv6.route.max_size and rely on garbage
     collection to free memory (similarly to IPv4).

   - Support Penultimate Segment Pop (PSP) flavor in SRv6 (RFC8986).

   - ICMP: add per-rate limit counters.

   - Add support for user scanning requests in ieee802154.

   - Remove static WEP support.

   - Support minimal Wi-Fi 7 Extremely High Throughput (EHT) rate
     reporting.

   - WiFi 7 EHT channel puncturing support (client & AP).

  BPF:

   - Add a rbtree data structure following the "next-gen data structure"
     precedent set by recently added linked list, that is, by using
     kfunc + kptr instead of adding a new BPF map type.

   - Expose XDP hints via kfuncs with initial support for RX hash and
     timestamp metadata.

   - Add BPF_F_NO_TUNNEL_KEY extension to bpf_skb_set_tunnel_key to
     better support decap on GRE tunnel devices not operating in collect
     metadata.

   - Improve x86 JIT's codegen for PROBE_MEM runtime error checks.

   - Remove the need for trace_printk_lock for bpf_trace_printk and
     bpf_trace_vprintk helpers.

   - Extend libbpf's bpf_tracing.h support for tracing arguments of
     kprobes/uprobes and syscall as a special case.

   - Significantly reduce the search time for module symbols by
     livepatch and BPF.

   - Enable cpumasks to be used as kptrs, which is useful for tracing
     programs tracking which tasks end up running on which CPUs in
     different time intervals.

   - Add support for BPF trampoline on s390x and riscv64.

   - Add capability to export the XDP features supported by the NIC.

   - Add __bpf_kfunc tag for marking kernel functions as kfuncs.

   - Add cgroup.memory=nobpf kernel parameter option to disable BPF
     memory accounting for container environments.

  Netfilter:

   - Remove the CLUSTERIP target. It has been marked as obsolete for
     years, and we still have WARN splats wrt races of the out-of-band
     /proc interface installed by this target.

   - Add 'destroy' commands to nf_tables. They are identical to the
     existing 'delete' commands, but do not return an error if the
     referenced object (set, chain, rule...) did not exist.

  Driver API:

   - Improve cpumask_local_spread() locality to help NICs set the right
     IRQ affinity on AMD platforms.

   - Separate C22 and C45 MDIO bus transactions more clearly.

   - Introduce new DCB table to control DSCP rewrite on egress.

   - Support configuration of Physical Layer Collision Avoidance (PLCA)
     Reconciliation Sublayer (RS) (802.3cg-2019). Modern version of
     shared medium Ethernet.

   - Support for MAC Merge layer (IEEE 802.3-2018 clause 99). Allowing
     preemption of low priority frames by high priority frames.

   - Add support for controlling MACSec offload using netlink SET.

   - Rework devlink instance refcounts to allow registration and
     de-registration under the instance lock. Split the code into
     multiple files, drop some of the unnecessarily granular locks and
     factor out common parts of netlink operation handling.

   - Add TX frame aggregation parameters (for USB drivers).

   - Add a new attr TCA_EXT_WARN_MSG to report TC (offload) warning
     messages with notifications for debug.

   - Allow offloading of UDP NEW connections via act_ct.

   - Add support for per action HW stats in TC.

   - Support hardware miss to TC action (continue processing in SW from
     a specific point in the action chain).

   - Warn if old Wireless Extension user space interface is used with
     modern cfg80211/mac80211 drivers. Do not support Wireless
     Extensions for Wi-Fi 7 devices at all. Everyone should switch to
     using nl80211 interface instead.

   - Improve the CAN bit timing configuration. Use extack to return
     error messages directly to user space, update the SJW handling,
     including the definition of a new default value that will benefit
     CAN-FD controllers, by increasing their oscillator tolerance.

  New hardware / drivers:

   - Ethernet:
      - nVidia BlueField-3 support (control traffic driver)
      - Ethernet support for imx93 SoCs
      - Motorcomm yt8531 gigabit Ethernet PHY
      - onsemi NCN26000 10BASE-T1S PHY (with support for PLCA)
      - Microchip LAN8841 PHY (incl. cable diagnostics and PTP)
      - Amlogic gxl MDIO mux

   - WiFi:
      - RealTek RTL8188EU (rtl8xxxu)
      - Qualcomm Wi-Fi 7 devices (ath12k)

   - CAN:
      - Renesas R-Car V4H

  Drivers:

   - Bluetooth:
      - Set Per Platform Antenna Gain (PPAG) for Intel controllers.

   - Ethernet NICs:
      - Intel (1G, igc):
         - support TSN / Qbv / packet scheduling features of i226 model
      - Intel (100G, ice):
         - use GNSS subsystem instead of TTY
         - multi-buffer XDP support
         - extend support for GPIO pins to E823 devices
      - nVidia/Mellanox:
         - update the shared buffer configuration on PFC commands
         - implement PTP adjphase function for HW offset control
         - TC support for Geneve and GRE with VF tunnel offload
         - more efficient crypto key management method
         - multi-port eswitch support
      - Netronome/Corigine:
         - add DCB IEEE support
         - support IPsec offloading for NFP3800
      - Freescale/NXP (enetc):
         - support XDP_REDIRECT for XDP non-linear buffers
         - improve reconfig, avoid link flap and waiting for idle
         - support MAC Merge layer
      - Other NICs:
         - sfc/ef100: add basic devlink support for ef100
         - ionic: rx_push mode operation (writing descriptors via MMIO)
         - bnxt: use the auxiliary bus abstraction for RDMA
         - r8169: disable ASPM and reset bus in case of tx timeout
         - cpsw: support QSGMII mode for J721e CPSW9G
         - cpts: support pulse-per-second output
         - ngbe: add an mdio bus driver
         - usbnet: optimize usbnet_bh() by avoiding unnecessary queuing
         - r8152: handle devices with FW with NCM support
         - amd-xgbe: support 10Mbps, 2.5GbE speeds and rx-adaptation
         - virtio-net: support multi buffer XDP
         - virtio/vsock: replace virtio_vsock_pkt with sk_buff
         - tsnep: XDP support

   - Ethernet high-speed switches:
      - nVidia/Mellanox (mlxsw):
         - add support for latency TLV (in FW control messages)
      - Microchip (sparx5):
         - separate explicit and implicit traffic forwarding rules, make
           the implicit rules always active
         - add support for egress DSCP rewrite
         - IS0 VCAP support (Ingress Classification)
         - IS2 VCAP filters (protos, L3 addrs, L4 ports, flags, ToS
           etc.)
         - ES2 VCAP support (Egress Access Control)
         - support for Per-Stream Filtering and Policing (802.1Q,
           8.6.5.1)

   - Ethernet embedded switches:
      - Marvell (mv88e6xxx):
         - add MAB (port auth) offload support
         - enable PTP receive for mv88e6390
      - NXP (ocelot):
         - support MAC Merge layer
         - support for the the vsc7512 internal copper phys
      - Microchip:
         - lan9303: convert to PHYLINK
         - lan966x: support TC flower filter statistics
         - lan937x: PTP support for KSZ9563/KSZ8563 and LAN937x
         - lan937x: support Credit Based Shaper configuration
         - ksz9477: support Energy Efficient Ethernet
      - other:
         - qca8k: convert to regmap read/write API, use bulk operations
         - rswitch: Improve TX timestamp accuracy

   - Intel WiFi (iwlwifi):
      - EHT (Wi-Fi 7) rate reporting
      - STEP equalizer support: transfer some STEP (connection to radio
        on platforms with integrated wifi) related parameters from the
        BIOS to the firmware.

   - Qualcomm 802.11ax WiFi (ath11k):
      - IPQ5018 support
      - Fine Timing Measurement (FTM) responder role support
      - channel 177 support

   - MediaTek WiFi (mt76):
      - per-PHY LED support
      - mt7996: EHT (Wi-Fi 7) support
      - Wireless Ethernet Dispatch (WED) reset support
      - switch to using page pool allocator

   - RealTek WiFi (rtw89):
      - support new version of Bluetooth co-existance

   - Mobile:
      - rmnet: support TX aggregation"

* tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1872 commits)
  page_pool: add a comment explaining the fragment counter usage
  net: ethtool: fix __ethtool_dev_mm_supported() implementation
  ethtool: pse-pd: Fix double word in comments
  xsk: add linux/vmalloc.h to xsk.c
  sefltests: netdevsim: wait for devlink instance after netns removal
  selftest: fib_tests: Always cleanup before exit
  net/mlx5e: Align IPsec ASO result memory to be as required by hardware
  net/mlx5e: TC, Set CT miss to the specific ct action instance
  net/mlx5e: Rename CHAIN_TO_REG to MAPPED_OBJ_TO_REG
  net/mlx5: Refactor tc miss handling to a single function
  net/mlx5: Kconfig: Make tc offload depend on tc skb extension
  net/sched: flower: Support hardware miss to tc action
  net/sched: flower: Move filter handle initialization earlier
  net/sched: cls_api: Support hardware miss to tc action
  net/sched: Rename user cookie and act cookie
  sfc: fix builds without CONFIG_RTC_LIB
  sfc: clean up some inconsistent indentings
  net/mlx4_en: Introduce flexible array to silence overflow warning
  net: lan966x: Fix possible deadlock inside PTP
  net/ulp: Remove redundant ->clone() test in inet_clone_ulp().
  ...

1 files changed, 1248 insertions, 0 deletions

diff --git a/drivers/ps3/ps3-vuart.c b/drivers/ps3/ps3-vuart.c
new file mode 100644
index 000000000..6328abd51
--- /dev/null
+++ b/drivers/ps3/ps3-vuart.c
@@ -0,0 +1,1248 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ *  PS3 virtual uart
+ *
+ *  Copyright (C) 2006 Sony Computer Entertainment Inc.
+ *  Copyright 2006 Sony Corp.
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/workqueue.h>
+#include <linux/bitops.h>
+#include <asm/ps3.h>
+
+#include <asm/firmware.h>
+#include <asm/lv1call.h>
+
+#include "vuart.h"
+
+MODULE_AUTHOR("Sony Corporation");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("PS3 vuart");
+
+/**
+ * vuart - An inter-partition data link service.
+ *  port 0: PS3 AV Settings.
+ *  port 2: PS3 System Manager.
+ *
+ * The vuart provides a bi-directional byte stream data link between logical
+ * partitions.  Its primary role is as a communications link between the guest
+ * OS and the system policy module.  The current HV does not support any
+ * connections other than those listed.
+ */
+
+enum {PORT_COUNT = 3,};
+
+enum vuart_param {
+	PARAM_TX_TRIGGER = 0,
+	PARAM_RX_TRIGGER = 1,
+	PARAM_INTERRUPT_MASK = 2,
+	PARAM_RX_BUF_SIZE = 3, /* read only */
+	PARAM_RX_BYTES = 4, /* read only */
+	PARAM_TX_BUF_SIZE = 5, /* read only */
+	PARAM_TX_BYTES = 6, /* read only */
+	PARAM_INTERRUPT_STATUS = 7, /* read only */
+};
+
+enum vuart_interrupt_bit {
+	INTERRUPT_BIT_TX = 0,
+	INTERRUPT_BIT_RX = 1,
+	INTERRUPT_BIT_DISCONNECT = 2,
+};
+
+enum vuart_interrupt_mask {
+	INTERRUPT_MASK_TX = 1,
+	INTERRUPT_MASK_RX = 2,
+	INTERRUPT_MASK_DISCONNECT = 4,
+};
+
+/**
+ * struct ps3_vuart_port_priv - private vuart device data.
+ */
+
+struct ps3_vuart_port_priv {
+	u64 interrupt_mask;
+
+	struct {
+		spinlock_t lock;
+		struct list_head head;
+	} tx_list;
+	struct {
+		struct ps3_vuart_work work;
+		unsigned long bytes_held;
+		spinlock_t lock;
+		struct list_head head;
+	} rx_list;
+	struct ps3_vuart_stats stats;
+};
+
+static struct ps3_vuart_port_priv *to_port_priv(
+	struct ps3_system_bus_device *dev)
+{
+	BUG_ON(!dev);
+	BUG_ON(!dev->driver_priv);
+	return (struct ps3_vuart_port_priv *)dev->driver_priv;
+}
+
+/**
+ * struct ports_bmp - bitmap indicating ports needing service.
+ *
+ * A 256 bit read only bitmap indicating ports needing service.  Do not write
+ * to these bits.  Must not cross a page boundary.
+ */
+
+struct ports_bmp {
+	u64 status;
+	u64 unused[3];
+} __attribute__((aligned(32)));
+
+#define dump_ports_bmp(_b) _dump_ports_bmp(_b, __func__, __LINE__)
+static void __maybe_unused _dump_ports_bmp(
+	const struct ports_bmp *bmp, const char *func, int line)
+{
+	pr_debug("%s:%d: ports_bmp: %016llxh\n", func, line, bmp->status);
+}
+
+#define dump_port_params(_b) _dump_port_params(_b, __func__, __LINE__)
+static void __maybe_unused _dump_port_params(unsigned int port_number,
+	const char *func, int line)
+{
+#if defined(DEBUG)
+	static const char *strings[] = {
+		"tx_trigger      ",
+		"rx_trigger      ",
+		"interrupt_mask  ",
+		"rx_buf_size     ",
+		"rx_bytes        ",
+		"tx_buf_size     ",
+		"tx_bytes        ",
+		"interrupt_status",
+	};
+	int result;
+	unsigned int i;
+	u64 value;
+
+	for (i = 0; i < ARRAY_SIZE(strings); i++) {
+		result = lv1_get_virtual_uart_param(port_number, i, &value);
+
+		if (result) {
+			pr_debug("%s:%d: port_%u: %s failed: %s\n", func, line,
+				port_number, strings[i], ps3_result(result));
+			continue;
+		}
+		pr_debug("%s:%d: port_%u: %s = %lxh\n",
+			func, line, port_number, strings[i], value);
+	}
+#endif
+}
+
+int ps3_vuart_get_triggers(struct ps3_system_bus_device *dev,
+	struct vuart_triggers *trig)
+{
+	int result;
+	u64 size;
+	u64 val;
+	u64 tx;
+
+	result = lv1_get_virtual_uart_param(dev->port_number,
+		PARAM_TX_TRIGGER, &tx);
+	trig->tx = tx;
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: tx_trigger failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	result = lv1_get_virtual_uart_param(dev->port_number,
+		PARAM_RX_BUF_SIZE, &size);
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: tx_buf_size failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	result = lv1_get_virtual_uart_param(dev->port_number,
+		PARAM_RX_TRIGGER, &val);
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: rx_trigger failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	trig->rx = size - val;
+
+	dev_dbg(&dev->core, "%s:%d: tx %lxh, rx %lxh\n", __func__, __LINE__,
+		trig->tx, trig->rx);
+
+	return result;
+}
+
+int ps3_vuart_set_triggers(struct ps3_system_bus_device *dev, unsigned int tx,
+	unsigned int rx)
+{
+	int result;
+	u64 size;
+
+	result = lv1_set_virtual_uart_param(dev->port_number,
+		PARAM_TX_TRIGGER, tx);
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: tx_trigger failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	result = lv1_get_virtual_uart_param(dev->port_number,
+		PARAM_RX_BUF_SIZE, &size);
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: tx_buf_size failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	result = lv1_set_virtual_uart_param(dev->port_number,
+		PARAM_RX_TRIGGER, size - rx);
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: rx_trigger failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	dev_dbg(&dev->core, "%s:%d: tx %xh, rx %xh\n", __func__, __LINE__,
+		tx, rx);
+
+	return result;
+}
+
+static int ps3_vuart_get_rx_bytes_waiting(struct ps3_system_bus_device *dev,
+	u64 *bytes_waiting)
+{
+	int result;
+
+	result = lv1_get_virtual_uart_param(dev->port_number,
+		PARAM_RX_BYTES, bytes_waiting);
+
+	if (result)
+		dev_dbg(&dev->core, "%s:%d: rx_bytes failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+
+	dev_dbg(&dev->core, "%s:%d: %llxh\n", __func__, __LINE__,
+		*bytes_waiting);
+	return result;
+}
+
+/**
+ * ps3_vuart_set_interrupt_mask - Enable/disable the port interrupt sources.
+ * @dev: The struct ps3_system_bus_device instance.
+ * @bmp: Logical OR of enum vuart_interrupt_mask values. A zero bit disables.
+ */
+
+static int ps3_vuart_set_interrupt_mask(struct ps3_system_bus_device *dev,
+	unsigned long mask)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	dev_dbg(&dev->core, "%s:%d: %lxh\n", __func__, __LINE__, mask);
+
+	priv->interrupt_mask = mask;
+
+	result = lv1_set_virtual_uart_param(dev->port_number,
+		PARAM_INTERRUPT_MASK, priv->interrupt_mask);
+
+	if (result)
+		dev_dbg(&dev->core, "%s:%d: interrupt_mask failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+
+	return result;
+}
+
+static int ps3_vuart_get_interrupt_status(struct ps3_system_bus_device *dev,
+	unsigned long *status)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	u64 tmp;
+
+	result = lv1_get_virtual_uart_param(dev->port_number,
+		PARAM_INTERRUPT_STATUS, &tmp);
+
+	if (result)
+		dev_dbg(&dev->core, "%s:%d: interrupt_status failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+
+	*status = tmp & priv->interrupt_mask;
+
+	dev_dbg(&dev->core, "%s:%d: m %llxh, s %llxh, m&s %lxh\n",
+		__func__, __LINE__, priv->interrupt_mask, tmp, *status);
+
+	return result;
+}
+
+int ps3_vuart_enable_interrupt_tx(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	return (priv->interrupt_mask & INTERRUPT_MASK_TX) ? 0
+		: ps3_vuart_set_interrupt_mask(dev, priv->interrupt_mask
+		| INTERRUPT_MASK_TX);
+}
+
+int ps3_vuart_enable_interrupt_rx(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	return (priv->interrupt_mask & INTERRUPT_MASK_RX) ? 0
+		: ps3_vuart_set_interrupt_mask(dev, priv->interrupt_mask
+		| INTERRUPT_MASK_RX);
+}
+
+int ps3_vuart_enable_interrupt_disconnect(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	return (priv->interrupt_mask & INTERRUPT_MASK_DISCONNECT) ? 0
+		: ps3_vuart_set_interrupt_mask(dev, priv->interrupt_mask
+		| INTERRUPT_MASK_DISCONNECT);
+}
+
+int ps3_vuart_disable_interrupt_tx(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	return (priv->interrupt_mask & INTERRUPT_MASK_TX)
+		? ps3_vuart_set_interrupt_mask(dev, priv->interrupt_mask
+		& ~INTERRUPT_MASK_TX) : 0;
+}
+
+int ps3_vuart_disable_interrupt_rx(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	return (priv->interrupt_mask & INTERRUPT_MASK_RX)
+		? ps3_vuart_set_interrupt_mask(dev, priv->interrupt_mask
+		& ~INTERRUPT_MASK_RX) : 0;
+}
+
+int ps3_vuart_disable_interrupt_disconnect(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	return (priv->interrupt_mask & INTERRUPT_MASK_DISCONNECT)
+		? ps3_vuart_set_interrupt_mask(dev, priv->interrupt_mask
+		& ~INTERRUPT_MASK_DISCONNECT) : 0;
+}
+
+/**
+ * ps3_vuart_raw_write - Low level write helper.
+ * @dev: The struct ps3_system_bus_device instance.
+ *
+ * Do not call ps3_vuart_raw_write directly, use ps3_vuart_write.
+ */
+
+static int ps3_vuart_raw_write(struct ps3_system_bus_device *dev,
+	const void *buf, unsigned int bytes, u64 *bytes_written)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	result = lv1_write_virtual_uart(dev->port_number,
+		ps3_mm_phys_to_lpar(__pa(buf)), bytes, bytes_written);
+
+	if (result) {
+		dev_warn(&dev->core, "%s:%d: lv1_write_virtual_uart failed: "
+			"%s\n", __func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	priv->stats.bytes_written += *bytes_written;
+
+	dev_dbg(&dev->core, "%s:%d: wrote %llxh/%xh=>%lxh\n", __func__, __LINE__,
+		*bytes_written, bytes, priv->stats.bytes_written);
+
+	return result;
+}
+
+/**
+ * ps3_vuart_raw_read - Low level read helper.
+ * @dev: The struct ps3_system_bus_device instance.
+ *
+ * Do not call ps3_vuart_raw_read directly, use ps3_vuart_read.
+ */
+
+static int ps3_vuart_raw_read(struct ps3_system_bus_device *dev, void *buf,
+	unsigned int bytes, u64 *bytes_read)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+
+	dev_dbg(&dev->core, "%s:%d: %xh\n", __func__, __LINE__, bytes);
+
+	result = lv1_read_virtual_uart(dev->port_number,
+		ps3_mm_phys_to_lpar(__pa(buf)), bytes, bytes_read);
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: lv1_read_virtual_uart failed: %s\n",
+			__func__, __LINE__, ps3_result(result));
+		return result;
+	}
+
+	priv->stats.bytes_read += *bytes_read;
+
+	dev_dbg(&dev->core, "%s:%d: read %llxh/%xh=>%lxh\n", __func__, __LINE__,
+		*bytes_read, bytes, priv->stats.bytes_read);
+
+	return result;
+}
+
+/**
+ * ps3_vuart_clear_rx_bytes - Discard bytes received.
+ * @dev: The struct ps3_system_bus_device instance.
+ * @bytes: Max byte count to discard, zero = all pending.
+ *
+ * Used to clear pending rx interrupt source.  Will not block.
+ */
+
+void ps3_vuart_clear_rx_bytes(struct ps3_system_bus_device *dev,
+	unsigned int bytes)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	u64 bytes_waiting;
+	void *tmp;
+
+	result = ps3_vuart_get_rx_bytes_waiting(dev, &bytes_waiting);
+
+	BUG_ON(result);
+
+	bytes = bytes ? min(bytes, (unsigned int)bytes_waiting) : bytes_waiting;
+
+	dev_dbg(&dev->core, "%s:%d: %u\n", __func__, __LINE__, bytes);
+
+	if (!bytes)
+		return;
+
+	/* Add some extra space for recently arrived data. */
+
+	bytes += 128;
+
+	tmp = kmalloc(bytes, GFP_KERNEL);
+
+	if (!tmp)
+		return;
+
+	ps3_vuart_raw_read(dev, tmp, bytes, &bytes_waiting);
+
+	kfree(tmp);
+
+	/* Don't include these bytes in the stats. */
+
+	priv->stats.bytes_read -= bytes_waiting;
+}
+EXPORT_SYMBOL_GPL(ps3_vuart_clear_rx_bytes);
+
+/**
+ * struct list_buffer - An element for a port device fifo buffer list.
+ */
+
+struct list_buffer {
+	struct list_head link;
+	const unsigned char *head;
+	const unsigned char *tail;
+	unsigned long dbg_number;
+	unsigned char data[];
+};
+
+/**
+ * ps3_vuart_write - the entry point for writing data to a port
+ * @dev: The struct ps3_system_bus_device instance.
+ *
+ * If the port is idle on entry as much of the incoming data is written to
+ * the port as the port will accept.  Otherwise a list buffer is created
+ * and any remaning incoming data is copied to that buffer.  The buffer is
+ * then enqueued for transmision via the transmit interrupt.
+ */
+
+int ps3_vuart_write(struct ps3_system_bus_device *dev, const void *buf,
+	unsigned int bytes)
+{
+	static unsigned long dbg_number;
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	unsigned long flags;
+	struct list_buffer *lb;
+
+	dev_dbg(&dev->core, "%s:%d: %u(%xh) bytes\n", __func__, __LINE__,
+		bytes, bytes);
+
+	spin_lock_irqsave(&priv->tx_list.lock, flags);
+
+	if (list_empty(&priv->tx_list.head)) {
+		u64 bytes_written;
+
+		result = ps3_vuart_raw_write(dev, buf, bytes, &bytes_written);
+
+		spin_unlock_irqrestore(&priv->tx_list.lock, flags);
+
+		if (result) {
+			dev_dbg(&dev->core,
+				"%s:%d: ps3_vuart_raw_write failed\n",
+				__func__, __LINE__);
+			return result;
+		}
+
+		if (bytes_written == bytes) {
+			dev_dbg(&dev->core, "%s:%d: wrote %xh bytes\n",
+				__func__, __LINE__, bytes);
+			return 0;
+		}
+
+		bytes -= bytes_written;
+		buf += bytes_written;
+	} else
+		spin_unlock_irqrestore(&priv->tx_list.lock, flags);
+
+	lb = kmalloc(sizeof(struct list_buffer) + bytes, GFP_KERNEL);
+
+	if (!lb)
+		return -ENOMEM;
+
+	memcpy(lb->data, buf, bytes);
+	lb->head = lb->data;
+	lb->tail = lb->data + bytes;
+	lb->dbg_number = ++dbg_number;
+
+	spin_lock_irqsave(&priv->tx_list.lock, flags);
+	list_add_tail(&lb->link, &priv->tx_list.head);
+	ps3_vuart_enable_interrupt_tx(dev);
+	spin_unlock_irqrestore(&priv->tx_list.lock, flags);
+
+	dev_dbg(&dev->core, "%s:%d: queued buf_%lu, %xh bytes\n",
+		__func__, __LINE__, lb->dbg_number, bytes);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ps3_vuart_write);
+
+/**
+ * ps3_vuart_queue_rx_bytes - Queue waiting bytes into the buffer list.
+ * @dev: The struct ps3_system_bus_device instance.
+ * @bytes_queued: Number of bytes queued to the buffer list.
+ *
+ * Must be called with priv->rx_list.lock held.
+ */
+
+static int ps3_vuart_queue_rx_bytes(struct ps3_system_bus_device *dev,
+	u64 *bytes_queued)
+{
+	static unsigned long dbg_number;
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	struct list_buffer *lb;
+	u64 bytes;
+
+	*bytes_queued = 0;
+
+	result = ps3_vuart_get_rx_bytes_waiting(dev, &bytes);
+	BUG_ON(result);
+
+	if (result)
+		return -EIO;
+
+	if (!bytes)
+		return 0;
+
+	/* Add some extra space for recently arrived data. */
+
+	bytes += 128;
+
+	lb = kmalloc(sizeof(struct list_buffer) + bytes, GFP_ATOMIC);
+
+	if (!lb)
+		return -ENOMEM;
+
+	ps3_vuart_raw_read(dev, lb->data, bytes, &bytes);
+
+	lb->head = lb->data;
+	lb->tail = lb->data + bytes;
+	lb->dbg_number = ++dbg_number;
+
+	list_add_tail(&lb->link, &priv->rx_list.head);
+	priv->rx_list.bytes_held += bytes;
+
+	dev_dbg(&dev->core, "%s:%d: buf_%lu: queued %llxh bytes\n",
+		__func__, __LINE__, lb->dbg_number, bytes);
+
+	*bytes_queued = bytes;
+
+	return 0;
+}
+
+/**
+ * ps3_vuart_read - The entry point for reading data from a port.
+ *
+ * Queue data waiting at the port, and if enough bytes to satisfy the request
+ * are held in the buffer list those bytes are dequeued and copied to the
+ * caller's buffer.  Emptied list buffers are retiered.  If the request cannot
+ * be statified by bytes held in the list buffers -EAGAIN is returned.
+ */
+
+int ps3_vuart_read(struct ps3_system_bus_device *dev, void *buf,
+	unsigned int bytes)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	unsigned long flags;
+	struct list_buffer *lb, *n;
+	unsigned long bytes_read;
+
+	dev_dbg(&dev->core, "%s:%d: %u(%xh) bytes\n", __func__, __LINE__,
+		bytes, bytes);
+
+	spin_lock_irqsave(&priv->rx_list.lock, flags);
+
+	/* Queue rx bytes here for polled reads. */
+
+	while (priv->rx_list.bytes_held < bytes) {
+		u64 tmp;
+
+		result = ps3_vuart_queue_rx_bytes(dev, &tmp);
+		if (result || !tmp) {
+			dev_dbg(&dev->core, "%s:%d: starved for %lxh bytes\n",
+				__func__, __LINE__,
+				bytes - priv->rx_list.bytes_held);
+			spin_unlock_irqrestore(&priv->rx_list.lock, flags);
+			return -EAGAIN;
+		}
+	}
+
+	list_for_each_entry_safe(lb, n, &priv->rx_list.head, link) {
+		bytes_read = min((unsigned int)(lb->tail - lb->head), bytes);
+
+		memcpy(buf, lb->head, bytes_read);
+		buf += bytes_read;
+		bytes -= bytes_read;
+		priv->rx_list.bytes_held -= bytes_read;
+
+		if (bytes_read < lb->tail - lb->head) {
+			lb->head += bytes_read;
+			dev_dbg(&dev->core, "%s:%d: buf_%lu: dequeued %lxh "
+				"bytes\n", __func__, __LINE__, lb->dbg_number,
+				bytes_read);
+			spin_unlock_irqrestore(&priv->rx_list.lock, flags);
+			return 0;
+		}
+
+		dev_dbg(&dev->core, "%s:%d: buf_%lu: free, dequeued %lxh "
+			"bytes\n", __func__, __LINE__, lb->dbg_number,
+			bytes_read);
+
+		list_del(&lb->link);
+		kfree(lb);
+	}
+
+	spin_unlock_irqrestore(&priv->rx_list.lock, flags);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ps3_vuart_read);
+
+/**
+ * ps3_vuart_work - Asynchronous read handler.
+ */
+
+static void ps3_vuart_work(struct work_struct *work)
+{
+	struct ps3_system_bus_device *dev =
+		ps3_vuart_work_to_system_bus_dev(work);
+	struct ps3_vuart_port_driver *drv =
+		ps3_system_bus_dev_to_vuart_drv(dev);
+
+	BUG_ON(!drv);
+	drv->work(dev);
+}
+
+int ps3_vuart_read_async(struct ps3_system_bus_device *dev, unsigned int bytes)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	unsigned long flags;
+
+	if (priv->rx_list.work.trigger) {
+		dev_dbg(&dev->core, "%s:%d: warning, multiple calls\n",
+			__func__, __LINE__);
+		return -EAGAIN;
+	}
+
+	BUG_ON(!bytes);
+
+	spin_lock_irqsave(&priv->rx_list.lock, flags);
+	if (priv->rx_list.bytes_held >= bytes) {
+		dev_dbg(&dev->core, "%s:%d: schedule_work %xh bytes\n",
+			__func__, __LINE__, bytes);
+		schedule_work(&priv->rx_list.work.work);
+		spin_unlock_irqrestore(&priv->rx_list.lock, flags);
+		return 0;
+	}
+
+	priv->rx_list.work.trigger = bytes;
+	spin_unlock_irqrestore(&priv->rx_list.lock, flags);
+
+	dev_dbg(&dev->core, "%s:%d: waiting for %u(%xh) bytes\n", __func__,
+		__LINE__, bytes, bytes);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ps3_vuart_read_async);
+
+void ps3_vuart_cancel_async(struct ps3_system_bus_device *dev)
+{
+	to_port_priv(dev)->rx_list.work.trigger = 0;
+}
+EXPORT_SYMBOL_GPL(ps3_vuart_cancel_async);
+
+/**
+ * ps3_vuart_handle_interrupt_tx - third stage transmit interrupt handler
+ *
+ * Services the transmit interrupt for the port.  Writes as much data from the
+ * buffer list as the port will accept.  Retires any emptied list buffers and
+ * adjusts the final list buffer state for a partial write.
+ */
+
+static int ps3_vuart_handle_interrupt_tx(struct ps3_system_bus_device *dev)
+{
+	int result = 0;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	unsigned long flags;
+	struct list_buffer *lb, *n;
+	unsigned long bytes_total = 0;
+
+	dev_dbg(&dev->core, "%s:%d\n", __func__, __LINE__);
+
+	spin_lock_irqsave(&priv->tx_list.lock, flags);
+
+	list_for_each_entry_safe(lb, n, &priv->tx_list.head, link) {
+
+		u64 bytes_written;
+
+		result = ps3_vuart_raw_write(dev, lb->head, lb->tail - lb->head,
+			&bytes_written);
+
+		if (result) {
+			dev_dbg(&dev->core,
+				"%s:%d: ps3_vuart_raw_write failed\n",
+				__func__, __LINE__);
+			break;
+		}
+
+		bytes_total += bytes_written;
+
+		if (bytes_written < lb->tail - lb->head) {
+			lb->head += bytes_written;
+			dev_dbg(&dev->core,
+				"%s:%d cleared buf_%lu, %llxh bytes\n",
+				__func__, __LINE__, lb->dbg_number,
+				bytes_written);
+			goto port_full;
+		}
+
+		dev_dbg(&dev->core, "%s:%d free buf_%lu\n", __func__, __LINE__,
+			lb->dbg_number);
+
+		list_del(&lb->link);
+		kfree(lb);
+	}
+
+	ps3_vuart_disable_interrupt_tx(dev);
+port_full:
+	spin_unlock_irqrestore(&priv->tx_list.lock, flags);
+	dev_dbg(&dev->core, "%s:%d wrote %lxh bytes total\n",
+		__func__, __LINE__, bytes_total);
+	return result;
+}
+
+/**
+ * ps3_vuart_handle_interrupt_rx - third stage receive interrupt handler
+ *
+ * Services the receive interrupt for the port.  Creates a list buffer and
+ * copies all waiting port data to that buffer and enqueues the buffer in the
+ * buffer list.  Buffer list data is dequeued via ps3_vuart_read.
+ */
+
+static int ps3_vuart_handle_interrupt_rx(struct ps3_system_bus_device *dev)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	unsigned long flags;
+	u64 bytes;
+
+	dev_dbg(&dev->core, "%s:%d\n", __func__, __LINE__);
+
+	spin_lock_irqsave(&priv->rx_list.lock, flags);
+	result = ps3_vuart_queue_rx_bytes(dev, &bytes);
+
+	if (result) {
+		spin_unlock_irqrestore(&priv->rx_list.lock, flags);
+		return result;
+	}
+
+	if (priv->rx_list.work.trigger && priv->rx_list.bytes_held
+		>= priv->rx_list.work.trigger) {
+		dev_dbg(&dev->core, "%s:%d: schedule_work %lxh bytes\n",
+			__func__, __LINE__, priv->rx_list.work.trigger);
+		priv->rx_list.work.trigger = 0;
+		schedule_work(&priv->rx_list.work.work);
+	}
+
+	spin_unlock_irqrestore(&priv->rx_list.lock, flags);
+	return result;
+}
+
+static int ps3_vuart_handle_interrupt_disconnect(
+	struct ps3_system_bus_device *dev)
+{
+	dev_dbg(&dev->core, "%s:%d\n", __func__, __LINE__);
+	BUG_ON("no support");
+	return -1;
+}
+
+/**
+ * ps3_vuart_handle_port_interrupt - second stage interrupt handler
+ *
+ * Services any pending interrupt types for the port.  Passes control to the
+ * third stage type specific interrupt handler.  Returns control to the first
+ * stage handler after one iteration.
+ */
+
+static int ps3_vuart_handle_port_interrupt(struct ps3_system_bus_device *dev)
+{
+	int result;
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	unsigned long status;
+
+	result = ps3_vuart_get_interrupt_status(dev, &status);
+
+	if (result)
+		return result;
+
+	dev_dbg(&dev->core, "%s:%d: status: %lxh\n", __func__, __LINE__,
+		status);
+
+	if (status & INTERRUPT_MASK_DISCONNECT) {
+		priv->stats.disconnect_interrupts++;
+		result = ps3_vuart_handle_interrupt_disconnect(dev);
+		if (result)
+			ps3_vuart_disable_interrupt_disconnect(dev);
+	}
+
+	if (status & INTERRUPT_MASK_TX) {
+		priv->stats.tx_interrupts++;
+		result = ps3_vuart_handle_interrupt_tx(dev);
+		if (result)
+			ps3_vuart_disable_interrupt_tx(dev);
+	}
+
+	if (status & INTERRUPT_MASK_RX) {
+		priv->stats.rx_interrupts++;
+		result = ps3_vuart_handle_interrupt_rx(dev);
+		if (result)
+			ps3_vuart_disable_interrupt_rx(dev);
+	}
+
+	return 0;
+}
+
+static struct vuart_bus_priv {
+	struct ports_bmp *bmp;
+	unsigned int virq;
+	struct mutex probe_mutex;
+	int use_count;
+	struct ps3_system_bus_device *devices[PORT_COUNT];
+} vuart_bus_priv;
+
+/**
+ * ps3_vuart_irq_handler - first stage interrupt handler
+ *
+ * Loops finding any interrupting port and its associated instance data.
+ * Passes control to the second stage port specific interrupt handler.  Loops
+ * until all outstanding interrupts are serviced.
+ */
+
+static irqreturn_t ps3_vuart_irq_handler(int irq, void *_private)
+{
+	struct vuart_bus_priv *bus_priv = _private;
+
+	BUG_ON(!bus_priv);
+
+	while (1) {
+		unsigned int port;
+
+		dump_ports_bmp(bus_priv->bmp);
+
+		port = (BITS_PER_LONG - 1) - __ilog2(bus_priv->bmp->status);
+
+		if (port == BITS_PER_LONG)
+			break;
+
+		BUG_ON(port >= PORT_COUNT);
+		BUG_ON(!bus_priv->devices[port]);
+
+		ps3_vuart_handle_port_interrupt(bus_priv->devices[port]);
+	}
+
+	return IRQ_HANDLED;
+}
+
+static int ps3_vuart_bus_interrupt_get(void)
+{
+	int result;
+
+	pr_debug(" -> %s:%d\n", __func__, __LINE__);
+
+	vuart_bus_priv.use_count++;
+
+	BUG_ON(vuart_bus_priv.use_count > 2);
+
+	if (vuart_bus_priv.use_count != 1)
+		return 0;
+
+	BUG_ON(vuart_bus_priv.bmp);
+
+	vuart_bus_priv.bmp = kzalloc(sizeof(struct ports_bmp), GFP_KERNEL);
+
+	if (!vuart_bus_priv.bmp) {
+		result = -ENOMEM;
+		goto fail_bmp_malloc;
+	}
+
+	result = ps3_vuart_irq_setup(PS3_BINDING_CPU_ANY, vuart_bus_priv.bmp,
+		&vuart_bus_priv.virq);
+
+	if (result) {
+		pr_debug("%s:%d: ps3_vuart_irq_setup failed (%d)\n",
+			__func__, __LINE__, result);
+		result = -EPERM;
+		goto fail_alloc_irq;
+	}
+
+	result = request_irq(vuart_bus_priv.virq, ps3_vuart_irq_handler,
+		0, "vuart", &vuart_bus_priv);
+
+	if (result) {
+		pr_debug("%s:%d: request_irq failed (%d)\n",
+			__func__, __LINE__, result);
+		goto fail_request_irq;
+	}
+
+	pr_debug(" <- %s:%d: ok\n", __func__, __LINE__);
+	return result;
+
+fail_request_irq:
+	ps3_vuart_irq_destroy(vuart_bus_priv.virq);
+	vuart_bus_priv.virq = 0;
+fail_alloc_irq:
+	kfree(vuart_bus_priv.bmp);
+	vuart_bus_priv.bmp = NULL;
+fail_bmp_malloc:
+	vuart_bus_priv.use_count--;
+	pr_debug(" <- %s:%d: failed\n", __func__, __LINE__);
+	return result;
+}
+
+static int ps3_vuart_bus_interrupt_put(void)
+{
+	pr_debug(" -> %s:%d\n", __func__, __LINE__);
+
+	vuart_bus_priv.use_count--;
+
+	BUG_ON(vuart_bus_priv.use_count < 0);
+
+	if (vuart_bus_priv.use_count != 0)
+		return 0;
+
+	free_irq(vuart_bus_priv.virq, &vuart_bus_priv);
+
+	ps3_vuart_irq_destroy(vuart_bus_priv.virq);
+	vuart_bus_priv.virq = 0;
+
+	kfree(vuart_bus_priv.bmp);
+	vuart_bus_priv.bmp = NULL;
+
+	pr_debug(" <- %s:%d\n", __func__, __LINE__);
+	return 0;
+}
+
+static int ps3_vuart_probe(struct ps3_system_bus_device *dev)
+{
+	int result;
+	struct ps3_vuart_port_driver *drv;
+	struct ps3_vuart_port_priv *priv = NULL;
+
+	dev_dbg(&dev->core, "%s:%d\n", __func__, __LINE__);
+
+	drv = ps3_system_bus_dev_to_vuart_drv(dev);
+	BUG_ON(!drv);
+
+	dev_dbg(&dev->core, "%s:%d: (%s)\n", __func__, __LINE__,
+		drv->core.core.name);
+
+	if (dev->port_number >= PORT_COUNT) {
+		BUG();
+		return -EINVAL;
+	}
+
+	mutex_lock(&vuart_bus_priv.probe_mutex);
+
+	result = ps3_vuart_bus_interrupt_get();
+
+	if (result)
+		goto fail_setup_interrupt;
+
+	if (vuart_bus_priv.devices[dev->port_number]) {
+		dev_dbg(&dev->core, "%s:%d: port busy (%d)\n", __func__,
+			__LINE__, dev->port_number);
+		result = -EBUSY;
+		goto fail_busy;
+	}
+
+	vuart_bus_priv.devices[dev->port_number] = dev;
+
+	/* Setup dev->driver_priv. */
+
+	dev->driver_priv = kzalloc(sizeof(struct ps3_vuart_port_priv),
+		GFP_KERNEL);
+
+	if (!dev->driver_priv) {
+		result = -ENOMEM;
+		goto fail_dev_malloc;
+	}
+
+	priv = to_port_priv(dev);
+
+	INIT_LIST_HEAD(&priv->tx_list.head);
+	spin_lock_init(&priv->tx_list.lock);
+
+	INIT_LIST_HEAD(&priv->rx_list.head);
+	spin_lock_init(&priv->rx_list.lock);
+
+	INIT_WORK(&priv->rx_list.work.work, ps3_vuart_work);
+	priv->rx_list.work.trigger = 0;
+	priv->rx_list.work.dev = dev;
+
+	/* clear stale pending interrupts */
+
+	ps3_vuart_clear_rx_bytes(dev, 0);
+
+	ps3_vuart_set_interrupt_mask(dev, INTERRUPT_MASK_RX);
+
+	ps3_vuart_set_triggers(dev, 1, 1);
+
+	if (drv->probe)
+		result = drv->probe(dev);
+	else {
+		result = 0;
+		dev_info(&dev->core, "%s:%d: no probe method\n", __func__,
+			__LINE__);
+	}
+
+	if (result) {
+		dev_dbg(&dev->core, "%s:%d: drv->probe failed\n",
+			__func__, __LINE__);
+		goto fail_probe;
+	}
+
+	mutex_unlock(&vuart_bus_priv.probe_mutex);
+
+	return result;
+
+fail_probe:
+	ps3_vuart_set_interrupt_mask(dev, 0);
+	kfree(dev->driver_priv);
+	dev->driver_priv = NULL;
+fail_dev_malloc:
+	vuart_bus_priv.devices[dev->port_number] = NULL;
+fail_busy:
+	ps3_vuart_bus_interrupt_put();
+fail_setup_interrupt:
+	mutex_unlock(&vuart_bus_priv.probe_mutex);
+	dev_dbg(&dev->core, "%s:%d: failed\n", __func__, __LINE__);
+	return result;
+}
+
+/**
+ * ps3_vuart_cleanup - common cleanup helper.
+ * @dev: The struct ps3_system_bus_device instance.
+ *
+ * Cleans interrupts and HV resources.  Must be called with
+ * vuart_bus_priv.probe_mutex held.  Used by ps3_vuart_remove and
+ * ps3_vuart_shutdown.  After this call, polled reading will still work.
+ */
+
+static int ps3_vuart_cleanup(struct ps3_system_bus_device *dev)
+{
+	dev_dbg(&dev->core, "%s:%d\n", __func__, __LINE__);
+
+	ps3_vuart_cancel_async(dev);
+	ps3_vuart_set_interrupt_mask(dev, 0);
+	ps3_vuart_bus_interrupt_put();
+	return 0;
+}
+
+/**
+ * ps3_vuart_remove - Completely clean the device instance.
+ * @dev: The struct ps3_system_bus_device instance.
+ *
+ * Cleans all memory, interrupts and HV resources.  After this call the
+ * device can no longer be used.
+ */
+
+static void ps3_vuart_remove(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_priv *priv = to_port_priv(dev);
+	struct ps3_vuart_port_driver *drv;
+
+	BUG_ON(!dev);
+
+	mutex_lock(&vuart_bus_priv.probe_mutex);
+
+	dev_dbg(&dev->core, " -> %s:%d: match_id %d\n", __func__, __LINE__,
+		dev->match_id);
+
+	if (!dev->core.driver) {
+		dev_dbg(&dev->core, "%s:%d: no driver bound\n", __func__,
+			__LINE__);
+		mutex_unlock(&vuart_bus_priv.probe_mutex);
+		return;
+	}
+
+	drv = ps3_system_bus_dev_to_vuart_drv(dev);
+
+	BUG_ON(!drv);
+
+	if (drv->remove) {
+		drv->remove(dev);
+	} else {
+		dev_dbg(&dev->core, "%s:%d: no remove method\n", __func__,
+		__LINE__);
+		BUG();
+	}
+
+	ps3_vuart_cleanup(dev);
+
+	vuart_bus_priv.devices[dev->port_number] = NULL;
+	kfree(priv);
+	priv = NULL;
+
+	dev_dbg(&dev->core, " <- %s:%d\n", __func__, __LINE__);
+	mutex_unlock(&vuart_bus_priv.probe_mutex);
+}
+
+/**
+ * ps3_vuart_shutdown - Cleans interrupts and HV resources.
+ * @dev: The struct ps3_system_bus_device instance.
+ *
+ * Cleans interrupts and HV resources.  After this call the
+ * device can still be used in polling mode.  This behavior required
+ * by sys-manager to be able to complete the device power operation
+ * sequence.
+ */
+
+static void ps3_vuart_shutdown(struct ps3_system_bus_device *dev)
+{
+	struct ps3_vuart_port_driver *drv;
+
+	BUG_ON(!dev);
+
+	mutex_lock(&vuart_bus_priv.probe_mutex);
+
+	dev_dbg(&dev->core, " -> %s:%d: match_id %d\n", __func__, __LINE__,
+		dev->match_id);
+
+	if (!dev->core.driver) {
+		dev_dbg(&dev->core, "%s:%d: no driver bound\n", __func__,
+			__LINE__);
+		mutex_unlock(&vuart_bus_priv.probe_mutex);
+		return;
+	}
+
+	drv = ps3_system_bus_dev_to_vuart_drv(dev);
+
+	BUG_ON(!drv);
+
+	if (drv->shutdown)
+		drv->shutdown(dev);
+	else if (drv->remove) {
+		dev_dbg(&dev->core, "%s:%d: no shutdown, calling remove\n",
+			__func__, __LINE__);
+		drv->remove(dev);
+	} else {
+		dev_dbg(&dev->core, "%s:%d: no shutdown method\n", __func__,
+			__LINE__);
+		BUG();
+	}
+
+	ps3_vuart_cleanup(dev);
+
+	dev_dbg(&dev->core, " <- %s:%d\n", __func__, __LINE__);
+
+	mutex_unlock(&vuart_bus_priv.probe_mutex);
+}
+
+static int __init ps3_vuart_bus_init(void)
+{
+	pr_debug("%s:%d:\n", __func__, __LINE__);
+
+	if (!firmware_has_feature(FW_FEATURE_PS3_LV1))
+		return -ENODEV;
+
+	mutex_init(&vuart_bus_priv.probe_mutex);
+
+	return 0;
+}
+
+static void __exit ps3_vuart_bus_exit(void)
+{
+	pr_debug("%s:%d:\n", __func__, __LINE__);
+}
+
+core_initcall(ps3_vuart_bus_init);
+module_exit(ps3_vuart_bus_exit);
+
+/**
+ * ps3_vuart_port_driver_register - Add a vuart port device driver.
+ */
+
+int ps3_vuart_port_driver_register(struct ps3_vuart_port_driver *drv)
+{
+	int result;
+
+	pr_debug("%s:%d: (%s)\n", __func__, __LINE__, drv->core.core.name);
+
+	BUG_ON(!drv->core.match_id);
+	BUG_ON(!drv->core.core.name);
+
+	drv->core.probe = ps3_vuart_probe;
+	drv->core.remove = ps3_vuart_remove;
+	drv->core.shutdown = ps3_vuart_shutdown;
+
+	result = ps3_system_bus_driver_register(&drv->core);
+	return result;
+}
+EXPORT_SYMBOL_GPL(ps3_vuart_port_driver_register);
+
+/**
+ * ps3_vuart_port_driver_unregister - Remove a vuart port device driver.
+ */
+
+void ps3_vuart_port_driver_unregister(struct ps3_vuart_port_driver *drv)
+{
+	pr_debug("%s:%d: (%s)\n", __func__, __LINE__, drv->core.core.name);
+	ps3_system_bus_driver_unregister(&drv->core);
+}
+EXPORT_SYMBOL_GPL(ps3_vuart_port_driver_unregister);