From 5b7c4cabbb65f5c469464da6c5f614cbd7f730f2 Mon Sep 17 00:00:00 2001 From: Linus Torvalds Date: Tue, 21 Feb 2023 18:24:12 -0800 Subject: Merge tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next Pull networking updates from Jakub Kicinski: "Core: - Add dedicated kmem_cache for typical/small skb->head, avoid having to access struct page at kfree time, and improve memory use. - Introduce sysctl to set default RPS configuration for new netdevs. - Define Netlink protocol specification format which can be used to describe messages used by each family and auto-generate parsers. Add tools for generating kernel data structures and uAPI headers. - Expose all net/core sysctls inside netns. - Remove 4s sleep in netpoll if carrier is instantly detected on boot. - Add configurable limit of MDB entries per port, and port-vlan. - Continue populating drop reasons throughout the stack. - Retire a handful of legacy Qdiscs and classifiers. Protocols: - Support IPv4 big TCP (TSO frames larger than 64kB). - Add IP_LOCAL_PORT_RANGE socket option, to control local port range on socket by socket basis. - Track and report in procfs number of MPTCP sockets used. - Support mixing IPv4 and IPv6 flows in the in-kernel MPTCP path manager. - IPv6: don't check net.ipv6.route.max_size and rely on garbage collection to free memory (similarly to IPv4). - Support Penultimate Segment Pop (PSP) flavor in SRv6 (RFC8986). - ICMP: add per-rate limit counters. - Add support for user scanning requests in ieee802154. - Remove static WEP support. - Support minimal Wi-Fi 7 Extremely High Throughput (EHT) rate reporting. - WiFi 7 EHT channel puncturing support (client & AP). BPF: - Add a rbtree data structure following the "next-gen data structure" precedent set by recently added linked list, that is, by using kfunc + kptr instead of adding a new BPF map type. - Expose XDP hints via kfuncs with initial support for RX hash and timestamp metadata. - Add BPF_F_NO_TUNNEL_KEY extension to bpf_skb_set_tunnel_key to better support decap on GRE tunnel devices not operating in collect metadata. - Improve x86 JIT's codegen for PROBE_MEM runtime error checks. - Remove the need for trace_printk_lock for bpf_trace_printk and bpf_trace_vprintk helpers. - Extend libbpf's bpf_tracing.h support for tracing arguments of kprobes/uprobes and syscall as a special case. - Significantly reduce the search time for module symbols by livepatch and BPF. - Enable cpumasks to be used as kptrs, which is useful for tracing programs tracking which tasks end up running on which CPUs in different time intervals. - Add support for BPF trampoline on s390x and riscv64. - Add capability to export the XDP features supported by the NIC. - Add __bpf_kfunc tag for marking kernel functions as kfuncs. - Add cgroup.memory=nobpf kernel parameter option to disable BPF memory accounting for container environments. Netfilter: - Remove the CLUSTERIP target. It has been marked as obsolete for years, and we still have WARN splats wrt races of the out-of-band /proc interface installed by this target. - Add 'destroy' commands to nf_tables. They are identical to the existing 'delete' commands, but do not return an error if the referenced object (set, chain, rule...) did not exist. Driver API: - Improve cpumask_local_spread() locality to help NICs set the right IRQ affinity on AMD platforms. - Separate C22 and C45 MDIO bus transactions more clearly. - Introduce new DCB table to control DSCP rewrite on egress. - Support configuration of Physical Layer Collision Avoidance (PLCA) Reconciliation Sublayer (RS) (802.3cg-2019). Modern version of shared medium Ethernet. - Support for MAC Merge layer (IEEE 802.3-2018 clause 99). Allowing preemption of low priority frames by high priority frames. - Add support for controlling MACSec offload using netlink SET. - Rework devlink instance refcounts to allow registration and de-registration under the instance lock. Split the code into multiple files, drop some of the unnecessarily granular locks and factor out common parts of netlink operation handling. - Add TX frame aggregation parameters (for USB drivers). - Add a new attr TCA_EXT_WARN_MSG to report TC (offload) warning messages with notifications for debug. - Allow offloading of UDP NEW connections via act_ct. - Add support for per action HW stats in TC. - Support hardware miss to TC action (continue processing in SW from a specific point in the action chain). - Warn if old Wireless Extension user space interface is used with modern cfg80211/mac80211 drivers. Do not support Wireless Extensions for Wi-Fi 7 devices at all. Everyone should switch to using nl80211 interface instead. - Improve the CAN bit timing configuration. Use extack to return error messages directly to user space, update the SJW handling, including the definition of a new default value that will benefit CAN-FD controllers, by increasing their oscillator tolerance. New hardware / drivers: - Ethernet: - nVidia BlueField-3 support (control traffic driver) - Ethernet support for imx93 SoCs - Motorcomm yt8531 gigabit Ethernet PHY - onsemi NCN26000 10BASE-T1S PHY (with support for PLCA) - Microchip LAN8841 PHY (incl. cable diagnostics and PTP) - Amlogic gxl MDIO mux - WiFi: - RealTek RTL8188EU (rtl8xxxu) - Qualcomm Wi-Fi 7 devices (ath12k) - CAN: - Renesas R-Car V4H Drivers: - Bluetooth: - Set Per Platform Antenna Gain (PPAG) for Intel controllers. - Ethernet NICs: - Intel (1G, igc): - support TSN / Qbv / packet scheduling features of i226 model - Intel (100G, ice): - use GNSS subsystem instead of TTY - multi-buffer XDP support - extend support for GPIO pins to E823 devices - nVidia/Mellanox: - update the shared buffer configuration on PFC commands - implement PTP adjphase function for HW offset control - TC support for Geneve and GRE with VF tunnel offload - more efficient crypto key management method - multi-port eswitch support - Netronome/Corigine: - add DCB IEEE support - support IPsec offloading for NFP3800 - Freescale/NXP (enetc): - support XDP_REDIRECT for XDP non-linear buffers - improve reconfig, avoid link flap and waiting for idle - support MAC Merge layer - Other NICs: - sfc/ef100: add basic devlink support for ef100 - ionic: rx_push mode operation (writing descriptors via MMIO) - bnxt: use the auxiliary bus abstraction for RDMA - r8169: disable ASPM and reset bus in case of tx timeout - cpsw: support QSGMII mode for J721e CPSW9G - cpts: support pulse-per-second output - ngbe: add an mdio bus driver - usbnet: optimize usbnet_bh() by avoiding unnecessary queuing - r8152: handle devices with FW with NCM support - amd-xgbe: support 10Mbps, 2.5GbE speeds and rx-adaptation - virtio-net: support multi buffer XDP - virtio/vsock: replace virtio_vsock_pkt with sk_buff - tsnep: XDP support - Ethernet high-speed switches: - nVidia/Mellanox (mlxsw): - add support for latency TLV (in FW control messages) - Microchip (sparx5): - separate explicit and implicit traffic forwarding rules, make the implicit rules always active - add support for egress DSCP rewrite - IS0 VCAP support (Ingress Classification) - IS2 VCAP filters (protos, L3 addrs, L4 ports, flags, ToS etc.) - ES2 VCAP support (Egress Access Control) - support for Per-Stream Filtering and Policing (802.1Q, 8.6.5.1) - Ethernet embedded switches: - Marvell (mv88e6xxx): - add MAB (port auth) offload support - enable PTP receive for mv88e6390 - NXP (ocelot): - support MAC Merge layer - support for the the vsc7512 internal copper phys - Microchip: - lan9303: convert to PHYLINK - lan966x: support TC flower filter statistics - lan937x: PTP support for KSZ9563/KSZ8563 and LAN937x - lan937x: support Credit Based Shaper configuration - ksz9477: support Energy Efficient Ethernet - other: - qca8k: convert to regmap read/write API, use bulk operations - rswitch: Improve TX timestamp accuracy - Intel WiFi (iwlwifi): - EHT (Wi-Fi 7) rate reporting - STEP equalizer support: transfer some STEP (connection to radio on platforms with integrated wifi) related parameters from the BIOS to the firmware. - Qualcomm 802.11ax WiFi (ath11k): - IPQ5018 support - Fine Timing Measurement (FTM) responder role support - channel 177 support - MediaTek WiFi (mt76): - per-PHY LED support - mt7996: EHT (Wi-Fi 7) support - Wireless Ethernet Dispatch (WED) reset support - switch to using page pool allocator - RealTek WiFi (rtw89): - support new version of Bluetooth co-existance - Mobile: - rmnet: support TX aggregation" * tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1872 commits) page_pool: add a comment explaining the fragment counter usage net: ethtool: fix __ethtool_dev_mm_supported() implementation ethtool: pse-pd: Fix double word in comments xsk: add linux/vmalloc.h to xsk.c sefltests: netdevsim: wait for devlink instance after netns removal selftest: fib_tests: Always cleanup before exit net/mlx5e: Align IPsec ASO result memory to be as required by hardware net/mlx5e: TC, Set CT miss to the specific ct action instance net/mlx5e: Rename CHAIN_TO_REG to MAPPED_OBJ_TO_REG net/mlx5: Refactor tc miss handling to a single function net/mlx5: Kconfig: Make tc offload depend on tc skb extension net/sched: flower: Support hardware miss to tc action net/sched: flower: Move filter handle initialization earlier net/sched: cls_api: Support hardware miss to tc action net/sched: Rename user cookie and act cookie sfc: fix builds without CONFIG_RTC_LIB sfc: clean up some inconsistent indentings net/mlx4_en: Introduce flexible array to silence overflow warning net: lan966x: Fix possible deadlock inside PTP net/ulp: Remove redundant ->clone() test in inet_clone_ulp(). ... --- drivers/net/wireless/mediatek/mt76/usb.c | 1131 ++++++++++++++++++++++++++++++ 1 file changed, 1131 insertions(+) create mode 100644 drivers/net/wireless/mediatek/mt76/usb.c (limited to 'drivers/net/wireless/mediatek/mt76/usb.c') diff --git a/drivers/net/wireless/mediatek/mt76/usb.c b/drivers/net/wireless/mediatek/mt76/usb.c new file mode 100644 index 000000000..b88959ef3 --- /dev/null +++ b/drivers/net/wireless/mediatek/mt76/usb.c @@ -0,0 +1,1131 @@ +// SPDX-License-Identifier: ISC +/* + * Copyright (C) 2018 Lorenzo Bianconi + */ + +#include +#include "mt76.h" +#include "usb_trace.h" +#include "dma.h" + +#define MT_VEND_REQ_MAX_RETRY 10 +#define MT_VEND_REQ_TOUT_MS 300 + +static bool disable_usb_sg; +module_param_named(disable_usb_sg, disable_usb_sg, bool, 0644); +MODULE_PARM_DESC(disable_usb_sg, "Disable usb scatter-gather support"); + +int __mt76u_vendor_request(struct mt76_dev *dev, u8 req, u8 req_type, + u16 val, u16 offset, void *buf, size_t len) +{ + struct usb_interface *uintf = to_usb_interface(dev->dev); + struct usb_device *udev = interface_to_usbdev(uintf); + unsigned int pipe; + int i, ret; + + lockdep_assert_held(&dev->usb.usb_ctrl_mtx); + + pipe = (req_type & USB_DIR_IN) ? usb_rcvctrlpipe(udev, 0) + : usb_sndctrlpipe(udev, 0); + for (i = 0; i < MT_VEND_REQ_MAX_RETRY; i++) { + if (test_bit(MT76_REMOVED, &dev->phy.state)) + return -EIO; + + ret = usb_control_msg(udev, pipe, req, req_type, val, + offset, buf, len, MT_VEND_REQ_TOUT_MS); + if (ret == -ENODEV) + set_bit(MT76_REMOVED, &dev->phy.state); + if (ret >= 0 || ret == -ENODEV) + return ret; + usleep_range(5000, 10000); + } + + dev_err(dev->dev, "vendor request req:%02x off:%04x failed:%d\n", + req, offset, ret); + return ret; +} +EXPORT_SYMBOL_GPL(__mt76u_vendor_request); + +int mt76u_vendor_request(struct mt76_dev *dev, u8 req, + u8 req_type, u16 val, u16 offset, + void *buf, size_t len) +{ + int ret; + + mutex_lock(&dev->usb.usb_ctrl_mtx); + ret = __mt76u_vendor_request(dev, req, req_type, + val, offset, buf, len); + trace_usb_reg_wr(dev, offset, val); + mutex_unlock(&dev->usb.usb_ctrl_mtx); + + return ret; +} +EXPORT_SYMBOL_GPL(mt76u_vendor_request); + +u32 ___mt76u_rr(struct mt76_dev *dev, u8 req, u8 req_type, u32 addr) +{ + struct mt76_usb *usb = &dev->usb; + u32 data = ~0; + int ret; + + ret = __mt76u_vendor_request(dev, req, req_type, addr >> 16, + addr, usb->data, sizeof(__le32)); + if (ret == sizeof(__le32)) + data = get_unaligned_le32(usb->data); + trace_usb_reg_rr(dev, addr, data); + + return data; +} +EXPORT_SYMBOL_GPL(___mt76u_rr); + +static u32 __mt76u_rr(struct mt76_dev *dev, u32 addr) +{ + u8 req; + + switch (addr & MT_VEND_TYPE_MASK) { + case MT_VEND_TYPE_EEPROM: + req = MT_VEND_READ_EEPROM; + break; + case MT_VEND_TYPE_CFG: + req = MT_VEND_READ_CFG; + break; + default: + req = MT_VEND_MULTI_READ; + break; + } + + return ___mt76u_rr(dev, req, USB_DIR_IN | USB_TYPE_VENDOR, + addr & ~MT_VEND_TYPE_MASK); +} + +static u32 mt76u_rr(struct mt76_dev *dev, u32 addr) +{ + u32 ret; + + mutex_lock(&dev->usb.usb_ctrl_mtx); + ret = __mt76u_rr(dev, addr); + mutex_unlock(&dev->usb.usb_ctrl_mtx); + + return ret; +} + +void ___mt76u_wr(struct mt76_dev *dev, u8 req, u8 req_type, + u32 addr, u32 val) +{ + struct mt76_usb *usb = &dev->usb; + + put_unaligned_le32(val, usb->data); + __mt76u_vendor_request(dev, req, req_type, addr >> 16, + addr, usb->data, sizeof(__le32)); + trace_usb_reg_wr(dev, addr, val); +} +EXPORT_SYMBOL_GPL(___mt76u_wr); + +static void __mt76u_wr(struct mt76_dev *dev, u32 addr, u32 val) +{ + u8 req; + + switch (addr & MT_VEND_TYPE_MASK) { + case MT_VEND_TYPE_CFG: + req = MT_VEND_WRITE_CFG; + break; + default: + req = MT_VEND_MULTI_WRITE; + break; + } + ___mt76u_wr(dev, req, USB_DIR_OUT | USB_TYPE_VENDOR, + addr & ~MT_VEND_TYPE_MASK, val); +} + +static void mt76u_wr(struct mt76_dev *dev, u32 addr, u32 val) +{ + mutex_lock(&dev->usb.usb_ctrl_mtx); + __mt76u_wr(dev, addr, val); + mutex_unlock(&dev->usb.usb_ctrl_mtx); +} + +static u32 mt76u_rmw(struct mt76_dev *dev, u32 addr, + u32 mask, u32 val) +{ + mutex_lock(&dev->usb.usb_ctrl_mtx); + val |= __mt76u_rr(dev, addr) & ~mask; + __mt76u_wr(dev, addr, val); + mutex_unlock(&dev->usb.usb_ctrl_mtx); + + return val; +} + +static void mt76u_copy(struct mt76_dev *dev, u32 offset, + const void *data, int len) +{ + struct mt76_usb *usb = &dev->usb; + const u8 *val = data; + int ret; + int current_batch_size; + int i = 0; + + /* Assure that always a multiple of 4 bytes are copied, + * otherwise beacons can be corrupted. + * See: "mt76: round up length on mt76_wr_copy" + * Commit 850e8f6fbd5d0003b0 + */ + len = round_up(len, 4); + + mutex_lock(&usb->usb_ctrl_mtx); + while (i < len) { + current_batch_size = min_t(int, usb->data_len, len - i); + memcpy(usb->data, val + i, current_batch_size); + ret = __mt76u_vendor_request(dev, MT_VEND_MULTI_WRITE, + USB_DIR_OUT | USB_TYPE_VENDOR, + 0, offset + i, usb->data, + current_batch_size); + if (ret < 0) + break; + + i += current_batch_size; + } + mutex_unlock(&usb->usb_ctrl_mtx); +} + +void mt76u_read_copy(struct mt76_dev *dev, u32 offset, + void *data, int len) +{ + struct mt76_usb *usb = &dev->usb; + int i = 0, batch_len, ret; + u8 *val = data; + + len = round_up(len, 4); + mutex_lock(&usb->usb_ctrl_mtx); + while (i < len) { + batch_len = min_t(int, usb->data_len, len - i); + ret = __mt76u_vendor_request(dev, MT_VEND_READ_EXT, + USB_DIR_IN | USB_TYPE_VENDOR, + (offset + i) >> 16, offset + i, + usb->data, batch_len); + if (ret < 0) + break; + + memcpy(val + i, usb->data, batch_len); + i += batch_len; + } + mutex_unlock(&usb->usb_ctrl_mtx); +} +EXPORT_SYMBOL_GPL(mt76u_read_copy); + +void mt76u_single_wr(struct mt76_dev *dev, const u8 req, + const u16 offset, const u32 val) +{ + mutex_lock(&dev->usb.usb_ctrl_mtx); + __mt76u_vendor_request(dev, req, + USB_DIR_OUT | USB_TYPE_VENDOR, + val & 0xffff, offset, NULL, 0); + __mt76u_vendor_request(dev, req, + USB_DIR_OUT | USB_TYPE_VENDOR, + val >> 16, offset + 2, NULL, 0); + mutex_unlock(&dev->usb.usb_ctrl_mtx); +} +EXPORT_SYMBOL_GPL(mt76u_single_wr); + +static int +mt76u_req_wr_rp(struct mt76_dev *dev, u32 base, + const struct mt76_reg_pair *data, int len) +{ + struct mt76_usb *usb = &dev->usb; + + mutex_lock(&usb->usb_ctrl_mtx); + while (len > 0) { + __mt76u_wr(dev, base + data->reg, data->value); + len--; + data++; + } + mutex_unlock(&usb->usb_ctrl_mtx); + + return 0; +} + +static int +mt76u_wr_rp(struct mt76_dev *dev, u32 base, + const struct mt76_reg_pair *data, int n) +{ + if (test_bit(MT76_STATE_MCU_RUNNING, &dev->phy.state)) + return dev->mcu_ops->mcu_wr_rp(dev, base, data, n); + else + return mt76u_req_wr_rp(dev, base, data, n); +} + +static int +mt76u_req_rd_rp(struct mt76_dev *dev, u32 base, struct mt76_reg_pair *data, + int len) +{ + struct mt76_usb *usb = &dev->usb; + + mutex_lock(&usb->usb_ctrl_mtx); + while (len > 0) { + data->value = __mt76u_rr(dev, base + data->reg); + len--; + data++; + } + mutex_unlock(&usb->usb_ctrl_mtx); + + return 0; +} + +static int +mt76u_rd_rp(struct mt76_dev *dev, u32 base, + struct mt76_reg_pair *data, int n) +{ + if (test_bit(MT76_STATE_MCU_RUNNING, &dev->phy.state)) + return dev->mcu_ops->mcu_rd_rp(dev, base, data, n); + else + return mt76u_req_rd_rp(dev, base, data, n); +} + +static bool mt76u_check_sg(struct mt76_dev *dev) +{ + struct usb_interface *uintf = to_usb_interface(dev->dev); + struct usb_device *udev = interface_to_usbdev(uintf); + + return (!disable_usb_sg && udev->bus->sg_tablesize > 0 && + (udev->bus->no_sg_constraint || + udev->speed == USB_SPEED_WIRELESS)); +} + +static int +mt76u_set_endpoints(struct usb_interface *intf, + struct mt76_usb *usb) +{ + struct usb_host_interface *intf_desc = intf->cur_altsetting; + struct usb_endpoint_descriptor *ep_desc; + int i, in_ep = 0, out_ep = 0; + + for (i = 0; i < intf_desc->desc.bNumEndpoints; i++) { + ep_desc = &intf_desc->endpoint[i].desc; + + if (usb_endpoint_is_bulk_in(ep_desc) && + in_ep < __MT_EP_IN_MAX) { + usb->in_ep[in_ep] = usb_endpoint_num(ep_desc); + in_ep++; + } else if (usb_endpoint_is_bulk_out(ep_desc) && + out_ep < __MT_EP_OUT_MAX) { + usb->out_ep[out_ep] = usb_endpoint_num(ep_desc); + out_ep++; + } + } + + if (in_ep != __MT_EP_IN_MAX || out_ep != __MT_EP_OUT_MAX) + return -EINVAL; + return 0; +} + +static int +mt76u_fill_rx_sg(struct mt76_dev *dev, struct mt76_queue *q, struct urb *urb, + int nsgs) +{ + int i; + + for (i = 0; i < nsgs; i++) { + void *data; + int offset; + + data = mt76_get_page_pool_buf(q, &offset, q->buf_size); + if (!data) + break; + + sg_set_page(&urb->sg[i], virt_to_head_page(data), q->buf_size, + offset); + } + + if (i < nsgs) { + int j; + + for (j = nsgs; j < urb->num_sgs; j++) + mt76_put_page_pool_buf(sg_virt(&urb->sg[j]), false); + urb->num_sgs = i; + } + + urb->num_sgs = max_t(int, i, urb->num_sgs); + urb->transfer_buffer_length = urb->num_sgs * q->buf_size; + sg_init_marker(urb->sg, urb->num_sgs); + + return i ? : -ENOMEM; +} + +static int +mt76u_refill_rx(struct mt76_dev *dev, struct mt76_queue *q, + struct urb *urb, int nsgs) +{ + enum mt76_rxq_id qid = q - &dev->q_rx[MT_RXQ_MAIN]; + int offset; + + if (qid == MT_RXQ_MAIN && dev->usb.sg_en) + return mt76u_fill_rx_sg(dev, q, urb, nsgs); + + urb->transfer_buffer_length = q->buf_size; + urb->transfer_buffer = mt76_get_page_pool_buf(q, &offset, q->buf_size); + + return urb->transfer_buffer ? 0 : -ENOMEM; +} + +static int +mt76u_urb_alloc(struct mt76_dev *dev, struct mt76_queue_entry *e, + int sg_max_size) +{ + unsigned int size = sizeof(struct urb); + + if (dev->usb.sg_en) + size += sg_max_size * sizeof(struct scatterlist); + + e->urb = kzalloc(size, GFP_KERNEL); + if (!e->urb) + return -ENOMEM; + + usb_init_urb(e->urb); + + if (dev->usb.sg_en && sg_max_size > 0) + e->urb->sg = (struct scatterlist *)(e->urb + 1); + + return 0; +} + +static int +mt76u_rx_urb_alloc(struct mt76_dev *dev, struct mt76_queue *q, + struct mt76_queue_entry *e) +{ + enum mt76_rxq_id qid = q - &dev->q_rx[MT_RXQ_MAIN]; + int err, sg_size; + + sg_size = qid == MT_RXQ_MAIN ? MT_RX_SG_MAX_SIZE : 0; + err = mt76u_urb_alloc(dev, e, sg_size); + if (err) + return err; + + return mt76u_refill_rx(dev, q, e->urb, sg_size); +} + +static void mt76u_urb_free(struct urb *urb) +{ + int i; + + for (i = 0; i < urb->num_sgs; i++) + mt76_put_page_pool_buf(sg_virt(&urb->sg[i]), false); + + if (urb->transfer_buffer) + mt76_put_page_pool_buf(urb->transfer_buffer, false); + + usb_free_urb(urb); +} + +static void +mt76u_fill_bulk_urb(struct mt76_dev *dev, int dir, int index, + struct urb *urb, usb_complete_t complete_fn, + void *context) +{ + struct usb_interface *uintf = to_usb_interface(dev->dev); + struct usb_device *udev = interface_to_usbdev(uintf); + unsigned int pipe; + + if (dir == USB_DIR_IN) + pipe = usb_rcvbulkpipe(udev, dev->usb.in_ep[index]); + else + pipe = usb_sndbulkpipe(udev, dev->usb.out_ep[index]); + + urb->dev = udev; + urb->pipe = pipe; + urb->complete = complete_fn; + urb->context = context; +} + +static struct urb * +mt76u_get_next_rx_entry(struct mt76_queue *q) +{ + struct urb *urb = NULL; + unsigned long flags; + + spin_lock_irqsave(&q->lock, flags); + if (q->queued > 0) { + urb = q->entry[q->tail].urb; + q->tail = (q->tail + 1) % q->ndesc; + q->queued--; + } + spin_unlock_irqrestore(&q->lock, flags); + + return urb; +} + +static int +mt76u_get_rx_entry_len(struct mt76_dev *dev, u8 *data, + u32 data_len) +{ + u16 dma_len, min_len; + + dma_len = get_unaligned_le16(data); + if (dev->drv->drv_flags & MT_DRV_RX_DMA_HDR) + return dma_len; + + min_len = MT_DMA_HDR_LEN + MT_RX_RXWI_LEN + MT_FCE_INFO_LEN; + if (data_len < min_len || !dma_len || + dma_len + MT_DMA_HDR_LEN > data_len || + (dma_len & 0x3)) + return -EINVAL; + return dma_len; +} + +static struct sk_buff * +mt76u_build_rx_skb(struct mt76_dev *dev, void *data, + int len, int buf_size) +{ + int head_room, drv_flags = dev->drv->drv_flags; + struct sk_buff *skb; + + head_room = drv_flags & MT_DRV_RX_DMA_HDR ? 0 : MT_DMA_HDR_LEN; + if (SKB_WITH_OVERHEAD(buf_size) < head_room + len) { + struct page *page; + + /* slow path, not enough space for data and + * skb_shared_info + */ + skb = alloc_skb(MT_SKB_HEAD_LEN, GFP_ATOMIC); + if (!skb) + return NULL; + + skb_put_data(skb, data + head_room, MT_SKB_HEAD_LEN); + data += head_room + MT_SKB_HEAD_LEN; + page = virt_to_head_page(data); + skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, + page, data - page_address(page), + len - MT_SKB_HEAD_LEN, buf_size); + + return skb; + } + + /* fast path */ + skb = build_skb(data, buf_size); + if (!skb) + return NULL; + + skb_reserve(skb, head_room); + __skb_put(skb, len); + + return skb; +} + +static int +mt76u_process_rx_entry(struct mt76_dev *dev, struct urb *urb, + int buf_size) +{ + u8 *data = urb->num_sgs ? sg_virt(&urb->sg[0]) : urb->transfer_buffer; + int data_len = urb->num_sgs ? urb->sg[0].length : urb->actual_length; + int len, nsgs = 1, head_room, drv_flags = dev->drv->drv_flags; + struct sk_buff *skb; + + if (!test_bit(MT76_STATE_INITIALIZED, &dev->phy.state)) + return 0; + + len = mt76u_get_rx_entry_len(dev, data, urb->actual_length); + if (len < 0) + return 0; + + head_room = drv_flags & MT_DRV_RX_DMA_HDR ? 0 : MT_DMA_HDR_LEN; + data_len = min_t(int, len, data_len - head_room); + + if (len == data_len && + dev->drv->rx_check && !dev->drv->rx_check(dev, data, data_len)) + return 0; + + skb = mt76u_build_rx_skb(dev, data, data_len, buf_size); + if (!skb) + return 0; + + len -= data_len; + while (len > 0 && nsgs < urb->num_sgs) { + data_len = min_t(int, len, urb->sg[nsgs].length); + skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, + sg_page(&urb->sg[nsgs]), + urb->sg[nsgs].offset, data_len, + buf_size); + len -= data_len; + nsgs++; + } + + skb_mark_for_recycle(skb); + dev->drv->rx_skb(dev, MT_RXQ_MAIN, skb, NULL); + + return nsgs; +} + +static void mt76u_complete_rx(struct urb *urb) +{ + struct mt76_dev *dev = dev_get_drvdata(&urb->dev->dev); + struct mt76_queue *q = urb->context; + unsigned long flags; + + trace_rx_urb(dev, urb); + + switch (urb->status) { + case -ECONNRESET: + case -ESHUTDOWN: + case -ENOENT: + case -EPROTO: + return; + default: + dev_err_ratelimited(dev->dev, "rx urb failed: %d\n", + urb->status); + fallthrough; + case 0: + break; + } + + spin_lock_irqsave(&q->lock, flags); + if (WARN_ONCE(q->entry[q->head].urb != urb, "rx urb mismatch")) + goto out; + + q->head = (q->head + 1) % q->ndesc; + q->queued++; + mt76_worker_schedule(&dev->usb.rx_worker); +out: + spin_unlock_irqrestore(&q->lock, flags); +} + +static int +mt76u_submit_rx_buf(struct mt76_dev *dev, enum mt76_rxq_id qid, + struct urb *urb) +{ + int ep = qid == MT_RXQ_MAIN ? MT_EP_IN_PKT_RX : MT_EP_IN_CMD_RESP; + + mt76u_fill_bulk_urb(dev, USB_DIR_IN, ep, urb, + mt76u_complete_rx, &dev->q_rx[qid]); + trace_submit_urb(dev, urb); + + return usb_submit_urb(urb, GFP_ATOMIC); +} + +static void +mt76u_process_rx_queue(struct mt76_dev *dev, struct mt76_queue *q) +{ + int qid = q - &dev->q_rx[MT_RXQ_MAIN]; + struct urb *urb; + int err, count; + + while (true) { + urb = mt76u_get_next_rx_entry(q); + if (!urb) + break; + + count = mt76u_process_rx_entry(dev, urb, q->buf_size); + if (count > 0) { + err = mt76u_refill_rx(dev, q, urb, count); + if (err < 0) + break; + } + mt76u_submit_rx_buf(dev, qid, urb); + } + if (qid == MT_RXQ_MAIN) { + local_bh_disable(); + mt76_rx_poll_complete(dev, MT_RXQ_MAIN, NULL); + local_bh_enable(); + } +} + +static void mt76u_rx_worker(struct mt76_worker *w) +{ + struct mt76_usb *usb = container_of(w, struct mt76_usb, rx_worker); + struct mt76_dev *dev = container_of(usb, struct mt76_dev, usb); + int i; + + rcu_read_lock(); + mt76_for_each_q_rx(dev, i) + mt76u_process_rx_queue(dev, &dev->q_rx[i]); + rcu_read_unlock(); +} + +static int +mt76u_submit_rx_buffers(struct mt76_dev *dev, enum mt76_rxq_id qid) +{ + struct mt76_queue *q = &dev->q_rx[qid]; + unsigned long flags; + int i, err = 0; + + spin_lock_irqsave(&q->lock, flags); + for (i = 0; i < q->ndesc; i++) { + err = mt76u_submit_rx_buf(dev, qid, q->entry[i].urb); + if (err < 0) + break; + } + q->head = q->tail = 0; + q->queued = 0; + spin_unlock_irqrestore(&q->lock, flags); + + return err; +} + +static int +mt76u_alloc_rx_queue(struct mt76_dev *dev, enum mt76_rxq_id qid) +{ + struct mt76_queue *q = &dev->q_rx[qid]; + int i, err; + + err = mt76_create_page_pool(dev, q); + if (err) + return err; + + spin_lock_init(&q->lock); + q->entry = devm_kcalloc(dev->dev, + MT_NUM_RX_ENTRIES, sizeof(*q->entry), + GFP_KERNEL); + if (!q->entry) + return -ENOMEM; + + q->ndesc = MT_NUM_RX_ENTRIES; + q->buf_size = PAGE_SIZE; + + for (i = 0; i < q->ndesc; i++) { + err = mt76u_rx_urb_alloc(dev, q, &q->entry[i]); + if (err < 0) + return err; + } + + return mt76u_submit_rx_buffers(dev, qid); +} + +int mt76u_alloc_mcu_queue(struct mt76_dev *dev) +{ + return mt76u_alloc_rx_queue(dev, MT_RXQ_MCU); +} +EXPORT_SYMBOL_GPL(mt76u_alloc_mcu_queue); + +static void +mt76u_free_rx_queue(struct mt76_dev *dev, struct mt76_queue *q) +{ + int i; + + for (i = 0; i < q->ndesc; i++) { + if (!q->entry[i].urb) + continue; + + mt76u_urb_free(q->entry[i].urb); + q->entry[i].urb = NULL; + } + page_pool_destroy(q->page_pool); +} + +static void mt76u_free_rx(struct mt76_dev *dev) +{ + int i; + + mt76_worker_teardown(&dev->usb.rx_worker); + + mt76_for_each_q_rx(dev, i) + mt76u_free_rx_queue(dev, &dev->q_rx[i]); +} + +void mt76u_stop_rx(struct mt76_dev *dev) +{ + int i; + + mt76_worker_disable(&dev->usb.rx_worker); + + mt76_for_each_q_rx(dev, i) { + struct mt76_queue *q = &dev->q_rx[i]; + int j; + + for (j = 0; j < q->ndesc; j++) + usb_poison_urb(q->entry[j].urb); + } +} +EXPORT_SYMBOL_GPL(mt76u_stop_rx); + +int mt76u_resume_rx(struct mt76_dev *dev) +{ + int i; + + mt76_for_each_q_rx(dev, i) { + struct mt76_queue *q = &dev->q_rx[i]; + int err, j; + + for (j = 0; j < q->ndesc; j++) + usb_unpoison_urb(q->entry[j].urb); + + err = mt76u_submit_rx_buffers(dev, i); + if (err < 0) + return err; + } + + mt76_worker_enable(&dev->usb.rx_worker); + + return 0; +} +EXPORT_SYMBOL_GPL(mt76u_resume_rx); + +static void mt76u_status_worker(struct mt76_worker *w) +{ + struct mt76_usb *usb = container_of(w, struct mt76_usb, status_worker); + struct mt76_dev *dev = container_of(usb, struct mt76_dev, usb); + struct mt76_queue_entry entry; + struct mt76_queue *q; + int i; + + if (!test_bit(MT76_STATE_RUNNING, &dev->phy.state)) + return; + + for (i = 0; i < IEEE80211_NUM_ACS; i++) { + q = dev->phy.q_tx[i]; + if (!q) + continue; + + while (q->queued > 0) { + if (!q->entry[q->tail].done) + break; + + entry = q->entry[q->tail]; + q->entry[q->tail].done = false; + + mt76_queue_tx_complete(dev, q, &entry); + } + + if (!q->queued) + wake_up(&dev->tx_wait); + + mt76_worker_schedule(&dev->tx_worker); + } + + if (dev->drv->tx_status_data && + !test_and_set_bit(MT76_READING_STATS, &dev->phy.state)) + queue_work(dev->wq, &dev->usb.stat_work); +} + +static void mt76u_tx_status_data(struct work_struct *work) +{ + struct mt76_usb *usb; + struct mt76_dev *dev; + u8 update = 1; + u16 count = 0; + + usb = container_of(work, struct mt76_usb, stat_work); + dev = container_of(usb, struct mt76_dev, usb); + + while (true) { + if (test_bit(MT76_REMOVED, &dev->phy.state)) + break; + + if (!dev->drv->tx_status_data(dev, &update)) + break; + count++; + } + + if (count && test_bit(MT76_STATE_RUNNING, &dev->phy.state)) + queue_work(dev->wq, &usb->stat_work); + else + clear_bit(MT76_READING_STATS, &dev->phy.state); +} + +static void mt76u_complete_tx(struct urb *urb) +{ + struct mt76_dev *dev = dev_get_drvdata(&urb->dev->dev); + struct mt76_queue_entry *e = urb->context; + + if (mt76u_urb_error(urb)) + dev_err(dev->dev, "tx urb failed: %d\n", urb->status); + e->done = true; + + mt76_worker_schedule(&dev->usb.status_worker); +} + +static int +mt76u_tx_setup_buffers(struct mt76_dev *dev, struct sk_buff *skb, + struct urb *urb) +{ + urb->transfer_buffer_length = skb->len; + + if (!dev->usb.sg_en) { + urb->transfer_buffer = skb->data; + return 0; + } + + sg_init_table(urb->sg, MT_TX_SG_MAX_SIZE); + urb->num_sgs = skb_to_sgvec(skb, urb->sg, 0, skb->len); + if (!urb->num_sgs) + return -ENOMEM; + + return urb->num_sgs; +} + +static int +mt76u_tx_queue_skb(struct mt76_dev *dev, struct mt76_queue *q, + enum mt76_txq_id qid, struct sk_buff *skb, + struct mt76_wcid *wcid, struct ieee80211_sta *sta) +{ + struct mt76_tx_info tx_info = { + .skb = skb, + }; + u16 idx = q->head; + int err; + + if (q->queued == q->ndesc) + return -ENOSPC; + + skb->prev = skb->next = NULL; + err = dev->drv->tx_prepare_skb(dev, NULL, qid, wcid, sta, &tx_info); + if (err < 0) + return err; + + err = mt76u_tx_setup_buffers(dev, tx_info.skb, q->entry[idx].urb); + if (err < 0) + return err; + + mt76u_fill_bulk_urb(dev, USB_DIR_OUT, q2ep(q->hw_idx), + q->entry[idx].urb, mt76u_complete_tx, + &q->entry[idx]); + + q->head = (q->head + 1) % q->ndesc; + q->entry[idx].skb = tx_info.skb; + q->entry[idx].wcid = 0xffff; + q->queued++; + + return idx; +} + +static void mt76u_tx_kick(struct mt76_dev *dev, struct mt76_queue *q) +{ + struct urb *urb; + int err; + + while (q->first != q->head) { + urb = q->entry[q->first].urb; + + trace_submit_urb(dev, urb); + err = usb_submit_urb(urb, GFP_ATOMIC); + if (err < 0) { + if (err == -ENODEV) + set_bit(MT76_REMOVED, &dev->phy.state); + else + dev_err(dev->dev, "tx urb submit failed:%d\n", + err); + break; + } + q->first = (q->first + 1) % q->ndesc; + } +} + +static u8 mt76u_ac_to_hwq(struct mt76_dev *dev, u8 ac) +{ + if (mt76_chip(dev) == 0x7663) { + static const u8 lmac_queue_map[] = { + /* ac to lmac mapping */ + [IEEE80211_AC_BK] = 0, + [IEEE80211_AC_BE] = 1, + [IEEE80211_AC_VI] = 2, + [IEEE80211_AC_VO] = 4, + }; + + if (WARN_ON(ac >= ARRAY_SIZE(lmac_queue_map))) + return 1; /* BE */ + + return lmac_queue_map[ac]; + } + + return mt76_ac_to_hwq(ac); +} + +static int mt76u_alloc_tx(struct mt76_dev *dev) +{ + struct mt76_queue *q; + int i, j, err; + + for (i = 0; i <= MT_TXQ_PSD; i++) { + if (i >= IEEE80211_NUM_ACS) { + dev->phy.q_tx[i] = dev->phy.q_tx[0]; + continue; + } + + q = devm_kzalloc(dev->dev, sizeof(*q), GFP_KERNEL); + if (!q) + return -ENOMEM; + + spin_lock_init(&q->lock); + q->hw_idx = mt76u_ac_to_hwq(dev, i); + + dev->phy.q_tx[i] = q; + + q->entry = devm_kcalloc(dev->dev, + MT_NUM_TX_ENTRIES, sizeof(*q->entry), + GFP_KERNEL); + if (!q->entry) + return -ENOMEM; + + q->ndesc = MT_NUM_TX_ENTRIES; + for (j = 0; j < q->ndesc; j++) { + err = mt76u_urb_alloc(dev, &q->entry[j], + MT_TX_SG_MAX_SIZE); + if (err < 0) + return err; + } + } + return 0; +} + +static void mt76u_free_tx(struct mt76_dev *dev) +{ + int i; + + mt76_worker_teardown(&dev->usb.status_worker); + + for (i = 0; i < IEEE80211_NUM_ACS; i++) { + struct mt76_queue *q; + int j; + + q = dev->phy.q_tx[i]; + if (!q) + continue; + + for (j = 0; j < q->ndesc; j++) { + usb_free_urb(q->entry[j].urb); + q->entry[j].urb = NULL; + } + } +} + +void mt76u_stop_tx(struct mt76_dev *dev) +{ + int ret; + + mt76_worker_disable(&dev->usb.status_worker); + + ret = wait_event_timeout(dev->tx_wait, !mt76_has_tx_pending(&dev->phy), + HZ / 5); + if (!ret) { + struct mt76_queue_entry entry; + struct mt76_queue *q; + int i, j; + + dev_err(dev->dev, "timed out waiting for pending tx\n"); + + for (i = 0; i < IEEE80211_NUM_ACS; i++) { + q = dev->phy.q_tx[i]; + if (!q) + continue; + + for (j = 0; j < q->ndesc; j++) + usb_kill_urb(q->entry[j].urb); + } + + mt76_worker_disable(&dev->tx_worker); + + /* On device removal we maight queue skb's, but mt76u_tx_kick() + * will fail to submit urb, cleanup those skb's manually. + */ + for (i = 0; i < IEEE80211_NUM_ACS; i++) { + q = dev->phy.q_tx[i]; + if (!q) + continue; + + while (q->queued > 0) { + entry = q->entry[q->tail]; + q->entry[q->tail].done = false; + mt76_queue_tx_complete(dev, q, &entry); + } + } + + mt76_worker_enable(&dev->tx_worker); + } + + cancel_work_sync(&dev->usb.stat_work); + clear_bit(MT76_READING_STATS, &dev->phy.state); + + mt76_worker_enable(&dev->usb.status_worker); + + mt76_tx_status_check(dev, true); +} +EXPORT_SYMBOL_GPL(mt76u_stop_tx); + +void mt76u_queues_deinit(struct mt76_dev *dev) +{ + mt76u_stop_rx(dev); + mt76u_stop_tx(dev); + + mt76u_free_rx(dev); + mt76u_free_tx(dev); +} +EXPORT_SYMBOL_GPL(mt76u_queues_deinit); + +int mt76u_alloc_queues(struct mt76_dev *dev) +{ + int err; + + err = mt76u_alloc_rx_queue(dev, MT_RXQ_MAIN); + if (err < 0) + return err; + + return mt76u_alloc_tx(dev); +} +EXPORT_SYMBOL_GPL(mt76u_alloc_queues); + +static const struct mt76_queue_ops usb_queue_ops = { + .tx_queue_skb = mt76u_tx_queue_skb, + .kick = mt76u_tx_kick, +}; + +int __mt76u_init(struct mt76_dev *dev, struct usb_interface *intf, + struct mt76_bus_ops *ops) +{ + struct usb_device *udev = interface_to_usbdev(intf); + struct mt76_usb *usb = &dev->usb; + int err; + + INIT_WORK(&usb->stat_work, mt76u_tx_status_data); + + usb->data_len = usb_maxpacket(udev, usb_sndctrlpipe(udev, 0)); + if (usb->data_len < 32) + usb->data_len = 32; + + usb->data = devm_kmalloc(dev->dev, usb->data_len, GFP_KERNEL); + if (!usb->data) + return -ENOMEM; + + mutex_init(&usb->usb_ctrl_mtx); + dev->bus = ops; + dev->queue_ops = &usb_queue_ops; + + dev_set_drvdata(&udev->dev, dev); + + usb->sg_en = mt76u_check_sg(dev); + + err = mt76u_set_endpoints(intf, usb); + if (err < 0) + return err; + + err = mt76_worker_setup(dev->hw, &usb->rx_worker, mt76u_rx_worker, + "usb-rx"); + if (err) + return err; + + err = mt76_worker_setup(dev->hw, &usb->status_worker, + mt76u_status_worker, "usb-status"); + if (err) + return err; + + sched_set_fifo_low(usb->rx_worker.task); + sched_set_fifo_low(usb->status_worker.task); + + return 0; +} +EXPORT_SYMBOL_GPL(__mt76u_init); + +int mt76u_init(struct mt76_dev *dev, struct usb_interface *intf) +{ + static struct mt76_bus_ops bus_ops = { + .rr = mt76u_rr, + .wr = mt76u_wr, + .rmw = mt76u_rmw, + .read_copy = mt76u_read_copy, + .write_copy = mt76u_copy, + .wr_rp = mt76u_wr_rp, + .rd_rp = mt76u_rd_rp, + .type = MT76_BUS_USB, + }; + + return __mt76u_init(dev, intf, &bus_ops); +} +EXPORT_SYMBOL_GPL(mt76u_init); + +MODULE_AUTHOR("Lorenzo Bianconi "); +MODULE_LICENSE("Dual BSD/GPL"); -- cgit v1.2.3