From 5b7c4cabbb65f5c469464da6c5f614cbd7f730f2 Mon Sep 17 00:00:00 2001 From: Linus Torvalds Date: Tue, 21 Feb 2023 18:24:12 -0800 Subject: Merge tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next Pull networking updates from Jakub Kicinski: "Core: - Add dedicated kmem_cache for typical/small skb->head, avoid having to access struct page at kfree time, and improve memory use. - Introduce sysctl to set default RPS configuration for new netdevs. - Define Netlink protocol specification format which can be used to describe messages used by each family and auto-generate parsers. Add tools for generating kernel data structures and uAPI headers. - Expose all net/core sysctls inside netns. - Remove 4s sleep in netpoll if carrier is instantly detected on boot. - Add configurable limit of MDB entries per port, and port-vlan. - Continue populating drop reasons throughout the stack. - Retire a handful of legacy Qdiscs and classifiers. Protocols: - Support IPv4 big TCP (TSO frames larger than 64kB). - Add IP_LOCAL_PORT_RANGE socket option, to control local port range on socket by socket basis. - Track and report in procfs number of MPTCP sockets used. - Support mixing IPv4 and IPv6 flows in the in-kernel MPTCP path manager. - IPv6: don't check net.ipv6.route.max_size and rely on garbage collection to free memory (similarly to IPv4). - Support Penultimate Segment Pop (PSP) flavor in SRv6 (RFC8986). - ICMP: add per-rate limit counters. - Add support for user scanning requests in ieee802154. - Remove static WEP support. - Support minimal Wi-Fi 7 Extremely High Throughput (EHT) rate reporting. - WiFi 7 EHT channel puncturing support (client & AP). BPF: - Add a rbtree data structure following the "next-gen data structure" precedent set by recently added linked list, that is, by using kfunc + kptr instead of adding a new BPF map type. - Expose XDP hints via kfuncs with initial support for RX hash and timestamp metadata. - Add BPF_F_NO_TUNNEL_KEY extension to bpf_skb_set_tunnel_key to better support decap on GRE tunnel devices not operating in collect metadata. - Improve x86 JIT's codegen for PROBE_MEM runtime error checks. - Remove the need for trace_printk_lock for bpf_trace_printk and bpf_trace_vprintk helpers. - Extend libbpf's bpf_tracing.h support for tracing arguments of kprobes/uprobes and syscall as a special case. - Significantly reduce the search time for module symbols by livepatch and BPF. - Enable cpumasks to be used as kptrs, which is useful for tracing programs tracking which tasks end up running on which CPUs in different time intervals. - Add support for BPF trampoline on s390x and riscv64. - Add capability to export the XDP features supported by the NIC. - Add __bpf_kfunc tag for marking kernel functions as kfuncs. - Add cgroup.memory=nobpf kernel parameter option to disable BPF memory accounting for container environments. Netfilter: - Remove the CLUSTERIP target. It has been marked as obsolete for years, and we still have WARN splats wrt races of the out-of-band /proc interface installed by this target. - Add 'destroy' commands to nf_tables. They are identical to the existing 'delete' commands, but do not return an error if the referenced object (set, chain, rule...) did not exist. Driver API: - Improve cpumask_local_spread() locality to help NICs set the right IRQ affinity on AMD platforms. - Separate C22 and C45 MDIO bus transactions more clearly. - Introduce new DCB table to control DSCP rewrite on egress. - Support configuration of Physical Layer Collision Avoidance (PLCA) Reconciliation Sublayer (RS) (802.3cg-2019). Modern version of shared medium Ethernet. - Support for MAC Merge layer (IEEE 802.3-2018 clause 99). Allowing preemption of low priority frames by high priority frames. - Add support for controlling MACSec offload using netlink SET. - Rework devlink instance refcounts to allow registration and de-registration under the instance lock. Split the code into multiple files, drop some of the unnecessarily granular locks and factor out common parts of netlink operation handling. - Add TX frame aggregation parameters (for USB drivers). - Add a new attr TCA_EXT_WARN_MSG to report TC (offload) warning messages with notifications for debug. - Allow offloading of UDP NEW connections via act_ct. - Add support for per action HW stats in TC. - Support hardware miss to TC action (continue processing in SW from a specific point in the action chain). - Warn if old Wireless Extension user space interface is used with modern cfg80211/mac80211 drivers. Do not support Wireless Extensions for Wi-Fi 7 devices at all. Everyone should switch to using nl80211 interface instead. - Improve the CAN bit timing configuration. Use extack to return error messages directly to user space, update the SJW handling, including the definition of a new default value that will benefit CAN-FD controllers, by increasing their oscillator tolerance. New hardware / drivers: - Ethernet: - nVidia BlueField-3 support (control traffic driver) - Ethernet support for imx93 SoCs - Motorcomm yt8531 gigabit Ethernet PHY - onsemi NCN26000 10BASE-T1S PHY (with support for PLCA) - Microchip LAN8841 PHY (incl. cable diagnostics and PTP) - Amlogic gxl MDIO mux - WiFi: - RealTek RTL8188EU (rtl8xxxu) - Qualcomm Wi-Fi 7 devices (ath12k) - CAN: - Renesas R-Car V4H Drivers: - Bluetooth: - Set Per Platform Antenna Gain (PPAG) for Intel controllers. - Ethernet NICs: - Intel (1G, igc): - support TSN / Qbv / packet scheduling features of i226 model - Intel (100G, ice): - use GNSS subsystem instead of TTY - multi-buffer XDP support - extend support for GPIO pins to E823 devices - nVidia/Mellanox: - update the shared buffer configuration on PFC commands - implement PTP adjphase function for HW offset control - TC support for Geneve and GRE with VF tunnel offload - more efficient crypto key management method - multi-port eswitch support - Netronome/Corigine: - add DCB IEEE support - support IPsec offloading for NFP3800 - Freescale/NXP (enetc): - support XDP_REDIRECT for XDP non-linear buffers - improve reconfig, avoid link flap and waiting for idle - support MAC Merge layer - Other NICs: - sfc/ef100: add basic devlink support for ef100 - ionic: rx_push mode operation (writing descriptors via MMIO) - bnxt: use the auxiliary bus abstraction for RDMA - r8169: disable ASPM and reset bus in case of tx timeout - cpsw: support QSGMII mode for J721e CPSW9G - cpts: support pulse-per-second output - ngbe: add an mdio bus driver - usbnet: optimize usbnet_bh() by avoiding unnecessary queuing - r8152: handle devices with FW with NCM support - amd-xgbe: support 10Mbps, 2.5GbE speeds and rx-adaptation - virtio-net: support multi buffer XDP - virtio/vsock: replace virtio_vsock_pkt with sk_buff - tsnep: XDP support - Ethernet high-speed switches: - nVidia/Mellanox (mlxsw): - add support for latency TLV (in FW control messages) - Microchip (sparx5): - separate explicit and implicit traffic forwarding rules, make the implicit rules always active - add support for egress DSCP rewrite - IS0 VCAP support (Ingress Classification) - IS2 VCAP filters (protos, L3 addrs, L4 ports, flags, ToS etc.) - ES2 VCAP support (Egress Access Control) - support for Per-Stream Filtering and Policing (802.1Q, 8.6.5.1) - Ethernet embedded switches: - Marvell (mv88e6xxx): - add MAB (port auth) offload support - enable PTP receive for mv88e6390 - NXP (ocelot): - support MAC Merge layer - support for the the vsc7512 internal copper phys - Microchip: - lan9303: convert to PHYLINK - lan966x: support TC flower filter statistics - lan937x: PTP support for KSZ9563/KSZ8563 and LAN937x - lan937x: support Credit Based Shaper configuration - ksz9477: support Energy Efficient Ethernet - other: - qca8k: convert to regmap read/write API, use bulk operations - rswitch: Improve TX timestamp accuracy - Intel WiFi (iwlwifi): - EHT (Wi-Fi 7) rate reporting - STEP equalizer support: transfer some STEP (connection to radio on platforms with integrated wifi) related parameters from the BIOS to the firmware. - Qualcomm 802.11ax WiFi (ath11k): - IPQ5018 support - Fine Timing Measurement (FTM) responder role support - channel 177 support - MediaTek WiFi (mt76): - per-PHY LED support - mt7996: EHT (Wi-Fi 7) support - Wireless Ethernet Dispatch (WED) reset support - switch to using page pool allocator - RealTek WiFi (rtw89): - support new version of Bluetooth co-existance - Mobile: - rmnet: support TX aggregation" * tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1872 commits) page_pool: add a comment explaining the fragment counter usage net: ethtool: fix __ethtool_dev_mm_supported() implementation ethtool: pse-pd: Fix double word in comments xsk: add linux/vmalloc.h to xsk.c sefltests: netdevsim: wait for devlink instance after netns removal selftest: fib_tests: Always cleanup before exit net/mlx5e: Align IPsec ASO result memory to be as required by hardware net/mlx5e: TC, Set CT miss to the specific ct action instance net/mlx5e: Rename CHAIN_TO_REG to MAPPED_OBJ_TO_REG net/mlx5: Refactor tc miss handling to a single function net/mlx5: Kconfig: Make tc offload depend on tc skb extension net/sched: flower: Support hardware miss to tc action net/sched: flower: Move filter handle initialization earlier net/sched: cls_api: Support hardware miss to tc action net/sched: Rename user cookie and act cookie sfc: fix builds without CONFIG_RTC_LIB sfc: clean up some inconsistent indentings net/mlx4_en: Introduce flexible array to silence overflow warning net: lan966x: Fix possible deadlock inside PTP net/ulp: Remove redundant ->clone() test in inet_clone_ulp(). ... --- drivers/usb/c67x00/c67x00-sched.c | 1148 +++++++++++++++++++++++++++++++++++++ 1 file changed, 1148 insertions(+) create mode 100644 drivers/usb/c67x00/c67x00-sched.c (limited to 'drivers/usb/c67x00/c67x00-sched.c') diff --git a/drivers/usb/c67x00/c67x00-sched.c b/drivers/usb/c67x00/c67x00-sched.c new file mode 100644 index 000000000..a09fa68a6 --- /dev/null +++ b/drivers/usb/c67x00/c67x00-sched.c @@ -0,0 +1,1148 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * c67x00-sched.c: Cypress C67X00 USB Host Controller Driver - TD scheduling + * + * Copyright (C) 2006-2008 Barco N.V. + * Derived from the Cypress cy7c67200/300 ezusb linux driver and + * based on multiple host controller drivers inside the linux kernel. + */ + +#include +#include + +#include "c67x00.h" +#include "c67x00-hcd.h" + +/* + * These are the stages for a control urb, they are kept + * in both urb->interval and td->privdata. + */ +#define SETUP_STAGE 0 +#define DATA_STAGE 1 +#define STATUS_STAGE 2 + +/* -------------------------------------------------------------------------- */ + +/* + * struct c67x00_ep_data: Host endpoint data structure + */ +struct c67x00_ep_data { + struct list_head queue; + struct list_head node; + struct usb_host_endpoint *hep; + struct usb_device *dev; + u16 next_frame; /* For int/isoc transactions */ +}; + +/* + * struct c67x00_td + * + * Hardware parts are little endiannes, SW in CPU endianess. + */ +struct c67x00_td { + /* HW specific part */ + __le16 ly_base_addr; /* Bytes 0-1 */ + __le16 port_length; /* Bytes 2-3 */ + u8 pid_ep; /* Byte 4 */ + u8 dev_addr; /* Byte 5 */ + u8 ctrl_reg; /* Byte 6 */ + u8 status; /* Byte 7 */ + u8 retry_cnt; /* Byte 8 */ +#define TT_OFFSET 2 +#define TT_CONTROL 0 +#define TT_ISOCHRONOUS 1 +#define TT_BULK 2 +#define TT_INTERRUPT 3 + u8 residue; /* Byte 9 */ + __le16 next_td_addr; /* Bytes 10-11 */ + /* SW part */ + struct list_head td_list; + u16 td_addr; + void *data; + struct urb *urb; + unsigned long privdata; + + /* These are needed for handling the toggle bits: + * an urb can be dequeued while a td is in progress + * after checking the td, the toggle bit might need to + * be fixed */ + struct c67x00_ep_data *ep_data; + unsigned int pipe; +}; + +struct c67x00_urb_priv { + struct list_head hep_node; + struct urb *urb; + int port; + int cnt; /* packet number for isoc */ + int status; + struct c67x00_ep_data *ep_data; +}; + +#define td_udev(td) ((td)->ep_data->dev) + +#define CY_TD_SIZE 12 + +#define TD_PIDEP_OFFSET 0x04 +#define TD_PIDEPMASK_PID 0xF0 +#define TD_PIDEPMASK_EP 0x0F +#define TD_PORTLENMASK_DL 0x03FF +#define TD_PORTLENMASK_PN 0xC000 + +#define TD_STATUS_OFFSET 0x07 +#define TD_STATUSMASK_ACK 0x01 +#define TD_STATUSMASK_ERR 0x02 +#define TD_STATUSMASK_TMOUT 0x04 +#define TD_STATUSMASK_SEQ 0x08 +#define TD_STATUSMASK_SETUP 0x10 +#define TD_STATUSMASK_OVF 0x20 +#define TD_STATUSMASK_NAK 0x40 +#define TD_STATUSMASK_STALL 0x80 + +#define TD_ERROR_MASK (TD_STATUSMASK_ERR | TD_STATUSMASK_TMOUT | \ + TD_STATUSMASK_STALL) + +#define TD_RETRYCNT_OFFSET 0x08 +#define TD_RETRYCNTMASK_ACT_FLG 0x10 +#define TD_RETRYCNTMASK_TX_TYPE 0x0C +#define TD_RETRYCNTMASK_RTY_CNT 0x03 + +#define TD_RESIDUE_OVERFLOW 0x80 + +#define TD_PID_IN 0x90 + +/* Residue: signed 8bits, neg -> OVERFLOW, pos -> UNDERFLOW */ +#define td_residue(td) ((__s8)(td->residue)) +#define td_ly_base_addr(td) (__le16_to_cpu((td)->ly_base_addr)) +#define td_port_length(td) (__le16_to_cpu((td)->port_length)) +#define td_next_td_addr(td) (__le16_to_cpu((td)->next_td_addr)) + +#define td_active(td) ((td)->retry_cnt & TD_RETRYCNTMASK_ACT_FLG) +#define td_length(td) (td_port_length(td) & TD_PORTLENMASK_DL) + +#define td_sequence_ok(td) (!td->status || \ + (!(td->status & TD_STATUSMASK_SEQ) == \ + !(td->ctrl_reg & SEQ_SEL))) + +#define td_acked(td) (!td->status || \ + (td->status & TD_STATUSMASK_ACK)) +#define td_actual_bytes(td) (td_length(td) - td_residue(td)) + +/* -------------------------------------------------------------------------- */ + +/* + * dbg_td - Dump the contents of the TD + */ +static void dbg_td(struct c67x00_hcd *c67x00, struct c67x00_td *td, char *msg) +{ + struct device *dev = c67x00_hcd_dev(c67x00); + + dev_dbg(dev, "### %s at 0x%04x\n", msg, td->td_addr); + dev_dbg(dev, "urb: 0x%p\n", td->urb); + dev_dbg(dev, "endpoint: %4d\n", usb_pipeendpoint(td->pipe)); + dev_dbg(dev, "pipeout: %4d\n", usb_pipeout(td->pipe)); + dev_dbg(dev, "ly_base_addr: 0x%04x\n", td_ly_base_addr(td)); + dev_dbg(dev, "port_length: 0x%04x\n", td_port_length(td)); + dev_dbg(dev, "pid_ep: 0x%02x\n", td->pid_ep); + dev_dbg(dev, "dev_addr: 0x%02x\n", td->dev_addr); + dev_dbg(dev, "ctrl_reg: 0x%02x\n", td->ctrl_reg); + dev_dbg(dev, "status: 0x%02x\n", td->status); + dev_dbg(dev, "retry_cnt: 0x%02x\n", td->retry_cnt); + dev_dbg(dev, "residue: 0x%02x\n", td->residue); + dev_dbg(dev, "next_td_addr: 0x%04x\n", td_next_td_addr(td)); + dev_dbg(dev, "data: %*ph\n", td_length(td), td->data); +} + +/* -------------------------------------------------------------------------- */ +/* Helper functions */ + +static inline u16 c67x00_get_current_frame_number(struct c67x00_hcd *c67x00) +{ + return c67x00_ll_husb_get_frame(c67x00->sie) & HOST_FRAME_MASK; +} + +/* + * frame_add + * Software wraparound for framenumbers. + */ +static inline u16 frame_add(u16 a, u16 b) +{ + return (a + b) & HOST_FRAME_MASK; +} + +/* + * frame_after - is frame a after frame b + */ +static inline int frame_after(u16 a, u16 b) +{ + return ((HOST_FRAME_MASK + a - b) & HOST_FRAME_MASK) < + (HOST_FRAME_MASK / 2); +} + +/* + * frame_after_eq - is frame a after or equal to frame b + */ +static inline int frame_after_eq(u16 a, u16 b) +{ + return ((HOST_FRAME_MASK + 1 + a - b) & HOST_FRAME_MASK) < + (HOST_FRAME_MASK / 2); +} + +/* -------------------------------------------------------------------------- */ + +/* + * c67x00_release_urb - remove link from all tds to this urb + * Disconnects the urb from it's tds, so that it can be given back. + * pre: urb->hcpriv != NULL + */ +static void c67x00_release_urb(struct c67x00_hcd *c67x00, struct urb *urb) +{ + struct c67x00_td *td; + struct c67x00_urb_priv *urbp; + + BUG_ON(!urb); + + c67x00->urb_count--; + + if (usb_pipetype(urb->pipe) == PIPE_ISOCHRONOUS) { + c67x00->urb_iso_count--; + if (c67x00->urb_iso_count == 0) + c67x00->max_frame_bw = MAX_FRAME_BW_STD; + } + + /* TODO this might be not so efficient when we've got many urbs! + * Alternatives: + * * only clear when needed + * * keep a list of tds with each urbp + */ + list_for_each_entry(td, &c67x00->td_list, td_list) + if (urb == td->urb) + td->urb = NULL; + + urbp = urb->hcpriv; + urb->hcpriv = NULL; + list_del(&urbp->hep_node); + kfree(urbp); +} + +/* -------------------------------------------------------------------------- */ + +static struct c67x00_ep_data * +c67x00_ep_data_alloc(struct c67x00_hcd *c67x00, struct urb *urb) +{ + struct usb_host_endpoint *hep = urb->ep; + struct c67x00_ep_data *ep_data; + int type; + + c67x00->current_frame = c67x00_get_current_frame_number(c67x00); + + /* Check if endpoint already has a c67x00_ep_data struct allocated */ + if (hep->hcpriv) { + ep_data = hep->hcpriv; + if (frame_after(c67x00->current_frame, ep_data->next_frame)) + ep_data->next_frame = + frame_add(c67x00->current_frame, 1); + return hep->hcpriv; + } + + /* Allocate and initialize a new c67x00 endpoint data structure */ + ep_data = kzalloc(sizeof(*ep_data), GFP_ATOMIC); + if (!ep_data) + return NULL; + + INIT_LIST_HEAD(&ep_data->queue); + INIT_LIST_HEAD(&ep_data->node); + ep_data->hep = hep; + + /* hold a reference to udev as long as this endpoint lives, + * this is needed to possibly fix the data toggle */ + ep_data->dev = usb_get_dev(urb->dev); + hep->hcpriv = ep_data; + + /* For ISOC and INT endpoints, start ASAP: */ + ep_data->next_frame = frame_add(c67x00->current_frame, 1); + + /* Add the endpoint data to one of the pipe lists; must be added + in order of endpoint address */ + type = usb_pipetype(urb->pipe); + if (list_empty(&ep_data->node)) { + list_add(&ep_data->node, &c67x00->list[type]); + } else { + struct c67x00_ep_data *prev; + + list_for_each_entry(prev, &c67x00->list[type], node) { + if (prev->hep->desc.bEndpointAddress > + hep->desc.bEndpointAddress) { + list_add(&ep_data->node, prev->node.prev); + break; + } + } + } + + return ep_data; +} + +static int c67x00_ep_data_free(struct usb_host_endpoint *hep) +{ + struct c67x00_ep_data *ep_data = hep->hcpriv; + + if (!ep_data) + return 0; + + if (!list_empty(&ep_data->queue)) + return -EBUSY; + + usb_put_dev(ep_data->dev); + list_del(&ep_data->queue); + list_del(&ep_data->node); + + kfree(ep_data); + hep->hcpriv = NULL; + + return 0; +} + +void c67x00_endpoint_disable(struct usb_hcd *hcd, struct usb_host_endpoint *ep) +{ + struct c67x00_hcd *c67x00 = hcd_to_c67x00_hcd(hcd); + unsigned long flags; + + if (!list_empty(&ep->urb_list)) + dev_warn(c67x00_hcd_dev(c67x00), "error: urb list not empty\n"); + + spin_lock_irqsave(&c67x00->lock, flags); + + /* loop waiting for all transfers in the endpoint queue to complete */ + while (c67x00_ep_data_free(ep)) { + /* Drop the lock so we can sleep waiting for the hardware */ + spin_unlock_irqrestore(&c67x00->lock, flags); + + /* it could happen that we reinitialize this completion, while + * somebody was waiting for that completion. The timeout and + * while loop handle such cases, but this might be improved */ + reinit_completion(&c67x00->endpoint_disable); + c67x00_sched_kick(c67x00); + wait_for_completion_timeout(&c67x00->endpoint_disable, 1 * HZ); + + spin_lock_irqsave(&c67x00->lock, flags); + } + + spin_unlock_irqrestore(&c67x00->lock, flags); +} + +/* -------------------------------------------------------------------------- */ + +static inline int get_root_port(struct usb_device *dev) +{ + while (dev->parent->parent) + dev = dev->parent; + return dev->portnum; +} + +int c67x00_urb_enqueue(struct usb_hcd *hcd, + struct urb *urb, gfp_t mem_flags) +{ + int ret; + unsigned long flags; + struct c67x00_urb_priv *urbp; + struct c67x00_hcd *c67x00 = hcd_to_c67x00_hcd(hcd); + int port = get_root_port(urb->dev)-1; + + /* Allocate and initialize urb private data */ + urbp = kzalloc(sizeof(*urbp), mem_flags); + if (!urbp) { + ret = -ENOMEM; + goto err_urbp; + } + + spin_lock_irqsave(&c67x00->lock, flags); + + /* Make sure host controller is running */ + if (!HC_IS_RUNNING(hcd->state)) { + ret = -ENODEV; + goto err_not_linked; + } + + ret = usb_hcd_link_urb_to_ep(hcd, urb); + if (ret) + goto err_not_linked; + + INIT_LIST_HEAD(&urbp->hep_node); + urbp->urb = urb; + urbp->port = port; + + urbp->ep_data = c67x00_ep_data_alloc(c67x00, urb); + + if (!urbp->ep_data) { + ret = -ENOMEM; + goto err_epdata; + } + + /* TODO claim bandwidth with usb_claim_bandwidth? + * also release it somewhere! */ + + urb->hcpriv = urbp; + + urb->actual_length = 0; /* Nothing received/transmitted yet */ + + switch (usb_pipetype(urb->pipe)) { + case PIPE_CONTROL: + urb->interval = SETUP_STAGE; + break; + case PIPE_INTERRUPT: + break; + case PIPE_BULK: + break; + case PIPE_ISOCHRONOUS: + if (c67x00->urb_iso_count == 0) + c67x00->max_frame_bw = MAX_FRAME_BW_ISO; + c67x00->urb_iso_count++; + /* Assume always URB_ISO_ASAP, FIXME */ + if (list_empty(&urbp->ep_data->queue)) + urb->start_frame = urbp->ep_data->next_frame; + else { + /* Go right after the last one */ + struct urb *last_urb; + + last_urb = list_entry(urbp->ep_data->queue.prev, + struct c67x00_urb_priv, + hep_node)->urb; + urb->start_frame = + frame_add(last_urb->start_frame, + last_urb->number_of_packets * + last_urb->interval); + } + urbp->cnt = 0; + break; + } + + /* Add the URB to the endpoint queue */ + list_add_tail(&urbp->hep_node, &urbp->ep_data->queue); + + /* If this is the only URB, kick start the controller */ + if (!c67x00->urb_count++) + c67x00_ll_hpi_enable_sofeop(c67x00->sie); + + c67x00_sched_kick(c67x00); + spin_unlock_irqrestore(&c67x00->lock, flags); + + return 0; + +err_epdata: + usb_hcd_unlink_urb_from_ep(hcd, urb); +err_not_linked: + spin_unlock_irqrestore(&c67x00->lock, flags); + kfree(urbp); +err_urbp: + + return ret; +} + +int c67x00_urb_dequeue(struct usb_hcd *hcd, struct urb *urb, int status) +{ + struct c67x00_hcd *c67x00 = hcd_to_c67x00_hcd(hcd); + unsigned long flags; + int rc; + + spin_lock_irqsave(&c67x00->lock, flags); + rc = usb_hcd_check_unlink_urb(hcd, urb, status); + if (rc) + goto done; + + c67x00_release_urb(c67x00, urb); + usb_hcd_unlink_urb_from_ep(hcd, urb); + + spin_unlock(&c67x00->lock); + usb_hcd_giveback_urb(hcd, urb, status); + spin_lock(&c67x00->lock); + + spin_unlock_irqrestore(&c67x00->lock, flags); + + return 0; + + done: + spin_unlock_irqrestore(&c67x00->lock, flags); + return rc; +} + +/* -------------------------------------------------------------------------- */ + +/* + * pre: c67x00 locked, urb unlocked + */ +static void +c67x00_giveback_urb(struct c67x00_hcd *c67x00, struct urb *urb, int status) +{ + struct c67x00_urb_priv *urbp; + + if (!urb) + return; + + urbp = urb->hcpriv; + urbp->status = status; + + list_del_init(&urbp->hep_node); + + c67x00_release_urb(c67x00, urb); + usb_hcd_unlink_urb_from_ep(c67x00_hcd_to_hcd(c67x00), urb); + spin_unlock(&c67x00->lock); + usb_hcd_giveback_urb(c67x00_hcd_to_hcd(c67x00), urb, status); + spin_lock(&c67x00->lock); +} + +/* -------------------------------------------------------------------------- */ + +static int c67x00_claim_frame_bw(struct c67x00_hcd *c67x00, struct urb *urb, + int len, int periodic) +{ + struct c67x00_urb_priv *urbp = urb->hcpriv; + int bit_time; + + /* According to the C67x00 BIOS user manual, page 3-18,19, the + * following calculations provide the full speed bit times for + * a transaction. + * + * FS(in) = 112.5 + 9.36*BC + HOST_DELAY + * FS(in,iso) = 90.5 + 9.36*BC + HOST_DELAY + * FS(out) = 112.5 + 9.36*BC + HOST_DELAY + * FS(out,iso) = 78.4 + 9.36*BC + HOST_DELAY + * LS(in) = 802.4 + 75.78*BC + HOST_DELAY + * LS(out) = 802.6 + 74.67*BC + HOST_DELAY + * + * HOST_DELAY == 106 for the c67200 and c67300. + */ + + /* make calculations in 1/100 bit times to maintain resolution */ + if (urbp->ep_data->dev->speed == USB_SPEED_LOW) { + /* Low speed pipe */ + if (usb_pipein(urb->pipe)) + bit_time = 80240 + 7578*len; + else + bit_time = 80260 + 7467*len; + } else { + /* FS pipes */ + if (usb_pipeisoc(urb->pipe)) + bit_time = usb_pipein(urb->pipe) ? 9050 : 7840; + else + bit_time = 11250; + bit_time += 936*len; + } + + /* Scale back down to integer bit times. Use a host delay of 106. + * (this is the only place it is used) */ + bit_time = ((bit_time+50) / 100) + 106; + + if (unlikely(bit_time + c67x00->bandwidth_allocated >= + c67x00->max_frame_bw)) + return -EMSGSIZE; + + if (unlikely(c67x00->next_td_addr + CY_TD_SIZE >= + c67x00->td_base_addr + SIE_TD_SIZE)) + return -EMSGSIZE; + + if (unlikely(c67x00->next_buf_addr + len >= + c67x00->buf_base_addr + SIE_TD_BUF_SIZE)) + return -EMSGSIZE; + + if (periodic) { + if (unlikely(bit_time + c67x00->periodic_bw_allocated >= + MAX_PERIODIC_BW(c67x00->max_frame_bw))) + return -EMSGSIZE; + c67x00->periodic_bw_allocated += bit_time; + } + + c67x00->bandwidth_allocated += bit_time; + return 0; +} + +/* -------------------------------------------------------------------------- */ + +/* + * td_addr and buf_addr must be word aligned + */ +static int c67x00_create_td(struct c67x00_hcd *c67x00, struct urb *urb, + void *data, int len, int pid, int toggle, + unsigned long privdata) +{ + struct c67x00_td *td; + struct c67x00_urb_priv *urbp = urb->hcpriv; + const __u8 active_flag = 1, retry_cnt = 3; + __u8 cmd = 0; + int tt = 0; + + if (c67x00_claim_frame_bw(c67x00, urb, len, usb_pipeisoc(urb->pipe) + || usb_pipeint(urb->pipe))) + return -EMSGSIZE; /* Not really an error, but expected */ + + td = kzalloc(sizeof(*td), GFP_ATOMIC); + if (!td) + return -ENOMEM; + + td->pipe = urb->pipe; + td->ep_data = urbp->ep_data; + + if ((td_udev(td)->speed == USB_SPEED_LOW) && + !(c67x00->low_speed_ports & (1 << urbp->port))) + cmd |= PREAMBLE_EN; + + switch (usb_pipetype(td->pipe)) { + case PIPE_ISOCHRONOUS: + tt = TT_ISOCHRONOUS; + cmd |= ISO_EN; + break; + case PIPE_CONTROL: + tt = TT_CONTROL; + break; + case PIPE_BULK: + tt = TT_BULK; + break; + case PIPE_INTERRUPT: + tt = TT_INTERRUPT; + break; + } + + if (toggle) + cmd |= SEQ_SEL; + + cmd |= ARM_EN; + + /* SW part */ + td->td_addr = c67x00->next_td_addr; + c67x00->next_td_addr = c67x00->next_td_addr + CY_TD_SIZE; + + /* HW part */ + td->ly_base_addr = __cpu_to_le16(c67x00->next_buf_addr); + td->port_length = __cpu_to_le16((c67x00->sie->sie_num << 15) | + (urbp->port << 14) | (len & 0x3FF)); + td->pid_ep = ((pid & 0xF) << TD_PIDEP_OFFSET) | + (usb_pipeendpoint(td->pipe) & 0xF); + td->dev_addr = usb_pipedevice(td->pipe) & 0x7F; + td->ctrl_reg = cmd; + td->status = 0; + td->retry_cnt = (tt << TT_OFFSET) | (active_flag << 4) | retry_cnt; + td->residue = 0; + td->next_td_addr = __cpu_to_le16(c67x00->next_td_addr); + + /* SW part */ + td->data = data; + td->urb = urb; + td->privdata = privdata; + + c67x00->next_buf_addr += (len + 1) & ~0x01; /* properly align */ + + list_add_tail(&td->td_list, &c67x00->td_list); + return 0; +} + +static inline void c67x00_release_td(struct c67x00_td *td) +{ + list_del_init(&td->td_list); + kfree(td); +} + +/* -------------------------------------------------------------------------- */ + +static int c67x00_add_data_urb(struct c67x00_hcd *c67x00, struct urb *urb) +{ + int remaining; + int toggle; + int pid; + int ret = 0; + int maxps; + int need_empty; + + toggle = usb_gettoggle(urb->dev, usb_pipeendpoint(urb->pipe), + usb_pipeout(urb->pipe)); + remaining = urb->transfer_buffer_length - urb->actual_length; + + maxps = usb_maxpacket(urb->dev, urb->pipe); + + need_empty = (urb->transfer_flags & URB_ZERO_PACKET) && + usb_pipeout(urb->pipe) && !(remaining % maxps); + + while (remaining || need_empty) { + int len; + char *td_buf; + + len = (remaining > maxps) ? maxps : remaining; + if (!len) + need_empty = 0; + + pid = usb_pipeout(urb->pipe) ? USB_PID_OUT : USB_PID_IN; + td_buf = urb->transfer_buffer + urb->transfer_buffer_length - + remaining; + ret = c67x00_create_td(c67x00, urb, td_buf, len, pid, toggle, + DATA_STAGE); + if (ret) + return ret; /* td wasn't created */ + + toggle ^= 1; + remaining -= len; + if (usb_pipecontrol(urb->pipe)) + break; + } + + return 0; +} + +/* + * return 0 in case more bandwidth is available, else errorcode + */ +static int c67x00_add_ctrl_urb(struct c67x00_hcd *c67x00, struct urb *urb) +{ + int ret; + int pid; + + switch (urb->interval) { + default: + case SETUP_STAGE: + ret = c67x00_create_td(c67x00, urb, urb->setup_packet, + 8, USB_PID_SETUP, 0, SETUP_STAGE); + if (ret) + return ret; + urb->interval = SETUP_STAGE; + usb_settoggle(urb->dev, usb_pipeendpoint(urb->pipe), + usb_pipeout(urb->pipe), 1); + break; + case DATA_STAGE: + if (urb->transfer_buffer_length) { + ret = c67x00_add_data_urb(c67x00, urb); + if (ret) + return ret; + break; + } + fallthrough; + case STATUS_STAGE: + pid = !usb_pipeout(urb->pipe) ? USB_PID_OUT : USB_PID_IN; + ret = c67x00_create_td(c67x00, urb, NULL, 0, pid, 1, + STATUS_STAGE); + if (ret) + return ret; + break; + } + + return 0; +} + +/* + * return 0 in case more bandwidth is available, else errorcode + */ +static int c67x00_add_int_urb(struct c67x00_hcd *c67x00, struct urb *urb) +{ + struct c67x00_urb_priv *urbp = urb->hcpriv; + + if (frame_after_eq(c67x00->current_frame, urbp->ep_data->next_frame)) { + urbp->ep_data->next_frame = + frame_add(urbp->ep_data->next_frame, urb->interval); + return c67x00_add_data_urb(c67x00, urb); + } + return 0; +} + +static int c67x00_add_iso_urb(struct c67x00_hcd *c67x00, struct urb *urb) +{ + struct c67x00_urb_priv *urbp = urb->hcpriv; + + if (frame_after_eq(c67x00->current_frame, urbp->ep_data->next_frame)) { + char *td_buf; + int len, pid, ret; + + BUG_ON(urbp->cnt >= urb->number_of_packets); + + td_buf = urb->transfer_buffer + + urb->iso_frame_desc[urbp->cnt].offset; + len = urb->iso_frame_desc[urbp->cnt].length; + pid = usb_pipeout(urb->pipe) ? USB_PID_OUT : USB_PID_IN; + + ret = c67x00_create_td(c67x00, urb, td_buf, len, pid, 0, + urbp->cnt); + if (ret) { + dev_dbg(c67x00_hcd_dev(c67x00), "create failed: %d\n", + ret); + urb->iso_frame_desc[urbp->cnt].actual_length = 0; + urb->iso_frame_desc[urbp->cnt].status = ret; + if (urbp->cnt + 1 == urb->number_of_packets) + c67x00_giveback_urb(c67x00, urb, 0); + } + + urbp->ep_data->next_frame = + frame_add(urbp->ep_data->next_frame, urb->interval); + urbp->cnt++; + } + return 0; +} + +/* -------------------------------------------------------------------------- */ + +static void c67x00_fill_from_list(struct c67x00_hcd *c67x00, int type, + int (*add)(struct c67x00_hcd *, struct urb *)) +{ + struct c67x00_ep_data *ep_data; + struct urb *urb; + + /* traverse every endpoint on the list */ + list_for_each_entry(ep_data, &c67x00->list[type], node) { + if (!list_empty(&ep_data->queue)) { + /* and add the first urb */ + /* isochronous transfer rely on this */ + urb = list_entry(ep_data->queue.next, + struct c67x00_urb_priv, + hep_node)->urb; + add(c67x00, urb); + } + } +} + +static void c67x00_fill_frame(struct c67x00_hcd *c67x00) +{ + struct c67x00_td *td, *ttd; + + /* Check if we can proceed */ + if (!list_empty(&c67x00->td_list)) { + dev_warn(c67x00_hcd_dev(c67x00), + "TD list not empty! This should not happen!\n"); + list_for_each_entry_safe(td, ttd, &c67x00->td_list, td_list) { + dbg_td(c67x00, td, "Unprocessed td"); + c67x00_release_td(td); + } + } + + /* Reinitialize variables */ + c67x00->bandwidth_allocated = 0; + c67x00->periodic_bw_allocated = 0; + + c67x00->next_td_addr = c67x00->td_base_addr; + c67x00->next_buf_addr = c67x00->buf_base_addr; + + /* Fill the list */ + c67x00_fill_from_list(c67x00, PIPE_ISOCHRONOUS, c67x00_add_iso_urb); + c67x00_fill_from_list(c67x00, PIPE_INTERRUPT, c67x00_add_int_urb); + c67x00_fill_from_list(c67x00, PIPE_CONTROL, c67x00_add_ctrl_urb); + c67x00_fill_from_list(c67x00, PIPE_BULK, c67x00_add_data_urb); +} + +/* -------------------------------------------------------------------------- */ + +/* + * Get TD from C67X00 + */ +static inline void +c67x00_parse_td(struct c67x00_hcd *c67x00, struct c67x00_td *td) +{ + c67x00_ll_read_mem_le16(c67x00->sie->dev, + td->td_addr, td, CY_TD_SIZE); + + if (usb_pipein(td->pipe) && td_actual_bytes(td)) + c67x00_ll_read_mem_le16(c67x00->sie->dev, td_ly_base_addr(td), + td->data, td_actual_bytes(td)); +} + +static int c67x00_td_to_error(struct c67x00_hcd *c67x00, struct c67x00_td *td) +{ + if (td->status & TD_STATUSMASK_ERR) { + dbg_td(c67x00, td, "ERROR_FLAG"); + return -EILSEQ; + } + if (td->status & TD_STATUSMASK_STALL) { + /* dbg_td(c67x00, td, "STALL"); */ + return -EPIPE; + } + if (td->status & TD_STATUSMASK_TMOUT) { + dbg_td(c67x00, td, "TIMEOUT"); + return -ETIMEDOUT; + } + + return 0; +} + +static inline int c67x00_end_of_data(struct c67x00_td *td) +{ + int maxps, need_empty, remaining; + struct urb *urb = td->urb; + int act_bytes; + + act_bytes = td_actual_bytes(td); + + if (unlikely(!act_bytes)) + return 1; /* This was an empty packet */ + + maxps = usb_maxpacket(td_udev(td), td->pipe); + + if (unlikely(act_bytes < maxps)) + return 1; /* Smaller then full packet */ + + remaining = urb->transfer_buffer_length - urb->actual_length; + need_empty = (urb->transfer_flags & URB_ZERO_PACKET) && + usb_pipeout(urb->pipe) && !(remaining % maxps); + + if (unlikely(!remaining && !need_empty)) + return 1; + + return 0; +} + +/* -------------------------------------------------------------------------- */ + +/* Remove all td's from the list which come + * after last_td and are meant for the same pipe. + * This is used when a short packet has occurred */ +static inline void c67x00_clear_pipe(struct c67x00_hcd *c67x00, + struct c67x00_td *last_td) +{ + struct c67x00_td *td, *tmp; + td = last_td; + tmp = last_td; + while (td->td_list.next != &c67x00->td_list) { + td = list_entry(td->td_list.next, struct c67x00_td, td_list); + if (td->pipe == last_td->pipe) { + c67x00_release_td(td); + td = tmp; + } + tmp = td; + } +} + +/* -------------------------------------------------------------------------- */ + +static void c67x00_handle_successful_td(struct c67x00_hcd *c67x00, + struct c67x00_td *td) +{ + struct urb *urb = td->urb; + + if (!urb) + return; + + urb->actual_length += td_actual_bytes(td); + + switch (usb_pipetype(td->pipe)) { + /* isochronous tds are handled separately */ + case PIPE_CONTROL: + switch (td->privdata) { + case SETUP_STAGE: + urb->interval = + urb->transfer_buffer_length ? + DATA_STAGE : STATUS_STAGE; + /* Don't count setup_packet with normal data: */ + urb->actual_length = 0; + break; + + case DATA_STAGE: + if (c67x00_end_of_data(td)) { + urb->interval = STATUS_STAGE; + c67x00_clear_pipe(c67x00, td); + } + break; + + case STATUS_STAGE: + urb->interval = 0; + c67x00_giveback_urb(c67x00, urb, 0); + break; + } + break; + + case PIPE_INTERRUPT: + case PIPE_BULK: + if (unlikely(c67x00_end_of_data(td))) { + c67x00_clear_pipe(c67x00, td); + c67x00_giveback_urb(c67x00, urb, 0); + } + break; + } +} + +static void c67x00_handle_isoc(struct c67x00_hcd *c67x00, struct c67x00_td *td) +{ + struct urb *urb = td->urb; + int cnt; + + if (!urb) + return; + + cnt = td->privdata; + + if (td->status & TD_ERROR_MASK) + urb->error_count++; + + urb->iso_frame_desc[cnt].actual_length = td_actual_bytes(td); + urb->iso_frame_desc[cnt].status = c67x00_td_to_error(c67x00, td); + if (cnt + 1 == urb->number_of_packets) /* Last packet */ + c67x00_giveback_urb(c67x00, urb, 0); +} + +/* -------------------------------------------------------------------------- */ + +/* + * c67x00_check_td_list - handle tds which have been processed by the c67x00 + * pre: current_td == 0 + */ +static inline void c67x00_check_td_list(struct c67x00_hcd *c67x00) +{ + struct c67x00_td *td, *tmp; + struct urb *urb; + int ack_ok; + int clear_endpoint; + + list_for_each_entry_safe(td, tmp, &c67x00->td_list, td_list) { + /* get the TD */ + c67x00_parse_td(c67x00, td); + urb = td->urb; /* urb can be NULL! */ + ack_ok = 0; + clear_endpoint = 1; + + /* Handle isochronous transfers separately */ + if (usb_pipeisoc(td->pipe)) { + clear_endpoint = 0; + c67x00_handle_isoc(c67x00, td); + goto cont; + } + + /* When an error occurs, all td's for that pipe go into an + * inactive state. This state matches successful transfers so + * we must make sure not to service them. */ + if (td->status & TD_ERROR_MASK) { + c67x00_giveback_urb(c67x00, urb, + c67x00_td_to_error(c67x00, td)); + goto cont; + } + + if ((td->status & TD_STATUSMASK_NAK) || !td_sequence_ok(td) || + !td_acked(td)) + goto cont; + + /* Sequence ok and acked, don't need to fix toggle */ + ack_ok = 1; + + if (unlikely(td->status & TD_STATUSMASK_OVF)) { + if (td_residue(td) & TD_RESIDUE_OVERFLOW) { + /* Overflow */ + c67x00_giveback_urb(c67x00, urb, -EOVERFLOW); + goto cont; + } + } + + clear_endpoint = 0; + c67x00_handle_successful_td(c67x00, td); + +cont: + if (clear_endpoint) + c67x00_clear_pipe(c67x00, td); + if (ack_ok) + usb_settoggle(td_udev(td), usb_pipeendpoint(td->pipe), + usb_pipeout(td->pipe), + !(td->ctrl_reg & SEQ_SEL)); + /* next in list could have been removed, due to clear_pipe! */ + tmp = list_entry(td->td_list.next, typeof(*td), td_list); + c67x00_release_td(td); + } +} + +/* -------------------------------------------------------------------------- */ + +static inline int c67x00_all_tds_processed(struct c67x00_hcd *c67x00) +{ + /* If all tds are processed, we can check the previous frame (if + * there was any) and start our next frame. + */ + return !c67x00_ll_husb_get_current_td(c67x00->sie); +} + +/* + * Send td to C67X00 + */ +static void c67x00_send_td(struct c67x00_hcd *c67x00, struct c67x00_td *td) +{ + int len = td_length(td); + + if (len && ((td->pid_ep & TD_PIDEPMASK_PID) != TD_PID_IN)) + c67x00_ll_write_mem_le16(c67x00->sie->dev, td_ly_base_addr(td), + td->data, len); + + c67x00_ll_write_mem_le16(c67x00->sie->dev, + td->td_addr, td, CY_TD_SIZE); +} + +static void c67x00_send_frame(struct c67x00_hcd *c67x00) +{ + struct c67x00_td *td; + + if (list_empty(&c67x00->td_list)) + dev_warn(c67x00_hcd_dev(c67x00), + "%s: td list should not be empty here!\n", + __func__); + + list_for_each_entry(td, &c67x00->td_list, td_list) { + if (td->td_list.next == &c67x00->td_list) + td->next_td_addr = 0; /* Last td in list */ + + c67x00_send_td(c67x00, td); + } + + c67x00_ll_husb_set_current_td(c67x00->sie, c67x00->td_base_addr); +} + +/* -------------------------------------------------------------------------- */ + +/* + * c67x00_do_work - Schedulers state machine + */ +static void c67x00_do_work(struct c67x00_hcd *c67x00) +{ + spin_lock(&c67x00->lock); + /* Make sure all tds are processed */ + if (!c67x00_all_tds_processed(c67x00)) + goto out; + + c67x00_check_td_list(c67x00); + + /* no td's are being processed (current == 0) + * and all have been "checked" */ + complete(&c67x00->endpoint_disable); + + if (!list_empty(&c67x00->td_list)) + goto out; + + c67x00->current_frame = c67x00_get_current_frame_number(c67x00); + if (c67x00->current_frame == c67x00->last_frame) + goto out; /* Don't send tds in same frame */ + c67x00->last_frame = c67x00->current_frame; + + /* If no urbs are scheduled, our work is done */ + if (!c67x00->urb_count) { + c67x00_ll_hpi_disable_sofeop(c67x00->sie); + goto out; + } + + c67x00_fill_frame(c67x00); + if (!list_empty(&c67x00->td_list)) + /* TD's have been added to the frame */ + c67x00_send_frame(c67x00); + + out: + spin_unlock(&c67x00->lock); +} + +/* -------------------------------------------------------------------------- */ + +static void c67x00_sched_work(struct work_struct *work) +{ + struct c67x00_hcd *c67x00; + + c67x00 = container_of(work, struct c67x00_hcd, work); + c67x00_do_work(c67x00); +} + +void c67x00_sched_kick(struct c67x00_hcd *c67x00) +{ + queue_work(system_highpri_wq, &c67x00->work); +} + +int c67x00_sched_start_scheduler(struct c67x00_hcd *c67x00) +{ + INIT_WORK(&c67x00->work, c67x00_sched_work); + return 0; +} + +void c67x00_sched_stop_scheduler(struct c67x00_hcd *c67x00) +{ + cancel_work_sync(&c67x00->work); +} -- cgit v1.2.3