HID reports of more than 64 bytes and libusb backend

[JoergAtGithub]

At Mixxx, users reported problems with USB 2.0 High-Speed devices, which are limited to Full-Speed mode for some reasons. Under MacOS and Windows they operate as High-Speed device.

This is not only a performance limitation, but it has functional implications for the HID communication, because USB Full-Speed is limited to a package size of 64 Bytes instead of 1024 in High-Speed mode.
The result is, that HIDAP returns a 79Byte input report correct on Windows and MacOS, but two reports on Linux (libusb backend), the first with correct report ID, the second with report ID 0x00.

How could a cross platform application using HIDAPI handle this:

• Can the speed or max. package size be determined using HIDAPI to report an error about the USB misconfiguration?
• Can the application tell the device somehow to switch to USB High-Speed mode?
• Can HIDAPI check, if a report descriptor contains report sizes of more than 64Bytes for a device in Full-Speed mode?

Windows-Output:
Debug [Controller]: Traktor Kontrol S4 MK3 HID A79A_3: t:92905 ms, 79 bytes:
02 6A 08 00 00 00 00 03 07 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Linux-Output (libusb):
Debug [Controller]: Traktor Kontrol S4 MK3 A79A_3: t:15558 ms, 64 bytes:
02 6E 08 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Debug [Controller]: Traktor Kontrol S4 MK3 A79A_3: t:15559 ms, 15 bytes:
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

This is the lsusb output:
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface             11 Traktor Kontrol S4 MK3 HID
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.10
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength     673
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               1
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               2
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        4
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    189 
      bInterfaceProtocol      0 
      iInterface             12 Traktor Kontrol S4 MK3 BD
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x03  EP 3 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               1
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        5
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass       254 Application Specific Interface
      bInterfaceSubClass      1 Device Firmware Update
      bInterfaceProtocol      1 
      iInterface             10 Traktor Kontrol S4 MK3 DFU
      Device Firmware Upgrade Interface Descriptor:
        bLength                             9
        bDescriptorType                    33
        bmAttributes                        7
          Will Not Detach
          Manifestation Tolerant
          Upload Supported
          Download Supported
        wDetachTimeout                    250 milliseconds
        wTransferSize                      64 bytes
        bcdDFUVersion                   1.10
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  bNumConfigurations      1

[Youw]

Do I understand it correctly that the problem on Linux with libusb backend?

[JoergAtGithub]

Yes, it's reported by two independend Linux libusb users. I'm myself just relaying this, since I've neither the device nor Linux.

[Youw]

How could a cross platform application using HIDAPI handle this:

Can the speed or max. package size be determined using HIDAPI to report an error about the USB misconfiguration?

It is hard to define in a first place what is a "misconfiguration". From USB stack POV, if a device is successfully initialized in Full-Speed it is a successfull configuration. Otherwise it wouldn't be configured at all.
How would a client application know what it should expect (unless it is a known device, expected to run in a specific mode)?

And one more time: hidapi is agnostic of the device bus most of the time. On Windows/macOS/hidraw communicationis done using an OS API and there is nothing USB-specific.

Can the application tell the device somehow to switch to USB High-Speed mode?

It is only negotiated by the OS/USB driver/stack and device itself. The application has no control of this. Each device always tries to initialize itself in a highest possible mode, unless explicitly required otherwise by the device FW/HW limitations.

Can HIDAPI check, if a report descriptor contains report sizes of more than 64Bytes for a device in Full-Speed mode?

Again no cross-platform solution is possible. And right now there is no API to get the descriptor. And the USB speed mode is ony easily can be checked by libusb backend.

The device's FW is way mo easily can check/know current USB configuration.
If device's FW can be modified, I'd rather suggest to make an API and ask the device in which USB mode is it running.

[JoergAtGithub]

Modification of the the device firmware is not possible. The device is a commerical DJ controller, which is sold for use with the commercial DJ software Traktor Pro for Windows and MacOS. The users try to use it on the cross platform DJ software Mixxx under Linux, which is based on HIDAPI.
Mixxx works with this device on MacOS and even on a Windows 10 VM under Linux, but not if under Linux with libusb backend.

[Youw]

What about hidraw backend, as anyone tried it with that device?

If there is no known issues with Windows/macOS, I think it would be feasible to perform an explicit check on Linux using libusb directly, even before trying to open it with hidapi.

[JoergAtGithub]

hidraw is not yet tried. Further investigation is needed to understand this specific case.
But I think this is a general issue which could also occur with other HID devices, maybe also on other operating systems. Therfore it would be better to detect this somehow as error in hid_open instead of adding a check on application level.

[Youw]

That's what I'm trying to say: a USB/HID device running at Full-Speed - is not an error.

As for the inability to send packets larger than 64 bytes: this looks like an undesirable behavior of a specific device, but that's definitely not a HID issue.
HID protocol is not design to send large packets in a first place. It is specifically designed to send many small packets (2/4/8/16 bytes of data), and that's what is being done by all devices I've seen. I've had one device that has been using 64 bytes data packets, and that was considered an edge case.
For ceses when large data packets needs to be send between host and device, usually different protocols are used, like USB BULK.

[Youw]

Can HIDAPI check, if a report descriptor contains report sizes of more than 64Bytes for a device in Full-Speed mode?

After giving it a second though, we can call a case like this a "broken device" or "hardware missconfiguration" which can be detected by libusb backend of hidapi.
If someone is going to develop a patch to detect such a thing for libusb backend, I don't see strong reasons not to accept it.

[Be-ing]

I have a Native Instruments Traktor Kontrol S4 Mk3, one of the affected devices. Here is the output of lsusb -t for the device:

    |__ Port 7: Dev 25, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 29, If 0, Class=Hub, Driver=hub/2p, 480M
            |__ Port 1: Dev 30, If 4, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 1: Dev 30, If 2, Class=Audio, Driver=snd-usb-audio, 480M
            |__ Port 1: Dev 30, If 0, Class=Audio, Driver=snd-usb-audio, 480M
            |__ Port 1: Dev 30, If 5, Class=Application Specific Interface, Driver=, 480M
            |__ Port 1: Dev 30, If 3, Class=Human Interface Device, Driver=usbhid, 480M
            |__ Port 1: Dev 30, If 1, Class=Audio, Driver=snd-usb-audio, 480M

The HID component is at 480M speed, so I don't think the bug is in the kernel. The audio interface of the device works fine with Linux.

I don't know if the bug here is in libusb or hidapi's use of libusb. My hunch is the latter but I have no evidence to demonstrate that.

[Youw]

I don't believe libusb (as a result - hidapi too) has any control of how to initialize the device, select speed, etc. USB stack driver is responsible for that.

Do you have the same issue on Application level on your machine with this configuration?

@JoergAtGithub can you ask users who reported this issue to share the output of lsusb -t too?

[Be-ing]

It seems that libusb can only read a device's speed with libusb_get_device_speed() but has no API to set the speed.

[Be-ing]

Here is the lsusb -v output with formatting... the wMaxPacketSize 0x0040 1x 64 bytes for the HID endpoint is interesting...

Bus 001 Device 032: ID 17cc:1720 Native Instruments Traktor Kontrol S4 MK3
Couldn't open device, some information will be missing
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x17cc Native Instruments
  idProduct          0x1720 
  bcdDevice            0.60
  iManufacturer           1 Native Instruments
  iProduct                3 Traktor Kontrol S4 MK3
  iSerial                 2 69DDA7A8
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0124
    bNumInterfaces          6
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower                0mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         3
      bFunctionClass          1 Audio
      bFunctionSubClass       0 
      bFunctionProtocol      32 
      iFunction               0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      1 Control Device
      bInterfaceProtocol     32 
      iInterface              3 
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdADC               2.00
        bCategory               8
        wTotalLength       0x0053
        bmControls           0x00
      AudioControl Interface Descriptor:
        bLength                 8
        bDescriptorType        36
        bDescriptorSubtype     10 (CLOCK_SOURCE)
        bClockID               41
        bmAttributes            3 Internal programmable clock 
        bmControls           0x07
          Clock Frequency Control (read/write)
          Clock Validity Control (read-only)
        bAssocTerminal          0
        iClockSource            9 
      AudioControl Interface Descriptor:
        bLength                 8
        bDescriptorType        36
        bDescriptorSubtype     11 (CLOCK_SELECTOR)
        bClockID               40
        bNrInPins               1
        baCSourceID(0)         41
        bmControls           0x03
          Clock Selector Control (read/write)
        iClockSelector          8 
      AudioControl Interface Descriptor:
        bLength                17
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             2
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bCSourceID             40
        bNrChannels             4
        bmChannelConfig    0x00000000
        iChannelNames          13 
        bmControls         0x0000
        iTerminal               6 
      AudioControl Interface Descriptor:
        bLength                12
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID            20
        wTerminalType      0x0301 Speaker
        bAssocTerminal          0
        bSourceID               2
        bCSourceID             40
        bmControls         0x0000
        iTerminal               0 
      AudioControl Interface Descriptor:
        bLength                17
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Microphone
        bAssocTerminal          0
        bCSourceID             40
        bNrChannels             8
        bmChannelConfig    0x00000000
        iChannelNames          17 
        bmControls         0x0000
        iTerminal               0 
      AudioControl Interface Descriptor:
        bLength                12
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID            22
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               1
        bCSourceID             40
        bmControls         0x0000
        iTerminal               7 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              4 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       1
      bNumEndpoints           2
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              4 
      AudioStreaming Interface Descriptor:
        bLength                16
        bDescriptorType        36
        bDescriptorSubtype      1 (AS_GENERAL)
        bTerminalLink           2
        bmControls           0x00
        bFormatType             1
        bmFormats          0x00000001
          PCM
        bNrChannels             4
        bmChannelConfig    0x00000000
        iChannelNames          13 
      AudioStreaming Interface Descriptor:
        bLength                 6
        bDescriptorType        36
        bDescriptorSubtype      2 (FORMAT_TYPE)
        bFormatType             1 (FORMAT_TYPE_I)
        bSubslotSize            4
        bBitResolution         24
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x00d0  1x 208 bytes
        bInterval               1
        AudioStreaming Endpoint Descriptor:
          bLength                 8
          bDescriptorType        37
          bDescriptorSubtype      1 (EP_GENERAL)
          bmAttributes         0x00
          bmControls           0x00
          bLockDelayUnits         2 Decoded PCM samples
          wLockDelay         0x0008
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes           17
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Feedback
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval               4
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              5 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              5 
      AudioStreaming Interface Descriptor:
        bLength                16
        bDescriptorType        36
        bDescriptorSubtype      1 (AS_GENERAL)
        bTerminalLink          22
        bmControls           0x00
        bFormatType             1
        bmFormats          0x00000001
          PCM
        bNrChannels             8
        bmChannelConfig    0x00000000
        iChannelNames          17 
      AudioStreaming Interface Descriptor:
        bLength                 6
        bDescriptorType        36
        bDescriptorSubtype      2 (FORMAT_TYPE)
        bFormatType             1 (FORMAT_TYPE_I)
        bSubslotSize            4
        bBitResolution         24
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x01a0  1x 416 bytes
        bInterval               1
        AudioStreaming Endpoint Descriptor:
          bLength                 8
          bDescriptorType        37
          bDescriptorSubtype      1 (EP_GENERAL)
          bmAttributes         0x00
          bmControls           0x00
          bLockDelayUnits         2 Decoded PCM samples
          wLockDelay         0x0008
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface             11 
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.10
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength     673
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               1
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               2
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        4
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    189 
      bInterfaceProtocol      0 
      iInterface             12 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x03  EP 3 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               1
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        5
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass       254 Application Specific Interface
      bInterfaceSubClass      1 Device Firmware Update
      bInterfaceProtocol      1 
      iInterface             10 
      Device Firmware Upgrade Interface Descriptor:
        bLength                             9
        bDescriptorType                    33
        bmAttributes                        7
          Will Not Detach
          Manifestation Tolerant
          Upload Supported
          Download Supported
        wDetachTimeout                    250 milliseconds
        wTransferSize                      64 bytes
        bcdDFUVersion                   1.10

[Be-ing]

How do I tell hidapi to use the hidraw backend instead of libusb? I don't see any documentation for this. ./configure --help has no information. It looks like make builds both hidraw and libusb. If I uninstall the libusbx-devel package on Fedora, hidapi fails to build.

[Youw]

You can still have both hidraw and libusb backends of hidapi installed and use either one of them.
Technically those are two different libraries (even though with identical public API/ABI).
If you want to switch the backend for you application, you need to link against libhidapi-hidraw instead of libhidapi-libusb.

If you have a prebuilt application, it is already hard linked to one of the backends (presumably libhidapi-libusb in your case), and there is no "official" way to change it.