Edinburgh University

Division of Informatics:


A USB Monitor


Computer Science 4th Year Project Report:


Author: David Harding

Supervisor: Archie Howitt

30th May 2001














Abstract: A Monitoring system for analysing the traffic being sent to peripherals on a Universal Serial Bus. The system is composed of a patch to the Linux kernel and a user space monitoring application.
































Acknowlegments: This project would have been much more difficult without the support and advice of my supervisor, the helpful and relaxed  atmosphere of the hardware bunker, and all the musings of the Linux USB developers on the linux-usb-devel mailing list.

1     Introduction:. 9

1.1     Universal Serial Bus and the need for monitoring. 9

1.2     Possible Solutions. 9

1.3     Linux and USB.. 10

1.4     The Filesystem Interface. 10

1.5     The Monitoring Application. 11

1.6     Summary of Results. 11


2     Background:. 14

2.1     USB Structure. 14

2.2     Topology and Configuration. 14

2.3     Transfer Types: 16

2.4     Host Controllers. 17

2.5     The Future of USB.. 17

2.6     The Linux Kernel 18

2.7     The Linux Virtual Filesystem.. 18

2.8     Linux and USB.. 19

2.9     Linux USB Host Controller Drivers. 20

2.10      Linux USB Data Transfers. 21

2.11      The Linux USB filesystem.. 21

2.12      Linux USB development in the future. 22

2.13      USB on other Operating Systems. 22

2.14      Existing USB Monitoring Solutions. 22


3     System Definition.. 25

3.1     The problem with hardware interception. 25

3.2     The Kernel Alternative. 26


4     System Design.. 29

4.1     Functional Requirements of the kernel patch. 29

4.2     Design of the Kernel Patch. 30

4.2.1     Intercepting URBs. 30

4.2.2     Interpreting the Contents of a URB.. 31

4.2.3     Storing the Monitored Information. 31

4.2.4     Passing the Monitored Information to User Space. 32

4.2.5     Formatting the Contents of the Monitoring Files. 33

4.2.6     Setting the Monitoring Level of Devices. 34

4.3     Kernel Patch Implementation Issues. 36

4.3.1     Adapting the USB filesystem.. 36

4.3.2     Implementing Timing in the kernel 37

4.3.3     Problems with Interrupt Handlers. 38

4.3.4     Changes to different Host Controller Drivers. 38

4.3.5     Where to put the code. 38

4.3.6     Compatibility on different kernel versions. 39


5     System Design – The Monitoring Application.. 41

5.1     Functional Requirements of a Monitoring Application. 41

5.2     Attributes desirable in the Monitoring Application. 41

5.3     Design of the Monitoring Application. 42

5.3.1     Structure of the USB Model 42

5.3.2     Obtaining the Topology and Configuration Information. 43

5.3.3     Polling for the Monitored Data. 43

5.3.4     Displaying Configuration and Monitored Data. 44

5.4     Monitoring Application Implementation Issues. 45

5.4.1     Java and Swing vs. C and Gtk. 45

5.4.2     The USB Object Model 45

5.4.3     The Threaded Monitoring File Reader 45

5.4.4     Displaying the Device Tree. 46

5.4.5     Splitting Up the Information. 47

5.4.6     Displaying the URB Tables. 47


6     System Evaluation.. 49

6.1     Suitability of Core Design. 49

6.2     Effectiveness of Implementation. 50

6.3     Impact on the Linux USB Subsystem of the kernel patch. 50

6.3.1     Impact of Functionality. 51

6.3.2     The impact of Monitoring on processing time. 51

6.4     Performance of the Monitoring Application. 55

6.5     User Adoption. 55

6.6     Future Development 55


Bibliography:. 57


Appendix A     - The URB Structure. 61


Appendix B     - The pipe structure. 63


Appendix C     - The Pre-Completion Functions. 65


Appendix D     - The USB Model Java Classes. 67

Class Bus. 67

Class Device. 68

Class Configuration. 71

Class Interface. 72

Class Endpoint 74

Class URB.. 76


1         Introduction:


1.1        Universal Serial Bus and the need for monitoring


The Universal Serial Bus is a system for connecting peripherals to a host computer. It was created to give a simpler, higher-bandwidth, and more flexible replacement to RS-232 serial and parallel interfaces. The first Specification for the Universal Serial Bus was released in 1996 by a consortium of Intel, Compaq, Microsoft, and NEC. Hardware was soon available and the system has grown to becoming a standard feature on 100% of new personal computers [30]. By the end of 1998 over 10 million USB devices had been shipped[31].


            One of the key features of USB is that it allows the chaining together of devices in a tree so that many devices can be connected together via hubs to one port on a host system. This capability can bring with it a difficulty in analysing what is going on with the bus in general or with a particular peripheral. USB’s greater complexities also make it difficult for developers of both devices and drivers to accurately and effectively debug them. This has lead to a number of expensive hardware systems that can physically monitor the signals over a USB cable and interpret the signals in a way that can be analysed by a separate monitoring computer[1]. This is an expensive but wholly unobtrusive method of USB monitoring.


            The aims of this project were to create a system that allowed a Host computer with USB attached to operate normally while selectively monitoring the traffic to and from devices on that USB. The resulting system was intended to produce minimal interference in terms of performance and functionality of the USB hardware, OS drivers, and user mode applications.


1.2        Possible Solutions


There are different possible solutions to this requirement for monitoring. The most obvious is to intercept the signals in the wires connecting USB devices to a host system. For many reasons this is both impractical and only provides a limited set of monitored information (See Section 3.1 on page 25 for details).


The only realistic alternative to hardware interception is to monitor information inside the operation system. This makes any system produced operating system dependent which is unfortunate but unavoidable given the difficulties of hardware interception. There is no option of an operating system independent Java implementation as a standardised java-USB interface is still at an early stage of definition.[8]

1.3        Linux and USB


            Linux is an open source operating system under a very high degree of active development. Originally created by one man (Linus Torvalds) Linux is now composed of code written by thousands of developers around the world. This means that all the source and documentation is available in the public domain, even down to the discussions and debates of the principal developers. Linux has come late to USB support compared to more widely used proprietary operating systems but Linux use is growing, especially in some niche areas. International Data Corporation estimates that Linux commercial shipments will increase at a growth rate of 25 percent from 1999 to 2003, compared with a 10 percent growth rate for all other client operating environments combined. Many devices drivers are being ported to Linux[25] and new devices are emerging with their first drivers available in Linux. This combination makes Linux a desirable platform to develop this project on and a platform with a demonstrable need for a system like this.


            The Linux USB sub-system almost entirely resides in the kernel of the operating system (The structure of the Linux USB subsystem is shown in Figure 2 on page 20). Although user-mode device drivers are possible most device drivers are written to exist as kernel modules. These modules are written to interface with the core of the Linux USB sub-system; this then interfaces via a driver to the host controller hardware.


The principal intention of the monitoring system is to provide information as to what data is being transferred when and to or from which part of which device. In order to implement this the kernel has to be changed. This is because the monitoring system must have access to all the data transfers that occur on the USB. These transfers are split up in the kernel, as many of the device drivers exist in the kernel.


It is not desirable to have the whole monitoring system in the kernel. The kernel is a dangerous place to program due to the damage that can be done. It also has much less library support for user-mode style applications. So the desired system must consist of two parts: a modification to the Linux kernel and a user space monitoring application.


1.4        The Filesystem Interface


The two parts of the system need to communicate. The user-space monitoring application must tell the kernel which transfers to monitor and the kernel based monitoring system must pass the monitored information back to user-space so that it can be displayed to the user. Moving data from the kernel space to the user space and vice versa is a complex issue. After a lot of deliberation the best method was determined to be using a filesystem interface so the user-space application could use standard file access functions to query the kernel space. An added advantage was that this could be implemented as an extension to an existing feature of the Linux USB sub system.


            Using this system allows maximum flexibility in the implementation of monitoring applications, as they require no new interface capabilities other simple file access. Simply opening a particular file and writing a set of characters will set the monitoring level for a particular connection on a particular USB device. Reading from the same file will dump the monitored data down to the user space monitoring application. This turns the system from a symbiotic pair of programs into a prototype monitoring architecture with an example monitoring application that supports the additional functionality provided by the kernel modification. This is a theme that exemplifies the difference between kernel programming and application programming. The kernel provides specific services, which can be used in different ways by different applications.


1.5        The Monitoring Application


            The sample monitoring application can be implemented in any programming language. This is because of the standard file access nature of the interface between the modified kernel and the monitoring application. The monitoring application uses normal file access to set the level of monitoring and read the monitored data buffers from the kernel.


The existing Linux USB subsystem provides a ‘file’ from which contains the topology and configuration of the buses. The content of this ‘devices’ file enables the monitoring application to build up a model of the bus. With this model the Monitoring Application has enough information to set monitoring levels in the modified kernel for particular types of transfer on particular devices. The monitoring application can then interpret the monitoring output of the modified kernel so as to display this information usefully to the user.



1.6        Summary of Results


            The system that has been produced consists of a 22kb patch to the 2.4 Linux kernel tree and a Java monitoring application. This combination is capable of selectively setting the monitoring level for different types of transfers on different devices, intercepting those transfers, and retrieving details as well as (optionally) the full data buffer. This information can then be retrieved from the kernel to user space and displayed in a graphical interface. This interface allows the comparison of different transfers visually and the analysis of the data contents using different encoding methods.


            This system has been released to the public domain under the title USBMon via the World Wide Web. All information relating to USBMon has been released at http://www.dcs.ed.ac.uk/~dxh/public/USB/. It is the only system available for Linux that can obtain real-time USB monitoring information from USB devices in Linux with no additional hardware. It also provides a combination of features unavailable on other proprietary operating systems.

Figure 1: USBMon showing details of monitored URBs on a USB floppy drive









2         Background:


2.1        USB Structure


            The universal serial bus was created as a replacement for the venerable RS-232 serial and parallel communication standards that have featured on almost every personal computer ever made. USB was intended to remove the need for costly proprietary interfaces to ISA or PCI busses or the use of costly SCSI interfaces for peripherals such as scanners and removable hard drives. The primary deficiencies in the serial and parallel interfaces were seen as being:


·        Limited Bandwidth – UART technology and types of cable specified by RS-232 limits the potential bandwidth of a serial connection to little over 100 kbits/s


·        Lack of Chaining – No matter how many serial ports a machine had it might never have enough and so the ability to connect more than one device to a single port was seen as necessary.


·        Lack of power distribution – With the exception of mice in most cases a serial or parallel connection cannot power the peripheral. A greater degree of power distribution was desired.


USB as defined by the 1.1 Specification[2] made the following characteristics widely available:


·        Up to 127 devices connected together on one bus utilizing up to five layers of hubs.


·        Total bandwidth capability of 12MBps.


·        Devices may each draw a maximum of 2.5W of power[3] from the bus.


2.2        Topology and Configuration


USB defines various entities that have specific meaning in a USB context: Bus, Device, Configuration, Interface, and Endpoint. With the exception of a Bus each of these entities has a descriptor that holds various characteristics. The descriptors are available to the host system as soon as a device attaches to the bus even if there is no device driver for the device. They are part of the topological and configuration information.


A USB Device has the following characteristics:


·        A Device has a Class, Subclass, and Protocol. These correspond to USB defined types of device[4]. For example Classes are defined for mass-storage, audio, modems, scanners, etc. These classes have different sub-classes and can conform to various USB defined protocols. It is possible for a device to be vendor-specific in which case it’s subclass and protocol values are not relevant. It is also possible for Devices to maintain different interfaces of different classes in this case the Device is not of a specific class but some of its interfaces have their own class.


·        A Device also carries values for it’s manufacturer serial number and version numbers as part of the device descriptor.


·        A Device has one or more possible configurations, one of which is active at any one time.



Each Configuration has the following characteristics:


·        A Configuration is either bus-powered or self-powered, and either capable of remote-wake up or not.


·        A Configuration has a maximum power usage specified in its descriptor


·        A Configuration has one or more Interfaces.



An Interface has the following characteristics:


·        An Interface has Class, Subclass, and Protocol. These are similar to the characteristics of Devices but relate only to a particular interface.


·        An Interface has one or more endpoints.



An Endpoint has the following characteristics:


·        An Endpoint is either an Input or an Output


·        An Endpoint is of a particular type relating to the type of data transfer that is possible:


o       Control

o       Bulk

o       Isochronous

o       Interrupt


·        An Endpoint has a Maximum Packet Length that it can send or receive.


·        For Interrupt Endpoints only there is an Interval value that signals the time between interrupts.






2.3        Transfer Types:


            The four different types of transfer are:


·        Control – This is a data transfer that attempts to change the configuration of the device. This can include a bulk-style data transfer.


·        Bulk – This is the lossless one-time bulk movement of data to or from the device.


·        Isochronous – This relates to a steady stream of real-time data where the data carries implicit timing in the rate it arrives. Such transfers require a set allocation of bandwidth. This type of transfer is normally reserved for real-world streamed data such as an audio channel or video stream from a camera.


·        Interrupt – This transfer type can transfer a set size of data at regular intervals, timed in milliseconds.



2.4        Host Controllers


            The USB specification is general enough to allow different implementations of the Host Controller hardware. There are two common implementations in wide usage, these are known as UHCI[6] and OHCI[5]. UHCI was created by Intel and is considered a more simplistic hardware implementation that requires slightly more software management. OHCI was created by Compaq and is commonly found in all Apple USB implementations. The two different host controllers both completely support the USB standard and will work with all USB devices; the only thing that must change is the Host Controller Driver on the Host Computer. [4]


            As part of the USB 2.0 improvements (as discussed below) a new host controller has emerged (the only one to support 2.0). This is called EHCI. Having an EHCI host controller will imply the ability to cope with USB 2.0 devices.


2.5        The Future of USB


            At about the same time as USB appeared another standard was announced called IEE 1394, sometimes branded as FireWire[5] an i.link[6]. IEE 1394 does not have all the chaining capabilities of USB but it does have vastly greater bandwidth (400MBPS) than USB. IEE 1394 also has the ability to carry out peer-to-peer connections instead of one acting as a host and the other as a much simpler device.


            The emerging consumer digital multimedia market has adopted IEE 1394 widely[7]. It is the most common interface on new digital camcorders and is now present on high-end games platforms. This has demonstrated the demand for higher-than-existing-USB bandwidth when connecting some peripherals, especially in the consumer electronics sector. This has spurred the extension of the existing 1.1 USB specification. In December 2000 the USB 2.0 Specification[3] was released. The principal advantage of USB 2.0 is a new higher bandwidth capability of 480MBPS.


            Currently the adoption of USB 2.0 and IEE 1394 is of particular note. There is uncertainty about the adoption of the two standards as many hardware producers wish to standardise on one system. It is currently uncertain as to whether USB 2.0 will be able to usurp IEE 1394’s dominant position in the consumer electronics market but it is unlikely that IEE 1394 will replace USB as a standardised method for connection of peripherals to Personal Computers. An IEE 1394 mouse is a little far fetched. It is unclear whether IEE 1394’s better peer-to-peer networking capability will make it more popular for the interconnection of broadband digital consumer devices.[29]



2.6        The Linux Kernel


            Due to its open source distribution the Linux Kernel is extremely configurable and it is very common for experienced users to recompile their own kernel with slightly altered configuration options. The principal area of configuration is what parts are included in the kernel as statically linked parts of a macrokernel and what as dynamically loadable modules, and what is not included at all. The Linux USB sub-system can be implemented in whole or in part as dynamically loadable modules. It is common to keep the Linux-USB core statically linked while dynamically loading the different device drivers. Other features that can be configured in this way is which Host Controller Driver is used by the system and the level of debug information sent to the kernel system log.


            The latest stable version of the Linux kernel (at the time of writing) is 2.4.5. Version 2.4.0 was released in January 2001; this was a major update from the previous stable kernel 2.2.17. Minor updates (changes to the third number) happen quite often and do not usually signify major change. Kernel versions with odd middle numbers (e.g. 2.3.x) are unstable kernels that are used for development and experimentation.



2.7        The Linux Virtual Filesystem


            The Virtual File System (VFS) is a core part of the Linux Operating System and key to much of its functionality. Linux can mount new filesystems within one unified file-structure. Not all files that appear in the unified filesystem are sitting on the same disk; in fact not all files are actually files on any disk. The Virtual File System allows storage devices that use very different formats to be used together in one filesystem. The VFS has a set interface and filesystems are written to support it. This results in the details of a particular filesystem being completely transparent from the user. The user cannot tell (from how it is accessed) if a file is held on a DOS or EXT2 partition[8] of an IDE or SCSI hard drive or even a Zip drive on a parallel port interface, it is completely transparent.


            This flexibility and transparency combined with a very strictly defined interface that has not changed much over time has lead to the use of the Virtual File System for different purposes. Files in the VFS can represent devices and services. Direct interfaces to devices are often found in the /dev file-space. Services such as CPU usage reporting are reported through files in the /proc filesystem.


            In Linux a filesystem has an entity called a superblock. The superblock is used to register and mount the filesystem. The superblock contains all details of how a filesystem is accessed and points to functions that define inode structures for particular files. An inode is the kernel reference to a file in a filesystem. The principal function of the Virtual Filesystem is to cache inodes and manage their access. User-programs never actually get to find out the inode of a file. The user program is given a file-descriptor, which acts as a pointer to a reference of the actual inode of the file. This indirection is a feature of the Virtual File System cache.


            An inode structure contains details about the file such as modification time, ownership (user and group), as well as a pointer to a file_operations structure. The file_operations structure contains pointers to functions that implement the possible actions on a file such as open, read, write, release, ioctl, etc.


2.8        Linux and USB


Linux adopted USB later than its proprietary rivals. USB support has only really become widespread since the release of the 2.4 kernel[9], which was first released in January 2001. Major Distributions have adopted this kernel in subsequent months and now USB support is a standard feature of most new Linux installations.


            Support for USB devices in Linux is now growing dramatically, Major device classes are well supported such as the Human Interface Device (HID) driver (this supports mice, keyboards etc.) and the mass-storage driver (this is used by floppy drives, Zip drives, CDRs, even some MP3 players and Digital Cameras). Proprietary interfaces have also begun to be supported; high profile examples include Alcatel’s ADSL USB modem and Phillips popular digital cameras[25].


            The Linux USB Subsystem has been developed by a group of developers spread over the world communicating via the public mailing lists.  The basic design is shown in Figure 2. The Linux USB subsystem is a part of the kernel. It is composed of three layers: The Device Drivers, the USB Core, and the Host Controller Driver. Due to the nature of the distributed and disjointed development of open source systems the interfaces between the layers have become tightly defined. This means that there is a tightly defined interface between drivers and the USB core, and the USB core and the Host Controller driver.


            Linux USB device drivers are normally kernel based drivers that interface to user applications via a number of methods such as the /dev device filesystem and the SCSI filesystem where USB storage devices are represented as if they are SCSI storage devices. It is possible however, to have user-space based device drivers. These user space drivers interface to the USB core via the Linux USB filesystem.

Figure 2: The existing Linux USB Subsystem

Diagram showing the layout of the Linux USB sub-system.


2.9        Linux USB Host Controller Drivers


            Linux has a stable driver for the OHCI host controller, it is commonly known as usb-ohci, however Linux has had several problems with its implementation of the most common UHCI host controller. Two different implementations now exist, known by the filenames of their object code: uhci and usb-uhci. Both drivers are under active development by different people but use significantly different internal methods. The history is complex but in brief the uhci driver is older and for a long time had problems with isochronous transfers; while the newer usb-uhci driver was developed to use similar internal structure and methods as the OHCI driver. There is active discussion about how best to move to a simpler situation for UHCI users. This, along with early development of the EHCI driver has lead to a strong discussion of the possibility and desirability of merging common functions in the different host controller drivers in what is referred to as a Host Controller layer. The exact future of this is currently uncertain.


In theory it should not matter to users or device driver writers which host controller driver a system is using, as the interface to the USB core is common between all the Host Controller drivers.

2.10    Linux USB Data Transfers


            The Linux USB core handles all four types of data transfer (control, bulk, interrupt, and isochronous) via one structure. This is called a URB (Universal Request Block). A URB contains all important information about a data transfer including a pointer to the actual data buffer. URBs are queued asynchronously and can trigger completion routines via a callback function specified in the URB (this is also extremely important to how monitoring can be achieved). The Host Controller determines the exact scheduling of the transfers. The URB structure contains parameters that are required to be set before a URB is submitted and results that will be filled in by the core and returned when the URB has completed.


2.11    The Linux USB filesystem


The Linux USB filesystem has its own filesystem. This can be mounted in the /proc filesystem at /proc/bus/usb. The /proc filesystem is a memory-only filesystem that allows the kernel mode to send and receive data as if it were the contents of a file being read from or written to respectively. When a file in the /proc filesystem is read or written the filesystem securely calls kernel mode functions that reply to the request with dynamically generated contents of the requested ‘file’.


The Linux USB filesystem is intended as both a method for relaying from the kernel to user mode applications the configuration details of the USB peripherals and as a method for user-mode device drivers – allowing the sending and receiving of URBs from user-mode processes.


The configuration information is found in two files in the top level of the usb filesystem (/proc/bus/usb). These files are called ‘devices’ and ‘drivers’. The ‘drivers’ file consists of a list of the currently loaded drivers. The ‘devices’ file contains the details of the bus topology and the contents of all the device descriptors of attached devices. This includes all characteristics of devices, configurations, interfaces, and endpoints. A detailed explanation of the rather complicated formatting of this file is in [15]. 


The device interface part of the Linux USB filesystem is provided in bus directories. There is one directory per bus, the name being the three-digit bus number allocated by the USB core. For single bus systems this will almost always be 001. Inside these directories there is a file for every device, the name being the three-digit device number allocated on connection. So the device interface for device 1 on bus 1 is at /proc/bus/usb/001/001. These device interface files can be used to submit data transfers and carry out other actions on the devices.





2.12    Linux USB development in the future


            There has been much discussion among the Linux USB developers about the future direction of development effort. One of the prime efforts is USB 2.0 support. As well as this there is active discussion of standardisation within Linux of the way peripheral interfaces such as PCI, USB, IEE 1394, IRDA, and possibly Bluetooth represent devices and their configuration. This has already led to a standardisation of the arrangements for Hot-plugging devices. The Hotplug system is more general than just the USB subsystem and is used by other parts of the operating system.[22]


            The important aspects of this are really what is not about to change rather than what is. There is little prospect of the URB system and the Host Controller interface changing even with the introduction of USB 2.0. The standardisation between peripheral interfaces in Linux should not affect the internals of the USB core or device driver design to any great extent (to break existing drivers on a large scale would be very bad and is extremely unlikely).



2.13    USB on other Operating Systems


USB has been extremely widely adopted. In 1998 there were 138 million USB enabled PCs in the world and that is projected to increase to over 500 million in 2001 with 100 per cent of new PCs supporting USB[30]. This level of support from the hardware providers is matched by almost all widely used operating systems. Different versions of Windows[27] and Mac-OS[28] both have extensive USB support. Because these different operating systems are closed source and tend to have more single-user orientated security models the position of the USB drivers is different. There are often things called ‘device layers’ which make all devices look similar to drivers whatever interface they have. This means that monitoring the goings on of a USB is very different in other operating systems.



2.14    Existing USB Monitoring Solutions


There is an application called USB Snoopy[19] available for Microsoft Windows 98 that allows the analysis of data sent between drivers and devices. This is often very useful to software engineers who are attempting to write ‘port’ a driver to a new operating system such as Linux. USB Snoopy is still at a delicate stage of development and can cause problems with other Windows applications. A Linux USB device driver called usb-robot[20] has been developed that can use the output of USB Snoopy to replay USB transactions on devices where there is no existing Linux device driver, this is used to reverse engineer drivers.



Figure 3: The main window of USBview showing a well populated bus.

On Linux there is no equivalent to the USB Snoopy Windows application in Linux; that is partially the aim of this project. There is an application available for Linux called USBview[21] that displays the topology and configuration of the USB attached to the host. Figure 3 shows a screenshot from USBview. USBview only shows information available in either the devices or drivers files in the Linux USB filesystem. This data is presented in a split frame with a tree format showing each device as a node on the left and the details of a selected device in the right-hand.


The functions that USBview provides are often inadequate for device developers and driver writers. These users need not only to be able to tell what configuration and topology devices and busses have, they need to know exactly what is being communicated and when between the device and the driver. This has resulted in a significant market for expensive hardware monitoring systems that physically monitor the wires in a USB bus. Providers of these include Intel, Cypress, and CATC. These devices are normally extremely expensive and require an additional monitoring computer to analyse the data that they produce.

3         System Definition


3.1        The problem with hardware interception

Figure 4: The traditional hardware monitoring method


            The most logical solution to USB monitoring on any platform is to unobtrusively monitor the electrical signals on the cable just outside the host system, analyse these signals into data transfers, and pass these data transfers to a monitoring application.



            The principal advantage of this method is that there is no impact on the operation of the system. No component of the USB system does anything it would not do in a normal USB set-up. This means that problems that appear when not monitoring are not likely to be obscured by the presence of monitoring and new problems are unlikely to be triggered by the presence of monitoring. This makes this solution very desirable to device developers and device driver writers attempting to debug exactly what is happening on a system. This type of system is also very desirable for reverse-engineers attempting to work out how closed source device/driver combinations are talking to each other.


            The main problem with this method is the technology required to perform it. The Monitoring hardware must be fast enough to analyse the electrical signals on the bus, and interface with a communication pipe fast enough to communicate all the data appearing on the USB. This means both making something almost as complicated as a host-controller and building a communication pipe with a bandwidth greater than that of USB (as it must take the maximum USB traffic plus an overhead of Monitoring configuration information). Since one of the reasons that USB was created was to increase the available bandwidth of peripheral connections this is difficult. IEE 1394, and a direct PCI interface are possibilities but both carry heavy implementation costs in both price of components and complexity.


            Given that this style of solution is offered by several large companies at high cost and the feasibility of completing this style of monitoring within the scope of this project it was decided not to attempt this option.


3.2        The Kernel Alternative        


            The alternative is to intercept the data transfers when already inside the host system. As is shown in Figure 2 on page 20 the Linux USB Subsystem is composed of three layers, Host Controller Driver, USB core, and Device Drivers. As different device drivers interfaced with the USB core only access information relating to the specific device they are interested in this top level is too far into the host system for effective monitoring of the USB as a whole. Individual device drivers are capable of providing their own debugging features if they want. Added to this device drivers are often distributed in binary only closed source form (device drivers are an exception in the open-source nature of the GNU public licence under which Linux is released). Binary only device drivers could not easily be modified to include added debugging features. It is therefore desirable to implement monitoring after the signals are decomposed by the hardware Host Controller and before the communication is split up and passed to different device drivers.


            This leaves two possible locations for effective USB monitoring: the USB core and the Host Controller Driver. Monitoring inside the host controller driver would give access to much more information. The host controller is responsible for the scheduling of actual transactions and this can be directed and determined by the driver and as such the timing over individual parts of data transfers could be monitored. On this evidence the host controller driver seems a very sensible place to monitor a USB; however there are several problems.





Figure 5: The Kernel Monitoring Method


            Practically, the number of different drivers and host controllers complicates making major additions to host controller drivers. The two most common host controllers (UHCI and OHCI) are significantly different in the amount of scheduling work that is carried out in the hardware. As such the variety of events and values that can be measured would be different on different host controllers and in the case of UHCI host controllers the measurable values would differ depending upon which driver was loaded. These differences would lead to radically different monitoring systems for the different combinations. Creating a unified interface for this style of monitoring would pose several challenges.


As well as these practical issues there is a strong argument that the additional information that can be garnered from within the host controller driver is only really relevant to host controller driver design and implementation which is a very narrow market compared with device developers and device driver writers. Why do device driver writers care about scheduling they have no control over? If Host Controller Driver writers want more information about what Host Controller Drivers are doing they can insert debugging calls themselves.


            So by a process of elimination the USB core is the only place to implement a kernel based monitoring system. This is the most sensible location because it is the only level that operates on a granularity of data that is the same as the device drivers while having easy access to all data transactions occurring on the bus. The information is the same across all host controllers and as such is relevant to the device driver developer.


            Having decided to monitor in the USB core the unit of monitoring becomes clear. This is the URB (Universal Resource Block). This is the structure used by the device drivers to submit transactions and understand their results. Events indexed by URBs are a logical frame of reference for device driver writers and a simple one for device developers to understand. The URB structure is defined in Appendix A.

4         System Design


The system design is clearly split into two parts: the kernel patch and the monitoring application. The kernel patch is a list of additions (and deletions) to the kernel source. This can be applied to a kernel source, the kernel then configured as to the system’s requirements in the usual way and compiled.


The monitoring should be flexible. Monitoring all the data that is passed to or from an endpoint may be excessive. So three levels of monitoring are defined – No Monitoring, URB Headers only, and Full Data Monitoring.


            The Monitoring Application is a user-space application that can set the monitoring level of the kernel element and read its output and display this information in a useful way.



4.1        Functional Requirements of the kernel patch


            The patch to the kernel needs to enable three things to happen:


1.      URBs passing to and from the host computer must be selectively intercepted. This will involve:


a.      Trapping all URBs after they have completed but before the driver is notified of completion.


b.      Analysing the trapped URB and determining what (if any) details to monitor of it, by referring to a maintained list of monitored endpoints.


c.      Additionally the monitored URB should be time-stamped at the time of monitoring.


2.      The relevant information obtained from a URB must be stored in a buffer. Execution must then return to the normal kernel operations.


3.      The information in the buffer must be transferred to user space when stimulated by the monitoring application.


4.      The kernel patch should be able to interpret a correctly formatted command sent from the monitoring application and set the monitoring level appropriately


Attributes that should be present in the kernel patch


1.      The patch should have minimal performance impact on USB transactions that are not being monitored. This should mean less than a few hundred extra processor cycles for a non-monitored URB to complete.


2.      The patch should not in any way affect the operation of any other kernel service or user space application. All existing kernel services should be completely usable with the patch in place.


3.      Changes to existing functions in the kernel source should be kept to a minimum so as to reduce the likelihood of developmental changes to the kernel tree breaking the patch.




4.2        Design of the Kernel Patch

4.2.1       Intercepting URBs


The URB structure is defined in usb.h (in the kernel include files) and is included by all the core source files (see Appendix A for URB definition). The URB contains a pointer to a completion function that is key to the monitoring.  The completion function is a device driver function that is referenced when the device driver submits the URB to the USB core. This function is then used as a callback as soon as the URB has completed. This callback is in fact called from within the hardware interrupt handler in the Host Controller Driver that is caused by the completion signal from the Host Controller[10].


Initially the idea was to replace this completion call with a new function that performed the monitoring required and then called the original device driver completion function. This was simplified slightly to adding in a new function that is called just before the completion call, with the Host Controller driver’s call to the completion function remaining in the code. So when modified the Host Controller driver will call a ‘pre-completion function’ that will perform the monitoring then the Host Controller driver will call the original device driver defined completion call.


The advantage of this method is that it means less change to the URB structure and causes less intermediate processing to all URBs, thus slightly reducing the performance impact of the kernel patch. The principal disadvantage is that this requires a modification to several places in each of the host controllers. This is an annoyance from a design point of view as it extends slightly the interface between the USB core and the Host Controller Driver. This decision is finely balanced between marginal performance impact versus source code ugliness and reduced future flexibility (in that any new or radically changed host controller driver will also require minor modifications). This is a deviation from the original plan where it was aimed to only modify the USB core while leaving the Host Controller Drivers unaffected by the kernel patch.


4.2.2       Interpreting the Contents of a URB


            It is all very well calling the pre-completion function for every URB but not all URBs will need to be monitored. For example it may be that a system includes a USB keyboard, which will cause a URB to occur every time a key is pressed.


            The URB structure stores the information about the device, endpoint, direction, and type of the URB in a structure called a pipe. This is in fact an 18-bit structure encoded in an unsigned integer (see Appendix B for details). Assuming that the pre-completion function has access to a list of monitored device-endpoint combinations (see section 4.2.6 on page 34) it can use values that represent masks of the pipe structure to check if the monitored URB is on the list of monitored pipes. So the pre-completion function will take a URB as a parameter and compare its pipe with each of the pipe masks it has in a list of monitored pipes.


            Once the pre-completion function has determined that the URB is one that should be monitored it can copy all the appropriate details into a new URB structure (including the data buffer depending on monitoring level). The pre-completion function must also timestamp this new URB. There is no timing information available within the Linux USB subsystem so this is an addition to the URB structure in the kernel patch.


4.2.3       Storing the Monitored Information


Once a URB has been copied the copy must be stored somewhere. It cannot immediately be sent to user space as it is being copied in the middle on an interrupt handler so time is of the essence. The monitored URBs must be stored in a buffer. All the monitored URBs could be held together in one big list. The disadvantage of this is that user-space applications would have to obtain the whole list when they may only want to see if one device has any URBs in its buffer. Separating the monitored URBs at the device level allows multiple different monitoring applications to operate concurrently and monitor different devices. The reason for this is explained below in 4.2.4 Passing the Monitored Information to User Space.


            The list of monitored URBs could be separated at the endpoint level, however this would add a small amount of extra processing on every monitored URB (calculating the endpoint, and then finding the correct endpoint structure) as well as increasing the complexity of transporting URBs to user-space. There are no obvious advantages in separating the monitored URBs at this level, as it is very unlikely that two concurrent monitoring applications would want to monitor different endpoints on the same device. The most logical answer then is to separate the URBs at the device level maintaining a separate buffer for URBs on each device.


As well as the pipe information the URB carries a pointer to a usb_device structure. One usb_device structure is maintained for each USB device that is currently on the Bus. This structure contains details about the device and references to structures describing the configurations. This structure provides the perfect location to store the monitored URBs. The functions that pass the URBs to user-space can easily access the correct usb_device structure; this is explained in section 4.3.1.


4.2.4       Passing the Monitored Information to User Space


Moving data from the kernel mode to the user mode and vice versa in a relatively secure operating system such as Linux is surprisingly difficult. The most classical method that was looked at for some time is the idea of new system calls that could be called from user mode to both set the monitoring level and request the dumping of a buffer of monitored URBs down to user mode. This method is largely workable but has various side effects that make it undesirable from an operating system design point of view. Firstly this method has some significant security issues in allowing the dumping of kernel space memory on request; there is ample room for the introduction of security holes. Also the new system calls would have to be statically compiled into the Linux kernel. This would change the nature of the USB sub-system, as at the moment it is possible to compile it all as dynamically loadable modules. Another problem would be the enumeration of new system calls. It is entirely probable that new system calls will be introduced to Linux in the future and these would then introduce incompatibilities with this kernel patch.


            A simpler method to move data from the kernel space to the user space is to use the /proc filesystem. Since each ‘file read’ on a file in the /proc filesystem is serviced by a definable kernel function this function can be set to dynamically generate the correctly formatted URBs. This system can therefore dump a monitored URB buffer from the kernel mode to a user application.


            The creation of an individual /proc file in kernel code is quite easy, however it is much simpler in terms of creating the correct files at the same time to use the Linux USB sub-system’s own filesystem that is mounted within the /proc structure. This filesystem contains a directory for every maintained bus with each bus directory containing a file for every device on that bus, with the three-digit device number (allocated by the USB core on connection) as the filename.


            So as not to break any currently working system all these files and their behaviour should not be changed. Under the modified kernel a new file could be added for every device. As well as a “***” file (where *** is the device number) a second “***M” file would also be created. These monitoring files would be set up in the same way as the existing device files but with different ‘read’ and ‘write’ calls. When a URB for a monitored endpoint completes it’s relevant details are copied to a monitoring buffer. When this file is next read then the buffer is outputted to the file. This action empties the buffer, thereby limiting this system to one monitoring application in operation at any one time per USB device. This is a limitation of this design and there is no simple method around it.


4.2.5       Formatting the Contents of the Monitoring Files


            The content of the monitoring file is the information that has been monitored by the kernel and is being passed to the user-space application. This information can contain two main parts: the header information and the actual data buffer. Only on endpoints set to ‘Full Data Monitoring’ is the data buffer included. The URB header contains values from the URB structure that could be useful to monitor as well as a time stamp generated at the time that particular URB was monitored. The values from the URB structure that are passed over are: device number, endpoint number, pipe, status, size, transfer flags, actual length, and error count. The meaning of these values is shown in Figure 6.


Device number

The number (as given by the USB core at connection) of the device this URB was sent to or from

Endpoint number

The number of the endpoint this URB was sent to or from


A structure containing device and endpoint numbers as well as transfer type and direction.


The status of the URB when monitored. This indicates whether it completed or failed in some way.[11]


The intended size of the data transfer

Transfer Flags

Flags that can affect transfer scheduling

Actual Length

The actual size of the associated data buffer.

Error Count

The number of physical transmission errors that occurred during the transfer of this URB

Figure 6: Table showing Items in a URB header


The information must be structured in such a way that the monitoring application can parse it easily and unambiguously. It is also useful for debugging and advanced use of the system that the content is easily readable by a user.


            A decision was taken to use plain ASCII text to convey information about a URB. This fits with the two attributes above and also continues the style of most /proc files. Linus Torvalds (the original author of Linux) has stated that binary data should only be present in the /proc filesystem when absolutely necessary [13].


For the URB header information plain ASCII text is fine, but the data buffer of the URB must be sent in full binary format. The header information contains details of the lengths of the data buffer. The structure of a monitored URB being reported in a monitoring file is shown in Figure 7.


[URB completed on Device=%03d, Endpoint=%02d, Pipe=%08X, status= % 02d, size= %08d, transfer_flags = %08X, actual_length = %08d, error count = %05d completion_time = %02d:%02d:%02d:%06d dp=%01d data=]\n


Figure 7: URB encoding in a Monitoring File (Number values are shown in C format where %2d is a 2 digit decimal, and %08X is an 8 character long hexadecimal number)



In order for the Monitoring Application to easily parse the content of the monitoring file it can be important that the length of the ASCII header is fixed. This can be achieved by placing leading zeros at the beginning of possibly multi-digit numbers. In C notation this is represented by a 0 before the number of digits in a number modifier, i.e. %02d instead of %2d for a 2 digit number. Note that the status number in Figure 7 uses the C notation %[SPACE]02d and opposed to %02d. This means that if the status is negative (which is possible) a minus sign will be inserted and if positive a space inserted so that the length of the header will not change. Keeping the header length constant is important for some parsing systems as some systems deal very differently with ASCII and binary information.


4.2.6       Setting the Monitoring Level of Devices


            Given that the ***M monitoring files are being created anyway to be read from, a logical extension of this idea is to use the writing case to set the monitoring level. A simple command structure can easily be defined where a few bytes represent a monitoring level setting command. Because there are monitoring files for every device the command need only state the particular endpoint and the desired monitoring level. The formatting of the command need not be very complex at all. One option would be simply to have a two-byte command where the first byte is the endpoint number and the second byte is the monitoring level (in fact given the maximum number of endpoints this could all be encoded in one byte), however this is too inflexible for future development. It was decided that a four-byte command structure where the first and third byte is fixed would provide greater redundancy for future features. The first and third byte can be verified by the kernel as a check against accidental input from another source.


Byte No.







Endpoint Number


Monitoring Level

Figure 8: Table containing the byte coding of commands to set the monitoring level. All bytes are plain-text encoded ASCII.


            Once the endpoint value and monitoring level (and the device and bus numbers derived implicitly from the path of the file) are known by the ‘write’ function in the filesystem (remember a ‘write’ means information going in to the kernel) it must store them somewhere. The two considerations on deciding where to store this small amount of data (the endpoint to be monitored and the level at which to monitor) are that the location must a) be easily known by the filesystem call that must write to it and b) be easily known by the pre-completion function during the monitoring of URBs. Of these two factors the second is much more important as the pre-completion function is inherently the most performance-sensitive part of the system.


An option would be to have a globally visible list of monitored device-endpoint combinations, however this is seen as bad in terms of kernel design due to it’s lack of scalability and reliance on one unified global structure. In fact, as it turns out, both functions can have very easy access to the usb_device structure of the relevant device. This is possible for the filesystem function because of a field in the file structure, which is used in the kernel to represent files. There is a space called ‘private_data’ where the files system can hide a pointer to a structure of its choice for later reference. For the data outputting functions a pointer to the appropriate usb_device structure was already being placed there so that the reading function (a user ‘read’ means a kernel output) had access to the monitored URBs. This can be re-used in the writing functions to allow the setting of a monitoring level variable in the usb_device structure.


At first it seems odd for the pre-completion function to have easy access to the relevant usb_device structure as it is dealing with URBs appearing on all devices not just one at a time, but in the URB structure that it gets passed as a parameter there is a pointer to the relevant usb_device structure so the pre-completion function will always be able to reference straight to the correct usb_device structure.


 Maintaining a list of monitored endpoints on a per device basis has significant implications on the performance of all URB transfers during any monitoring compared with keeping a unified list of all monitored device-endpoint combinations. The speed with which the pre-completion function can determine whether the URB it is analysing is to be monitored or not is proportional to the number of comparisons it must make with device-endpoint combinations. Using this method of storing the monitored endpoints of a device in the usb_device structure reduces the number of comparisons to the number of monitored endpoints in that device rather than on the whole system. This means that performance is reduced only for a device on which there is monitoring being performed, others are largely unaffected.


4.3        Kernel Patch Implementation Issues

4.3.1       Adapting the USB filesystem


            Within the Linux USB subsystem source code the implementation of the Linux usb filesystem to which additions are made is split between two main source files known as inode.c and devio.c. The devio.c file contains all the filesystem functions that provide the functionality for the device interface files in the bus directories. The inode.c file contains the functions to define these files and build them up in a filesystem and register that filesystem. So for example when a new device is attached a function in inode.c is called that defines a new inode structure, adds references to the functions in devio.c, and adds this inode to the superblock. Then when a directory in the filesystem is read a function in inode.c will return the list of inodes in that directory. If the user space wants the filesystem to open a file it will use the function pointers in the relevant inode structure to call the device file opening functions defined in devio.c. The same method will cause the reading functions in devio.c to be called if the file is read.


            So in order to add the monitoring files to this system a number of things must be done. Changes must be made to inode.c to create inodes for the additional monitoring files. These inodes must be added to the superblock in the same way as the device interface files but must have pointers to different file operation functions replacing the existing functions in devio.c.


            These additional file operations functions must handle the opening, closing, reading and writing of the files. In order to have access to the appropriate data there is a field in the inode structure that allows a pointer to the relevant usb_device structure. This is what allows access to the URB data that has been monitored. It is important to remember here that file reads translates to the kernel ‘writing’ information to user space and vice versa in reading.


            The added opening function for the monitoring files moves the monitored URB buffer to a new buffer, so that monitoring can continue to the old buffer while the user reads this temporary one. This temporary buffer is destroyed by the closing function. This means that when a monitoring file is opened and closed the monitored URBs up until the open are lost from kernel space. This is the attribute of the system that prevents multiple user programs monitoring the same device at one time.


4.3.2       Implementing Timing in the kernel


            Implementing timing routines proved more difficult than expected. This was primarily due to incompatibilities between the different versions of timing structures in Linux. Kernel functions do not have access to the standard C library at all but have most of the functions normally provided by the C library provided by very similar Linux calls. However there are sometimes conflicts between these definitions.


            There were several options of what structure the timing data could be stored in. The classical C time_t type could be used and set by the time() function. The disadvantage of this is that the granularity is only in seconds, which is not small enough for many of the possible applications that this timing information might be wanted for.


            Another option is the timeval structure that includes a time_t-like value and a value for the number of microseconds within that second. This is perfectly adequate for the USB monitoring purposes. There is another option that is a result of the POSIX 4 specification and that is called the timespec structure, which is doe to supersede the timeval structure. This is similar to timeval but stores nanoseconds instead of microseconds. Unfortunately timespec support is incomplete in Linux and although some of the support functions are available it is not considered ready to be used.


            So the logical decision to use timeval structures was made and implementation attempted. Obtaining and storing the timeval structure proved no problem. However, it proved impossible to use the normal functions available in C to decompose the time_t part of the timeval structure due to the fact that in the Linux kernel include files there are two possible time.h files that can be included. One file that provides the timeval structures and the time-obtaining functions and one that provide the original time_t structures and the functions to decompose this time_t functions. These two files cannot be included together due to subtle incompatibilities. This is only the case for kernel functions.


            After attempting many possible solutions the only workable possibility was to implicitly code the decomposition of the time_t type rather than rely on the normally available functions. This is not too complex as date information is not really relevant so can be disregarded. The time_t type is merely a value giving the number of seconds since January 1st 1970.






4.3.3       Problems with Interrupt Handlers


For a period of time during the development and testing of the kernel patch the development system exhibited occasional but very fatal kernel crashes (the whole system suddenly froze). This was very intermittent but usually during Usb activity and almost certainly connected to the kernel patch. Searching for a bug that can cause this kind of error is not easy and after some attempts there appeared no easy way of determining the cause. At a later stage of development when full data monitoring was finally implemented in full the system exhibited exactly the same crash symptoms but this time in an entirely repeatable way. The system crashed when attempting to allocate some memory.


After some research the cause was discovered. When in the kernel, memory is allocated using kmalloc() instead of the normal malloc() function. kmalloc() takes an additional parameter which is a flag specifying the priority. GFP_KERNEL had always been used as the second value, which specified the correct priority for normal kernel service. However, the URB intercepting pre_completion function is run from within an interrupt handler and as such is not allowed to sleep under any circumstances. In order to stop kmalloc from sleeping under any circumstances it must be called with GFP_ATOMIC. This means that kmalloc will occasionally fail to allocate memory but will not sleep and therefore will not cause a systemic crash. Changing all the kmallocs in the interrupt handler fixed the intermittent crashes. Only very occasionally does a kmalloc fail in which case it is reported to the system log.


4.3.4       Changes to different Host Controller Drivers


            The only alterations to the host controller drivers that is required is the addition of calls to the pre-completion function just prior to all calls to URB completion functions. The different Host Controller Drivers have different internal structure so the number and location of these calls varies widely. The ohci host controller driver centralises its completion call in one function but the usb-uhci driver makes USB completion calls from 11 different locations in the source code. This means that there is literally a one line addition required to the latest ohci host controller driver, whereas it about 15 for the usb-uhci driver. The smaller the number of changes the more likely the patch is to work on future versions of these drivers.


4.3.5       Where to put the code


            For a lot of the changes to the kernel code there is no option on where to put the code as it is made up of small modifications to existing functions, however when introducing new functions into the kernel there is a choice between adding to an existing file or adding in a new source file to package up all the functions to do with the patch. This second method would make it easier to make the patch a fully-fledged kernel configuration option, however this would mean changing various Makefiles as well as creating the new files.


            Primarily due to the required changes to Makefiles and configuration files it was chosen not to create a new file for the new monitoring files. Instead the new functions were placed at the bottom of the devio.c file as almost all are related to the extensions to the Linux USB filesystem functionality, which is implemented in the devio.c file.



4.3.6       Compatibility on different kernel versions


            The kernel modifications were developed using a stock Linux kernel version 2.4.1[12]. The vast majority of changes that are introduced in successive versions of the kernel are fixes to old device drivers, new drivers, or fixes to the Host Controller Drivers.  This means that as most of the modifications for this application are to the USB core and in particular the usbdevfs filesystem there should be little conflict with later versions of the Linux kernel. The modifications made to the kernel for this system are mostly additions and specifically adding in new functions, this should reduce further the likelihood of conflict.


When releasing a working example of this system the patch was applied to a 2.4.4 kernel and found to fail in two areas. These two areas were very quickly fixed and were a result of design changes within one of the Host Controller Drivers (usb-ohci). This highlights the disadvantages of the method chosen for intercepting URBs (as discussed in section 4.2.1 on page 30). Whenever the code in a host controller is changed around the calling of the URB completion callback the kernel patch may no longer apply. The Linux Patch utility can cope with some degree of small change but if the actual design changes then this requires manual intervention.




5         System Design – The Monitoring Application


            Monitoring applications can have a multitude of functionality that helps the user deal with the data that is monitored. This is ancillary to the core purpose of this application. The main aim of this monitoring application is to fully demonstrate the functionality of the kernel patch and provide a useful and usable interface to it.


A simple command-line interface is too simplistic and inadequate at displaying effectively the monitored data. A graphical user interface provides a much greater ability to show the complexity and flexible nature of the monitoring abilities of the kernel patch. The production of advanced applications using complex GUIs is much simpler now than it has been in the past and there are many libraries that make the implementation of ‘standard’ looking GUIs a minimal extra effort on top of the main application functionality.


5.1        Functional Requirements of a Monitoring Application


1.      The Monitoring Application should be able to detect from the existing Linux USB subsystem the topology and configuration of all the attached busses.


2.      The Monitoring Application should be capable of setting the monitoring level in the modified kernel of different endpoints independently.


3.      The Monitoring Application should be able to read the new monitoring files in the /proc filesystem and parse them in a way such that it can store data about monitored URBs.


4.      The Monitoring Application should be able to display the details of URBs on different endpoints on different devices.



5.2        Attributes desirable in the Monitoring Application


1.      Running the Application should not heavily impact General System Performance.


2.      The Application should be relatively simple to use and data should be easily obtained by anyone experienced in the concepts of USBs.


5.3        Design of the Monitoring Application

5.3.1       Structure of the USB Model


            The Monitoring Application must maintain a model of the USB connected to the Host system that the Application is running on. This model must be flexible enough to hold any possible state that a USB can be in. It must also be able to hold all available configuration information about all devices attached to the bus including details about the configurations, interfaces, and endpoints.

Figure 9: Simple Object Model representing a USB configuration

Using Object Orientated Methodology this model contains various classes of object such as Bus, Device, Configuration, Endpoint, and Interface. These classes are interconnected by a form of ownership. For Example a Configuration has one or more Interfaces and each interface has one or more endpoints associated with it. This leads easily to a simple Object Model of the whole bus. In Figure 9 a simple object model is shown for a bus, obviously more than one of these can be held for a system. The details of the implemented USB model objects are contained in Appendix D.


            This Object Model provides the perfect location to store the monitored data that is received from the kernel via the monitoring files in the /proc filesystem. This is best kept in the Endpoint objects as this is the level at which monitoring can be switched on or off. Note that this is different from the kernel patch where URBs are buffered in device-level buffers as opposed to endpoint-level buffers. This is not a problem but simply highlights different priorities in different parts of the system. The monitored data can be kept in a new object called URB. This will contain all the attributes that can be monitored about a URB as well as a time stamp that is attached by the kernel patch at the time of monitoring. Optionally the URB object can also contain the data that the URB was transferring.


5.3.2       Obtaining the Topology and Configuration Information


All the topological and configuration information is obtained from the ‘devices’ file in the Linux usb filesystem, as referred to in section 2.11. Its structure is exhaustively defined in [15]. The file gives seven different types of line each describing different sets of characteristics that devices, endpoints, configurations, and interfaces can have. This contains all the characteristics from the device descriptors available to the Linux USB subsystem. This file can be parsed to give the structure and contents of the object model. As the object is parsed the object model can be built up and the fields filled in.


5.3.3       Polling for the Monitored Data


            In order to keep receiving monitored URBs as they are monitored in the kernel the monitoring application must poll the /proc monitoring files at regular intervals. This is best achieved in a multi-threaded architecture where a separate thread is started for each monitored device. These threads can simply read the contents of the monitoring file (thus emptying the kernel buffer), construct a URB object for every URB in the monitoring file and place that URB in the list in the relevant endpoint structure.


            This design does call for a multithreaded architecture accessing a unified set of data structures. This would imply that there are thread safety contention issues, however because the ‘reader’ threads are only updating a particular set of structures that no other reader thread can be updating and the display threads will only ever read values from these structures for display there is no danger of thread contention problems.

5.3.4       Displaying Configuration and Monitored Data


            In deciding how to best present the data it is important to remember the different information that people may be interested in finding out. One of the primary concerns is making it simple to navigate around a bus and select devices of interest. It can be assumed that target users would want to focus on the details of one device at a time although retaining the ability to monitor multiple devices simultaneously.


            It is clear that USBview (see page 22) has an efficient way of navigating around a USB bus. A tree is a logical mechanism to navigate around a bus and select a device of interest and USBview shows this. In USBview another pane contains all information about a selected device, however here USBview’s solution of having one pane for all information is inadequate for this system. The potential amount of information that will be stored in the USB model about a device could be huge and far too big to display all in one pane. The solution is to break this information down into different areas and make the second pane selectable as to which information is shown.


            The different types of data that a user of the monitoring application will be able to access includes:


·        Basic configuration information from the device descriptors (as is given in USBview).


·        The Monitoring Level settings of the different endpoints in each device.


·        The monitored URBs for each of the endpoints showing the characteristics of each of the URBs and the ability to show their data (if present).


This division lends itself to a second pane having three possible states each serving the three possible uses described above. The three states should be easily switchable between but need not be displayed simultaneously.


Given the way USB devices work, using different endpoints for different parts of one task, it is important to be able to see easily the characteristics of monitored URBs on different endpoints. To this end the URB characteristics can be placed in tables with scrollbars that allow the viewing of a small number of URBs for one endpoint at the same time as a small number of URBs on other endpoints.


On monitored URBs with data attached the data need not be shown all the time. Often there will be a lot of it and most of the time may not be essential to the detail of monitoring. USB monitored data can represent many different things. And as such it would be useful to be able to display the data using different


5.4        Monitoring Application Implementation Issues

5.4.1       Java and Swing vs. C and Gtk


            The implementation language is significant primarily because of the library available. The Gtk library is a C based library that is widely used in Linux GUIs. The Swing Library is an integral part of the Java language that allows extremely flexible and intelligent presentation in a standardised cross-platform style.


            The decision as to which language to implement the monitoring application in is largely irrelevant. Both language/library combinations are more than adequate for the proposed functionality. In the end the decision was made to use Java and Swing primarily to use the stronger object-orientated features of Java. This proved very useful in implementing the USB object model in the Swing GUI.


5.4.2       The USB Object Model


            As described on page 42 the model of the USB can easily be based on a collection of five different classes of object: Bus, Device, Configuration, Interface, and Endpoint. The exact details of these objects are shown in Appendix D.


            Many of the objects need to keep variable sized lists of objects. For example a device needs to keep a list of child devices attached to it and a list of possible configuration objects. In these cases variable sized lists have been implemented as Vectors.


5.4.3       The Threaded Monitoring File Reader


            Interfacing with the kernel via the monitoring files in the /proc filesystem the Monitoring application must update its usb model as and when it receives new monitoring information. In order to achieve this the Monitoring Application must regularly poll all the relevant monitoring files in the /proc filesystem. Java provides an easy method to set up a thread that will repeat an action repeatedly after a specified time interval. This is via the Timer and TimerTask Classes. The Timer class sets up a timer that will run a method in a TimerTask Object at a set frequency. In setting the frequency of polling it is desirable to have it often enough so that the monitoring appears almost instantaneous while not so fast as to impose too high a workload on the kernel functions that dump the buffers down to the monitoring application.


            The actions inside this TimerTask method are simply to:


1.      Open the relevant monitoring file.


2.      Read all the contents into a byte array.


3.      Parse the contents of the byte array and construct URB objects to represent the contents.


4.      Place those URB objects in the relevant Endpoint object.


5.      Close the file.



The parsing element of this process is slightly complicated by the way java treats text. Since its inception Java has treated all characters as being 16-bit Unicode entities. It has different objects for dealing with character streams (which are converted into Unicode) and byte streams (which are left as unsigned bytes). A problem arises with the use in this system of one monitoring file to deliver a stream containing plain ASCII text descriptions of the URB headers and binary data representing the data content of the URB in the same file. Here the fixed length nature of the URB header in the monitoring file becomes important. Different methods can be used for the header part and the data part. Once the header part has been read, the presence and size of the data part can be determined. Then the data part can be read using an alternative data method that reads the data as bytes instead of Unicode characters.


5.4.4       Displaying the Device Tree


            One of the most useful features of the Java/Swing combination is the ability to create standardised user features like tables and trees from different object types as long as they implement an interface by including a few specific methods. This means that the USB model objects can be used as the models for the display structures.


            In order to achieve a tree of devices that can be navigated to select devices of interest the Device class can be made to implement the TreeModel interface. This requires the addition of eight methods that allow the implementation of the tree such as get_child(), get_child_count(), is_root(), etc…


            Using this method the collection of Device objects in the USB model implements the tree model. A navigable tree can easily then be created using the JTree class. The navigable tree is visible in Figure 10.



5.4.5       Splitting Up the Information


            The information displayed about a device selected on the tree has to be selected from three options. The best solution for this is to use a feature of Swing called Tabbed Panes. This allows the display to be quickly changed between ‘screens’.


5.4.6       Displaying the URB Tables


The same method that was used to implement the trees can be used to implement tables. A class can be made to be an extension of the AbstractTableModel class. This involves adding six new methods such as getRowCount(),  getColumnName(), getValue(), and setValue(). This turns the class into a table model and a simple table can easily be displayed using the JTable class.


            There are two locations where tables are useful. In the Configurations pane different interfaces need to show all the details of their endpoints and the set monitoring level.       


The first case can be achieved by making changes to the Interface class. Making a table from the interface class gives a table of endpoints with their characteristics. The monitoring level of the endpoint needs to be editable. There are only three possible values. To avoid erroneous input an editor called a ComboBox can be attached to this column that allows the selection from a drop-down menu. Figure 10 shows the configuration pane showing the endpoints of a USB storage device (in fact a floppy drive).

Figure 10: The Configuration pane showing the endpoints of a USB storage device


The second case is where the details of URBs monitored on a URB need to be displayed. In this case each table model can be represented by an Endpoint class. Similar extensions to those made to the Interface class can be made to the Endpoint class to allow the display of the URBs’ details.


Figure 11: The Endpoints pane showing URBs monitored on a USB storage device


Figure 12: URB data window showing the buffer in hex view

            The data element of the URB need not be shown in the table. It is much better if this is shown in an independent window as then it can be compared side to side with other URB data buffers. To achieve this another ComboBox can be used to provide a drop-down menu on URB rows that have an associated data buffer. The data can then be displayed in a new window. The data that is passed in URBs can have very different meanings and so different types of encoding can be used to view the data. A tabbed pane is used to allow access to Hexadecimal, Binary, and Textual views of the data.




6         System Evaluation


6.1        Suitability of Core Design


            On purely functional and performance grounds a system that unobtrusively intercepts the USB traffic on the cable outside the host subsystem and passed the monitored information to a separate monitoring computer provides the most unobtrusive and accurate monitoring system. This is why large companies succeed in selling such systems at very high cost. However this system provides a solution to many problems as effectively as an expensive hardware solution.


            Types of problem that can be detected and analysed using this system that it was not possible to do before without hardware monitoring:


1.      Analyse the data sent to and from a closed source driver.


2.      Debug the data sent to and from an open source driver without changing the code.


3.      Clearly show the relative activity of different devices on a bus.


4.      Determine the time taken between a driver receiving a URB and sending a reply URB.


5.      Determine whether errors in the data transmissions are occurring (as drivers may not report them).



Areas that this system cannot observe and is not able to aid:


1.      Analysing bugs inside the Host Controller hardware.


2.      Analysing bugs inside the Host Controller Driver software.


3.      Determining the accuracy of power distribution across the bus.


4.      Obtaining the timings of sub-parts of data-transfers such as the time difference between a control request and the acknowledgment.


5.      Debug erroneous transfers from devices that are not of the correct format.






Prior to this project Linux users were unable to carry out any real monitoring of USBs without specialist hardware. USBview (as described on page 22) only provides configuration information and only really displays the information that is already available in the devices file in the Linux USB filesystem. This project does provide a new capability to USB on Linux but it does not replace the need for hardware monitoring systems completely. Experts who are designing complex devices and fine-tuning first time device drivers will doubtless want to be certain about the exact signals that are being transmitted. However there is a group of part-time developers who are writing device drivers for niche devices that have been implemented on other operating systems. For this class of device driver developer this system is a very cost-effective solution to debugging the operation of devices and device drivers.


6.2        Effectiveness of Implementation


            Writing code for the kernel is often portrayed as a kind of black art where the traditional rules of software do not wholly apply, a bit like the quantum mechanics of programming. This is less true of Linux due to the number of people involved in its development and the amount of documentation available, however there are still some major complexities that must be taken into account.


            The kernel patch has proved largely stable. Although not thoroughly or rigorously tested for memory leaks and other possible low-visibility defects, the patch has proved stable being run as the primary kernel on the development system throughout the progression of the project. This has shown both that the (patched) kernel is stable for prolonged periods of time (over a week between re-boots) in that it shows no outward sign of deterioration in either normal use or USB performance.


            After the kmalloc() priority changes (see section 4.3.3) there were no more unexplained kernel failures.


            There are some areas where the monitoring system is less stable than might be desired. It only has a partial solution to the issue of hotplugging, and can give unwanted error messages when devices are disconnected.


The way the monitoring application handles alternate interfaces is not ideal. The concept of alternate interfaces was largely forgotten about during the design as none of the devices that were experimented on had them. The problem is that each alternate device’s endpoints are given their own endpoint object and assumed to be active (which is not always true). This does not prevent monitoring but is certainly not ideal as the display becomes very cluttered by endpoints that are not relevant to the user.



6.3        Impact on the Linux USB Subsystem of the kernel patch


            One of the key attributes desired in the system was that is did not impact on normal USB operation. Verification of this was done in two parts:


1.      Impact on Functionality - The Impact on performing normal USB operations and looking for any loss of functionality in USB using operations.


2.      The Timing Impact of Monitoring - Obtaining timing information for the added processing done to non-monitored URBs.




6.3.1       Impact of Functionality


Testing functionality was achieved by carrying out a variety of normal tasks that used a range of transfer types and amounts of bandwidth. Items that were tested included:


·        A USB mouse (high frequency of interrupt transfers). This device was as a standard mouse input to X Windows environment.


·        A USB web cam streaming video image (primarily using isochronous transfer). A video stream was set up and displayed on screen using the Video4Linux API.


·        A USB floppy drive using the usb-storage driver (uses bulk, control, and occasional interrupt transfers). File transfers were carried out on a variety of file sizes.



The results were that no difference in behaviour was observed under any of the specified operations.


6.3.2       The impact of Monitoring on processing time


            The principal addition in the kernel patch that affects all URB transfers regardless of what the monitoring level is the pre-completion function that is run on every URB prior to the driver being informed of completion. The amount of processing that is carried out in this function is a measure of the cost of monitoring.


            Analysis of the code in this function can give factors that affect the amount of processing that will occur. The source code to the pre-completion functions is in Appendix C. The factors that can be deduced from the structure of the code are:


·        For devices on which none of the endpoints is monitored in any way there is very little processing that will occur. This is because no comparisons are necessary to check whether the URB needs to be monitored.


·        The amount of processing for all transfers on a device with monitored endpoints will increase with the number of endpoints that are monitored. This is because even for transfers on endpoints that are not monitored the function will have to check the endpoint value against more ‘monitored endpoint’ values to determine whether or not this endpoint is on the list to be monitored.



·        Transfers on monitored endpoints will require more processing than transfers on endpoints that are not monitored. This is because allocating the memory and copying the details of the URB takes processing time.


·        Transfers on endpoints with ‘full data monitoring’ will require more processing than transfers on endpoints set to ‘URB headers only’. This is because of the extra memory that must be allocated and the extra copying.



            To obtain the timing information, a modification can be made to the pre-compilation function to get the length of time that the system takes to carry out the processing in that function. In a normal user-space C program this could easily be done using the clock() call and clock_t variables, however the problem that was discovered when implementing timing in the monitoring system (see section 4.3.2 on page 37) resurfaces. The clock() function gives the number of clock cycles that have occurred. This value looks back to zero every few minutes on a fast 32-bit machine but is useful for very accurate timing. Unfortunately the timing functions that are available within the Linux kernel do not include the clock() function. It seems odd that user space can access a service from the kernel that code in the kernel cannot easily access itself, but it was not possible to find a way to do this. There is no evidence in the kernel itself or in any kernel documentation of a similar function.


   An alternative to the clock() is to use the timing functions used to give a monitoring time accurate to 1 microsecond. On a fast PC several hundred machine instructions will occur every microsecond but this is just enough to determine the delay caused by the processing in the pre-completion function. The absolute time (in microseconds) can be got at the start of the function and again at the end. The difference can then be sent to the system log. This does not include the additional cost of calling the pre-completion function and returning, this is assumed to be irrelevantly small. The timing results give the approximate amount of time taken to perform the contents of the pre-completion function. It is not completely accurate because the start and finish times are not necessarily on microsecond boundaries so there could be an error of up to 2 microseconds. Taking that into account and after experimenting with this timing mechanism it was decided to average the results over several transfers.


            The testing that was timed consisted of file transfers to and from a USB storage device. USB storage devices use the Primary Control pie as well as  two bulk pipes (one in and one out), and an incoming Interrupt pipe. File transfers will cause activity on all these pipes. The device was re-mounted between every test in order to clear any file caches. The first run was also repeated to counter the possible effect of memory caches in the system.


            Two sets of tests were performed on a relatively high specification machine (650 MHz Pentium III with 128Mb RAM):


Set 1: URB Only Monitoring – All endpoints were initially set to ‘No Monitoring’ with the number of endpoints set to ‘URB only monitoring’ incremented after each timed run. After each timed run the pre-compilation time for 10 transfers on monitored endpoints and 10 on unmonitored endpoints were averaged to a figure.


            Set 2: Full Data Monitoring – All endpoints were initially set to ‘No Monitoring’ with the number of endpoints set to ‘Full Data Monitoring’ incremented after each timed run. After each timed run the pre-compilation time for 10 transfers on monitored endpoints and 10 on unmonitored endpoints were averaged to a figure. Here the timing of monitored data points refers only to 512 byte bulk transfers. The usb-storage driver will only shift data in blocks of 512 bytes so this is a good standardisation to eliminate variance depending on the size of the data that is copied.























Set 1: URB Headers only:


Number of Monitored Endpoints

Average time taken on unmonitored endpoints / microseconds

Average time taken on monitored endpoints / microseconds

















Figure 13: Table and Graph showing the results of timing analysis of the pre-completion function.



            The results of this first set of tests are largely as expected with one slight anomaly. The fact that time taken by monitored transfers is greater than unmonitored transfers is as expected, as is the fact that the time taken by unmonitored endpoints appears to marginally increase with the number of endpoints. The anomaly is that the results indicate that as the number of monitored endpoints is increased the length of time taken inside monitored endpoints appears to decrease. There is no obvious explanation for this. The important conclusions that can be made from these results are that the number of monitored endpoints (up to at least four of a possible sixteen) does not have significant impact on the USB performance of a device and that even monitored transfers are only impacted by about two microseconds per transfer.




Set 2: Full Data Monitoring:


Number of Monitored Endpoints

Average time taken on unmonitored endpoints / microseconds

Average time taken on monitored endpoints / microseconds


















            The results of the second set of tests shows that Full Data monitoring is much more expensive in processing time that simple URB header only monitoring. Full Data monitoring on a bulk transfer of 512 bytes adds about 5 microseconds to the amount of time taken to process a URB transfer.


            The significance of this timing data on potential users can only really be guessed at. USB allows interrupt transfers to happen every millisecond and it is conceivable that when monitoring such an endpoint the difference between a five microsecond penalty as opposed to a two microsecond penalty might be important. Unfortunately it is very difficult (and beyond the scope of this project) to put these figures in context by benchmarking the other elements of the Linux-USB subsystem that are essential to its operation.





6.4        Performance of the Monitoring Application


The monitoring Application’s performance is much less important than the performance of key parts of the kernel patch as the monitoring application runs at a much lower priority in user space and can be interrupted at any time. It is still however relevant as the monitoring application must be run I order for the monitoring that the kernel performs is retrieved and displayed. The performance of the Monitoring Application is heavily dependent on the java virtual machine that is being run on the host system. On a sample system (650MHz Pentium III with 128Mb SDRAM running IBM JVM for Linux 1.3) the Monitoring Application never used more than 10 per cent of available CPU utilisation and mostly much less. This is perfectly acceptable.


6.5        User Adoption


            At two points during this project code was released for public consumption and a notice was made on the linux-usb-developers and linux-usb-users mailing lists. The code was released as under the name USBMon. These releases were named versions 0.1 and 0.2 respectively. Version 0.1 only contained a kernel patch that did not support full data monitoring. Simple command line tools were supplied with version 0.1 to control and view the monitoring.


            The USBMon system (as well as this document) can be downloaded from http://www.dcs.ed.ac.uk/~dxh/public/USB/. Over thirty-five downloads have been recorded since version 0.2 was released.


6.6        Future Development


Possible future development of this system could take many possible routes depending on the usage of the system and implementation decisions made in the main Linux USB tree. The most obvious next step is to develop the kernel patch into a configuration option in the Kernel. This could even prepare the patch for possible submission into the Linux kernel. Before any submission to the Linux kernel the absolute stability under a much wider range of systems would be necessary. A lot more reviewing of kernel code by experienced kernel developers would also be desirable.


The amount of monitored data values monitored by the kernel could also be increased. The interface to user-space could be made more flexibly defined so as to allow greater selectivity of monitored events by the monitoring application, for example only monitoring URBs that returned with error codes.


     A possible addition to the kernel patch is the provision of generalised statistics for usage and activity over the whole USB. This could take the form of a file in the /proc filesystem that gave a few global statistics much in the same way as CPU usage is reported. This could then be used by simple desktop applets to signal activity on the USB bus.


     On the monitoring application side there are many possible data analysis mechanisms that could be implemented such as graphs and charts as well as the calculation of generalised statistics on a device basis.



[1]           USB Implementers Forum. USB Home Page. http://www.usb.org


[2]           Microsoft, Intel, NEC, Compaq. USB Specification 1.1, 23rd September 1998. http://www.usb.org/developers/data/usbspec.zip


[3]           Microsoft, Intel, NEC, Compaq. USB Specification 2.0. 27th April 2000.  http://www.usb.org/developers/data/usb_20.zip


[4]           USB Device Class Working Group. Approved Device Class Specifications. http://www.usb.org/developers/devclass_docs.html


[5]           Compaq. OHCI host controller Specification. http://www.compaq.com/productinfo/development/openhci.html


[6]           Intel. UHCI Design Guide. http://developer.intel.com/design/USB/UHCI11D.htm


[7]           Microsoft, Intel, NEC, Compaq. Guide to USB 2.0. http://www.usb.org/developers/data/usb_20g.pdf


[8]           Java USB expert group, Java Specification Request: USB API, 14th May 2001. http://java.sun.com/aboutJava/communityprocess/jsr/jsr_080_usb.html


[9]           John Garney, Intel Architecture Labs. An analysis of throughput. Characteristics of Universal Serial Busses. http://www.usb.org/developers/data/whitepapers/bwpaper2.pdf


[10]      USB Workshop. http://www.usbworkshop.com


[11]      Linux Documentation Project. http://www.linuxdoc.org


[12]      Linux Kernel Download page. http://www.kernel.org


[13]      Linus Torvalds summarised by Zack Brown, Linus on devfs, 24th April 2000.



[14]      Linux USB Home Page. http://www.linux-usb.org


[15]      Linux USB Guide. http://www.linux-usb.org/USB-guide/book1.html


[16]      Linux USB SourceForge Page. http://sourceforge.net/projects/linux-usb


[17]      Linux USB Developers mailing list archive. http://www.geocrawler.com/lists/3/SourceForge/2571/0/


[18]      Linux USB Users mailing list archive. http://www.geocrawler.com/lists/3/SourceForge/4563/0/


[19]      Roland and Tom. USBSnoopy Web Page. http://www.jps.net/~koma/


[20]      USB robot Homepage. http://usb-robot.sourceforge.net/


[21]      Greg Kroah-Hartman. USBView Web Site. http://www.kroah.com/linux-usb/


[22]      Linux Hotplugging Homepage. http://linux-hotplug.sourceforge.net/


[23]      David Brownell. jUSB Web Site. http://jusb.sourceforge.net/


[24]      Video4Linux Web Site. http://roadrunner.swansea.linux.org.uk/v4l.shtml


[25]      Linux USB supported devices page. http://www.linux-usb.org/devices.html


[26]      Sun. Sun Java Homepage. http://www.java.sun.com


[27]      Microsoft. USB Support on Windows 98 vs. Windows 95, 3rd January 2001. http://www.microsoft.com/hwdev/busbios/usbwin98.htm


[28]      Apple Corp, Apple Usb systems. http://www.apple.com/usb/


[29]      S Hughes and DJ Thorne, British Telecom. Broadband in home Networks http://www.bt.com/bttj/vol16no4/06.pdf


[30]      Mark A Kelner, Government Computer News. USB: This Bus is going places, July 19 1999 http://www.gcn.com/shopper/vol18_no22/peripherals/278-1.html


[31]      Alan Zisman, Canada Computer Paper Inc. USB is (finally) a contender. June 1999. http://www.ccwmag.com/articles/99-06/9906/TECHTALK/TECHTALK/TECHTALK.html


[32]      Steven Brody, CNN. Linux projected to outpace all contenders through 2003, 22nd April 1999. http://www.cnn.com/TECH/computing/9904/02/linuxgrow.ent.idg/


[33]      Kelvin Taylor, PC Magazine.  Intel announces royalty-free USB 2.0, March 2001. http://www.zdnet.co.uk/pcmag/trends/2001/03/07.html


[34]      CATC – Computer Access Technology Corporation USB products. http://www.catc.com/products/d_usb.htm




Appendix A     - The URB Structure



This is the modified URB structure as found in include/linux/usb.h in the Linux source code. The only modification is the addition of the final timestamp.


typedef struct urb


       spinlock_t lock;           // lock for the URB

       void *hcpriv;              // private data for host controller

       struct list_head urb_list; // list pointer to all active urbs

       struct urb *next;          // pointer to next URB    

       struct usb_device *dev;           // pointer to associated USB device

       unsigned int pipe;         // pipe information

       int status;                // returned status

       unsigned int transfer_flags;      // USB_DISABLE_SPD | USB_ISO_ASAP | etc.

       void *transfer_buffer;            // associated data buffer

       int transfer_buffer_length; // data buffer length

       int actual_length;              // actual data buffer length 

       int bandwidth;                  // bandwidth for this transfer request (INT or ISO)

       unsigned char *setup_packet;      // setup packet (control only)


       int start_frame;           // start frame (iso/irq only)

       int number_of_packets;            // number of packets in this request (iso)

       int interval;                   // polling interval (irq only)

       int error_count;           // number of errors in this transfer (iso only)

       int timeout;               // timeout (in jiffies)


       void *context;                    // context for completion routine

       usb_complete_t complete;   // pointer to completion routine


       iso_packet_descriptor_t iso_frame_desc[0];


       struct timeval completion_time; // for USBMon timestamping

} urb_t, *purb_t;

Appendix B     - The pipe structure


This is a comment taken from include/linux/usb.h in the Linux source code version 2.4.4. It describes the pipe structure.



 * Calling this entity a "pipe" is glorifying it. A USB pipe

 * is something embarrassingly simple: it basically consists

 * of the following information:

 *  - device number (7 bits)

 *  - endpoint number (4 bits)

 *  - current Data0/1 state (1 bit)

 *  - direction (1 bit)

 *  - speed (1 bit)

 *  - max packet size (2 bits: 8, 16, 32 or 64) [Historical; now gone.]

 *  - pipe type (2 bits: control, interrupt, bulk, isochronous)


 * That's 18 bits. Really. Nothing more. And the USB people have

 * documented these eighteen bits as some kind of glorious

 * virtual data structure.


 * Let's not fall in that trap. We'll just encode it as a simple

 * unsigned int. The encoding is:


 *  - max size:            bits 0-1      (00 = 8, 01 = 16, 10 = 32, 11 = 64) [Historical; now gone.]

 *  - direction:     bit 7         (0 = Host-to-Device [Out], 1 = Device-to-Host [In])

 *  - device:        bits 8-14

 *  - endpoint:            bits 15-18

 *  - Data0/1:             bit 19

 *  - speed:         bit 26        (0 = Full, 1 = Low Speed)

 *  - pipe type:     bits 30-31    (00 = isochronous, 01 = interrupt, 10 = control, 11 = bulk)


 * Why? Because it's arbitrary, and whatever encoding we select is really

 * up to us. This one happens to share a lot of bit positions with the UHCI

 * specification, so that much of the uhci driver can just mask the bits

 * appropriately.


Appendix C     - The Pre-Completion Functions


Below is the source code of the pre-completion function that is part of the kernel patch. On the next page is the function that copies the URB and is called from the pre_completion function.




void USBMon_urb_pre_completion(urb_t *purb)


       urb_t *pcpy_urb;

       struct usbmon_endpoint_ll *ep;


       ep = (struct usbmon_endpoint_ll *)




       while(ep != NULL){


              /* test this ep against list */

              if((purb->pipe & ((0x4f << 8) | (0xf << 15))) == ep->value){


                     /* We are now monitoring this ep*/                    

                     pcpy_urb = USBMon_cpy_urb(purb, ep->level);

                     if(pcpy_urb != NULL){

                           /* copying did not fail -> add to linked list */



                           if(purb->dev->USBMon_data->USBMon_urb_ll_head ==




                                                USBMon_urb_ll_head =



                                                USBMon_urb_ll_last =





                                              USBMon_urb_ll_last->next =









                           printk("[USBMon] failed to copy URB for monitoring\n");



                     ep = NULL; /* So as to break out of while loop faster*/


              }else{ /* endif check on endpoint */


                     ep = ep->next;















purb_t USBMon_cpy_urb(urb_t *orig, int level)


       urb_t *cpy;


       cpy = kmalloc(sizeof(*cpy), GFP_ATOMIC);


              printk("[USBMon] Unable to kmalloc URB copy\n");

              return NULL;



       cpy->dev = orig->dev;

       cpy->pipe = orig->pipe;

       cpy->status = orig->status;

       cpy->transfer_flags = orig->transfer_flags;

       cpy->transfer_buffer_length = orig->transfer_buffer_length;

       cpy->actual_length = orig->actual_length;

       cpy->bandwidth = orig->bandwidth;

       cpy->setup_packet = orig->setup_packet;

       cpy->start_frame = orig->start_frame;

       cpy->number_of_packets = orig->number_of_packets;

       cpy->interval = orig->interval;

       cpy->error_count = orig->error_count;





              cpy->transfer_buffer = (void *)  




                     printk("[USBMon] Unable to kmalloc URB buffer\n");

                     cpy->transfer_buffer = NULL;


                     memcpy( cpy->transfer_buffer,





              cpy->transfer_buffer = NULL;





Appendix D     - The USB Model Java Classes


Class Bus


public class Bus

extends java.lang.Object

Class used to store information about a Bus.

Field Summary


          The amount of bandwidth allocated to this Bus


          The total amount of bandwidth possible on this Bus


          The number of interrupt requests


          The number of isochronous requests


          The number of the Bus.


          The root hub device.


Constructor Summary

Bus(int bus_no)



Method Summary


search(int num)
          Search for a particular device and return it.


toString(int indent)
          This returns a textual representation of the Device and its children.


Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait


Class Device


All Implemented Interfaces:


public class Device

extends java.lang.Object

implements javax.swing.tree.TreeModel

Class used to store information about device.

Field Summary


          The number bus that this device is on.


          The list of configurations.


          The list of devices connected to this one.


          The Class number.


          The Class as a string.


          This holds the Configurations Pane that is plays the tables of interfaces and Endpoints.


          The Primary Control Pipe that all USB devices must have.


          This holds the Details Pane that holds the lists of URBs by endpoints.


          The Driver that is associated with this device (or 'none').


          The level within the bus.




          This is the mfilereader for this device.


          The size of packets from endpoint 0.


          The Device Number.


          The number of different configurations.


          Number of endpointd (** INTERNAL **)


          The number of ports (0 for hub).


          The device number of the parent


          The port number of the parent that this device is connected to.


          The Manufaturer's device number.


          The Manufaturer's device name.


          The Protocol.


          The revision number of the hardware.


          The revision number of the hardware.


          The root_hub device


          The Manufaturer's serial number.


          The Speed of the device in Mbps.


          The Sub-class number.


          This holds the tabbed pane for this device.


          The list of URBs monitored on this device


          The Manufacturer number.


          The Manufacturer as a string.


          The version of USB that this device claims to support (before point).


          The version of USB that this device claims to support (after point).


Constructor Summary




Method Summary


addTreeModelListener(javax.swing.event.TreeModelListener l)
          addTreeModelListener not fully implemented.


          This returns a full textual representation of the Device


getChild(java.lang.Object parent, int index)
          This returns the child of the particular.


getChildCount(java.lang.Object parent)
          This method gets the number of children for a parent.


getIndexOfChild(java.lang.Object parent, java.lang.Object child)
          This method returns the index of the child in parent's children.


          This returns the Bus object on which this device sits.


isLeaf(java.lang.Object dev)
          This method returns whether the object is a leaf or not.


          This makes the configs pane.


          This makes the details pane.


          This makes the details pane.


removeTreeModelListener(javax.swing.event.TreeModelListener l)
          removeTreeModelListener not fully implemented.


search(int num)
          The search method searches for devices with a particular device number.


          This returns a brief textual representation of the Device


toString(int indent)
          This returns a textual representation of the Device and its children.


valueForPathChanged(javax.swing.tree.TreePath path, java.lang.Object NewValue)
          valueForPathChanged not fully implemented.


Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait


Class Configuration


public class Configuration

extends java.lang.Object

Class used to store information about a configuration of a device

See Also:


Field Summary


          true if this is the active configuration of the device.


          The attributes of this configuration.




          The number of this configuration.


          The number of different interfaces that this configuration has.


Constructor Summary




Method Summary


          bool that is derived from the attr integer.


          bool that is derived from the attr integer.




          bool that is derived from the attr integer.


Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait


Class Interface


All Implemented Interfaces:

java.io.Serializable, javax.swing.table.TableModel

public class Interface

extends javax.swing.table.AbstractTableModel

Class used to store information about an interface (part of a cfg of a device.

See Also:

Device, Configuration, Serialized Form

Field Summary


          The alternate that is being described.


          The alternate's class as an int.


          The alternate's class as a String.


          The alternate's driver as an int.


          The alternate's protocol as an int.


          The alternate's subclass as an int.


          List of the endpoints for this interface.


          The number of endpoints in this interface.


          The interface number.


Fields inherited from class javax.swing.table.AbstractTableModel



Constructor Summary




Method Summary


          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.


getColumnName(int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.


          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.


getValueAt(int row, int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.


isCellEditable(int row, int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.


setValueAt(java.lang.Object value, int row, int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.


Methods inherited from class javax.swing.table.AbstractTableModel

addTableModelListener, findColumn, fireTableCellUpdated, fireTableChanged, fireTableDataChanged, fireTableRowsDeleted, fireTableRowsInserted, fireTableRowsUpdated, fireTableStructureChanged, getColumnClass, getListeners, removeTableModelListener


Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait


Class Endpoint


All Implemented Interfaces:

java.io.Serializable, javax.swing.table.TableModel

public class Endpoint

extends javax.swing.table.AbstractTableModel

Class used to store information about device.

See Also:

Serialized Form

Field Summary


          The address of the endpoint.


          The type of transfer as an int.


          The type of transfer as an String.


          The endpoint is an input.


          The interval in ms between the polling of interrupt endpoints.


          The maximum packet size of this endpoint.


          The level at which this endpoint is currently monitored.


          The endpoint number


          The endpoint is an input.


          The list of URBs monitored on this device


Fields inherited from class javax.swing.table.AbstractTableModel



Constructor Summary

Endpoint(Device d)



Method Summary


          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.


getColumnName(int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Deatils Page for a device.


          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.


getValueAt(int row, int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.


isCellEditable(int row, int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.


set_monitoring_level(int val)


setValueAt(java.lang.Object value, int row, int col)
          In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.


Methods inherited from class javax.swing.table.AbstractTableModel

addTableModelListener, findColumn, fireTableCellUpdated, fireTableChanged, fireTableDataChanged, fireTableRowsDeleted, fireTableRowsInserted, fireTableRowsUpdated, fireTableStructureChanged, getColumnClass, getListeners, removeTableModelListener


Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait


Class URB


public class URB

extends java.lang.Object

Class used to store details of a URB that occurred in a Linux USB subsytem. The exact configuration of this URB object relates to the attributes that this monitoring application needs to maintain. This is not a definition of what a URB contains.

Field Summary


          This is the actual length of the transmitted URB


          Device Configuration - at time of load.


          The Vector containing the data


          True if data is present in this URB


          The Device that this URB was sent/recieved on


          The actual device number as passed in ***M


          The Endpoint that this URB was sent/recieved on


          The actual Endpoint number as passed in ***M


          Error Count - the number of transmission errors.


          the hour that this URB was completed


          the millisecond that this URB was completed


          the minute that this URB was completed


          The Pipe information: This contains the direction and type of pipe information.


          the second that this URB was completed


          True if data is present in this URB


          The size of the transmitted buffer.




          transfer flags


          A JFrame placeholder to store the Frame showing the contents of the URB.


Constructor Summary

URB(java.lang.String str, byte[] rec_data, Bus bus)
          The constructor that creates the URB structures from the line in the ***M file in the USB Monitoring part of the /proc file system.



Method Summary




Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait



[1] Manufacturers of these systems include Intel, Compaq, CATC, and Catalyst Enterprises

[2] A minor update to the 1.0 specification released primarily to clarify issues relating to hardware device design.

[3] This is derived from a maximum of 500mA current from a supply that can vary 4.75V – 5.25V as specified in [2] p.134-139

[4] This situation was not necessarily intended. It is primarily a result of Intel refusing to licence the UHCI design very widely prompting the development of OHCI by competitors. Intel has now agreed to licence the USB 2.0 EHCI design freely [33].

[5] Apple Computers Inc usually refers this to.

[6] Sony Corp uses this.

[7] According to Apple Corp. over 125 peripherals are available at this time, in the year 2000 over 12 million IEE 1394 devices were shipped.

[8] DOS is the Disk Operating System format common on floppy disks. EXT2 is the most common native Linux hard drive filesystem.

[9] USB support was started in the previous 2.2 kernel (starting at 2.2.7). However development soon moved to the 2.3 experimental kernel tree. USB usage on 2.2 kernels was possible via the use of a kernel back-port patch that some distributions used to allow basic USB mouse and Keyboard use. USB usage beyond this was possible but not easy.

[10] This is the validation for the accuracy of the timestamp taken at time of monitoring.

[11] Valid values for return status are detailed in [15].