Tips on using expanded memory dumps to improve 1394 kernel debug
1394 is the de facto connection used in the industry for kernel debugging of processor-based systems. It is supported in Windows, Linux, MacOS, VMware, is by far the easiest to use, and most efficient. It’s the only “legacy free” debug solution and is easily extensible for multiple target systems.
Using IEEE 1394 can be as easy as plugging in another 1394 cable, and it allows debugging of as many as 62 target systems. There are numerous “how to do kernel debugging with 1394” articles and resources on the web; several are listed at the end of this article.
But with the explosion of installed memory per system values from 64 megabytes to 64 gigabytes, 1394 now makes it possible for system developers to analyze increasingly large memory dump files easily through kernel debuggers. FireWire (1394) has kept pace with these ever increasing memory systems while USB and RS-232 have fallen behind.
Kernel Debug Primer
Kernel debuggers enable a remote PC to monitor/interrupt/modify the operation of a target system, for example a PC, under test or development. Readers may recognize words like “set a breakpoint,” “break-in,” “dump memory” as being the jargon of kernel debugging. All the major operating system vendors provide kernel debuggers: WinDBG for Windows and Firescope for Linux are two examples. In addition, there are many third-party providers.
The first kernel debuggers used RS-232 to communicate with the system under development. Later on 1394, USB, JTAG and Ethernet were used as communication links for kernel debugging.
Why is 1394 the Best?
By design, 1394 has direct access to the physical memory of the target system. If the target system does not have 1394 on-board, 1394 add-in cards (PCI, PCI-Express, Cardbus, Express Card) can provide access to physical memory for the kernel debugger.
Any 1394 controller supporting OHCI 1.0 works. Standard installation for Windows, Linux or MacOs systems is not difficult. The remote PC’s 1394 controller handles all data transfers with the target system and minimal code is required on the target system.
While not impossible to surmount, the barriers to kernel debugging with USB are high. A short list of the USB barriers includes the following: optional features in the USB controller are required (off-the-shelf USB controllers may not have them); special USB cables are required (A-to-A); BIOS changes may be required to identify the USB debug port and keep it in high-speed mode. Additional code is required to run on the target system.
By design, 1394 transfers files quickly. A 2004-era full system memory dump of 128MB takes approximately 15 seconds using 1394a (400mbps) compared with more than three full hours with RS232 (115kbps). Today, system memory is typically several gigabytes, so that memory dump over 1394a takes 4 minutes or takes 2 minutes with 1394b (800mbps). And the RS-232 port is no longer to be found if you had 45 hours to spare.
How 1394 supports dumps greater than 4GB?
The kernel debug challenges today are personal computers and systems running 64-bit operating systems with memory ranging from 4GB up to 64GB. The 1394 Open Host Controller Interface (OHCI) specification includes an optional physical upper bound register, which provides a mechanism for implementations to specify physical access for offsets above 48’0000_FFFF_FFFF (4GB), as shown in the figure below.
(To view larger image, click here)
Both WinDBG and Firescope kernel debuggers make use of the OHCI upper bound register and support full system memory dumps for 64-bit systems.
Planned and implemented improvements
Microsoft has now removed the dependency between 1394 debug and core 1394 stack for power state notifications and improved HAL PCI and PC Card device initialization.
There is added support for 1394 user mode debug, along with auto-disable of the host controller when used for debugging. 1394 products have been introduced at 1600 and 3200mbps so that 64gb memory dumps can be done in just 16 minutes.
Here is a list of references to help you implement kernel debugging with 1394.
1 - http://bandwidthco.com/whitepapers/compforensics (pdf)
2 - https://www.virtualbox.org/wiki/Windows_Kernel_Debugging
3 - http://msdn.microsoft.com/en-us/library/
4 - http://www.mjmwired.net/kernel/Documentation/
5 - http://smartclientfactory.blogspot.com/2008
6 - https://lkml.org/lkml/2006/4/3/301
David Thompson is a distinguished engineer in the Systems Applications engineering area at LSI Corporation. He has an MSEE from Stanford University, is a Senior member of IEEE and has been with Bell Laboratories and its inheritors for 32 years before becoming LSI Corporation. His group develops 1394, USB and other serial-bus technology standard products for LSI. He is serving in his eleventh year as a member of the 1394 Trade Association’s Board of Directors.
This article provided courtesy of Embedded.com and Embedded Systems Design Magazine. Sign up for subscriptions and newsletters. Copyright © 2011 UBM--All rights reserved.