Article status
This article covers all important aspects of BVMS dump creation:
- introduction to dumps
- tools for dump creation
- error scenarios and corresponding dump creation methods
Quick links:
Introduction to dumps
For the Windows operating system, there are two types of dumps: user-mode dumps and kernel-mode dumps.
User-mode dumps
User-mode dumps are either created automatically by the Windows operating system in case of an application crash, or may also be created manually by using task manager or another tool like ntsd or procdump.
There are two types of user-mode dumps:
- full user-mode dumps
- minidumps
Minidumps are usually more compact, but they may also contain more information than a full user-mode dump depending on the options that were used when creating the dump.
The recommended dump type for BVMS dumps are Minidumps, created with the "-ma" option.
Minidumps which are created with this option, contain full memory data (including virtual memory layout) as well as handle, thread and lock information.
This article is mainly about user-mode dumps - user-mode dumps are the dump type needed for BVMS issue analysis.
Kernel-mode dumps
Kernel-mode dumps are created when the Windows OS runs into a blue screen.
From BVMS maintenance perspective, kernel-mode dumps are not relevant.
The experience from the past years is, that not one single case was reported where BVMS caused a blue screen.
Dump contents
A user-mode dump is a full process image. It contains memory contents, thread information, acquired locks and crash reasons for both managed and unmanaged code.
Dump contents allow to track down the causes of these types of issues:
- Application crashes and their causes and origins (managed or unmanaged code)
- Memory or handle leaks and their origins
- Deadlocks, freezes and hangs and their origins
Tools for dump creation
There are a number of tools that can be used for dump creation:
- Task Manager
- ntsd
- Procdump
The recommended tool for creating BVMS dumps is Procdump.
The Procdump tool is recommended for the following reasons:
- Procdump is easier to use than ntsd
- Procdump can be used on different platforms (XP, Win2003 Server, Win7, Win2008 Server, Win8)
- Prodcump is an official Microsoft tool (as opposed to ntsd which was only part of Windows XP)
- Procdump can be used for all relevant dump creation scenarios - see details below
- Procdump supports several advanced options for triggering dump creation
Procdump is a command line tool which must be started from the command shell.
Batch files for dump creation
For easier BVMS maintenance, BVMS development provides a number of batch files for dump creation with procdump.
These dump files cover the most common dump creation cases.
The collection of batch files can be found here.
The batch file names follow a naming guideline: <Executable>_<Dump use case>
where <Executable> is one of:
- ConfigClient
- ConfigWizard
- OperatorClient
- ManagementServer
- ArchivePlayer
- ExceptionTest
and <Dump use case> is one of:
- SingleDump
- TwoDumps
- DumpOnException
E.g.
- batch file OperatorClient_SingleDump.bat immediately creates a dump of the Operator Client process,
- while OperatorClient_TwoDumps.bat creates two dumps of the Operator Client process with a delay of 1 hour between the dumps.
- OperatorClient_DumpOnException.bat creates a dump of the Operator Client process as soon as it throws an unhandled exception.
Note that also batches for executable "ExceptionTest.exe" are provided. These batches can be used for testing purposes - ExceptionTest.exe is a little test application that simply crashes when being started.
There is an additional batch file "ActivatePostmortemDebugging.bat" which registers procdump as Just-In-Time debugger.
Calling this batch will ensure that a dump is created when Windows Error Reporting is triggered by a crash.
To learn more about the advanced capabilities of the procdump tool, start batches ShowProcdumpHelp.bat and ShowProcdumpHelpSamples.bat. These batches display procdump options and some samples for how to use prodcump.
Error scenarios and corresponding dump creation methods
The procdump tool is useful for creating dumps for different problem scenarios:
Issues which can be easily reproduced
- application crashes which trigger windows error reporting
- memory or handle leaks
Issues which are hard to reproduce
- sporadic application crashes or crashes which do not trigger windows error reporting
- application sporadically becoming unresponsive (freezes/hangs)
Step-by-step guide
A. Dump creation for reproducible crashes that trigger Windows Error Reporting
To create a dump for a reproducible crash, follow these steps:
|
B. Dump creation for reproducible memory or handle leaks
To create a dump for a memory/handle leak, follow these steps:
1.Run the affected application/service (Operator Client, Configuration Client or Management Server Service) 2. Run the corresponding batch file for creating two dumps: *_TwoDumps.bat. E.g. run OperatorClient_TwoDumps.bat to create two dumps of the Operator Client process. Note that by default, the delay between the two dumps is set to one hour (3600 seconds). 3. Reproduce the steps that lead to the memory leak. Wait until the second dump was created. 4. Provide support with
|
C. Dump creation for crashes/hangs/freezes that are hard to reproduce
It is impossible to create a manual dump for sporadic crashes/hangs/freezes or for those cases where a crash does not result in display of a Windows Error Reporting dialog. For these cases, procdump should be registered as just-in-time debugger.
To register procdump as just-in-time debugger, follow these steps:
To deactivate the restarting option for Operator Client / Configuration Client: edit ..\Bosch\VMS\bin\ConfigClient.exe.config or ..\Bosch\VMS\bin\OperatorClient.exe.config.
|
Important note: in most cases, automatic generation of dumps works fine after registering procdump as just-in-time debugger. In some cases (especially on XP Systems), just-in-time debugging does not work reliably i.e. automatic creation of the dump file might fail despite having registered procdump as just-in-time debugger. 1.Don't click away the error dialog and Run *_SingleDump.bat to manually create a dump file or 2. Run *_DumpOnException.bat before starting the Operator Client/Configuration Client/Management Server. Method 1) should work in any case. Method 2) should work in most cases. |
D. Dump creation for other hard-to-reproduce issues
Procdump offers several other triggers for creating dumps:
- CPU peaks
- Performance counter peaks
- Application hanging for x seconds
- Unhandled exceptions
See procdump help for details on how to create dumps for these special situations.
E. Dump creation with "user mode stack trace database"
In some cases, the information provided by a "normal" dump is not sufficient for development analysis. For these cases, the OperatorClient.exe file must be temporarily modified while taking the dump.
The steps below describe the required prerequisites and the modification steps:
Prerequisites
In order to create a dump with user mode stack trace database, you need to get the gflags.exe tool.
This tool is part of the "Debugging Tools for Windows" which is part of "Windows Driver Kit Version 7.1.0".
1. Download the ISO image of Windows Driver Kit Version 7.1.0 (GRMWDK_EN_7600_1.ISO)
2. Extract ISO image using e.g. 7Zip
3. Run "KitSetup.exe" which is located in the extracted folder. An installation dialog opens.
4. Select "Debugging Tools for Windows 7.1.0", and click "OK" to install to an installation directory of your choice
5. In the installation directory, locate gflags.exe in subdirectory "Debuggers"
F. Dump creation with user mode stack trace database
To create a dump with user mode stack trace database, follow these steps:
- Copy file gflags.exe to folder <Program Files>\Bosch\VMS\bin
- Open a command shell and navigate to <Program Files>\Bosch\VMS\bin
- Activate user mode stack trace database in Operator Client by running command gflags.exe /i OperatorClient.exe +ust
- Reproduce the problem scenario and take the dump as described above
- Deactivate user mode stack trace database in Operator Client by running command gflags /i OperatorClient.exe -ust
