More on Real-time Testing

This article is in response to a request from one of our readers to write more on real-time testing for small systems.

Simple Testing on the Host

Embedded systems can benefit from testing on the host prior to testing on the target system for the following reasons.

1. The host generally has a better development and debugging environment than the target which makes the process of edit, compile, link, and test go faster.

2. All the overhead of target system debugging is eliminated — downloading, acquiring the ICE which may be shared with others on the project, and so forth.

Some parts of real-time applications may be ideally suited for host testing. For example, consider a real-time application that, among other things, requires a trivial in memory real-time database. The database could be coded and debugged on the host with no need for the target system.

Other parts of the real-time application may require a small bit of fudging before host testing can take place. Consider the requirement to periodically read an analog channel every 20 ms and report an error if the channel reading is out of range. There are two problems: (1) the host does not allow for multi-tasking or timing increments down to 20 ms, and (2) there are no analog channels to read. The first problem is solved if your kernel will run on the host computer. If this is not the case, then the sample rate can be ignored and the channel can be sampled in a tight loop. The second problem can be solved by writing a function called readChannel, which on the host returns a meaningful random number. When running on the target system the function would be changed to read the actual value. One may be tempted to using conditional compiles in this type of situation, but we recommend against it - it makes for messy code.

Overall testing on the host requires that the kernel/OS run on the host as well as the target, otherwise, the dynamics of multi-tasking and inter-task communication cannot be tested. If the kernel/OS does not run on the host, the best you can do is test tasks individually and integrate them together on the target for final testing.

Simple Testing on the Target

Once host testing is finished target testing can begin. If your target machine is a stand-alone embedded board with no disk, display, or keyboard, you will almost certainly need an in-circuit emulator (ICE). The ICE stands between the host and the target, typically connected to the host via RS-232, and to the target via the processor interface cable. The ICE provides debugging software which allows the programmer to debug the target from the host. For more information about the ICE and its features see the definition.

Simple target testing is performed by getting the software loaded onto the target and running the target through its normal modes of operation. Try to test and debug one thing at a time, starting with the simple features first. When all features of the target seem to work, then try using the target in the way that the end user will use it. When the target seems to be working without problems it is time to try to make it fail. This is called stress testing.

Stress Testing

Stress testing is achieved by loading the system down until it fails. As the system approaches failure, bugs may surface that otherwise would not have been seen. Depending upon the system, stress testing can be performed on the actual equipment, or by writing software to simulate real world events.

Scenario and Random Simulation Testing

Complex real-time systems consist of concurrent inter-communicating state machines. Typically each state machine is a task. From any given state within a state machine, various events will trigger various actions, and a possible transition into another state. Testing such systems is a formidable task. It is not enough to test all events that a given state with a given state machine reacts to because the state machine may also depend on conditions that are functions of states in one or more of the other state machines. One approach to testing such a system is by random generation of all system events and very long test periods, the theory being that sooner or later all possible scenarios will be tested.

As was stated in last month's testing article, this is a large subject and if you would like more coverage on this or other subjects please send us email.