Testing Orca Components

Note:

Reviewed for release 2.0.0.

The general consensus in the software testing community is that testing component-based software is generally more challenging compared to monolithic. We are planning on doing much more to test existing Orca components and simplify testing of new ones. For an introduction to the topic, see this relatively new book: Testing and Quality Assurance for Component-Based Software.

Considerations

These are some of abnormal conditions which most Orca components must be able to deal with:

• no network
• no Registry
• no IceStorm
• no hardware
• no required interfaces

(if the component attempts to "deal with the failure", e.g. reconnect to missing hardware or a required interface, make sure it's possible to stop it by Ctrl-C)

Repeat tests under these conditions:

• on startup / while running
• both stand-alone and as a service
• for all available drivers
• for key configuration options
• high/low CPU load (?)

Components dedicated to testing other components

One way to test components is to write dedicated "test machines" in the distributions. They hook up to other components to excersize their functionality.