Apache Harmony is retired at the Apache Software Foundation since Nov 16, 2011.

The information on these pages may be out of date, or may refer to resources that have moved or have been made read-only.
For more information please refer to the Apache Attic

Test Design

The following guidelines are currently PROPOSED and being discussed on the development mailing list dev@harmony.apache.org. Please direct comments and questions there.

Back to top

Further Steps

Back to top

Comparative Approach

The simplest example of comparative apporach is the following.

Tester: My test fails on Harmony VM and passes on RI. Please, fix Harmony VM.

This usually does not work for stress tests.

Developer: Who told you that OutOfMemoryError should be thrown in your thread? My finalizer thread is just a normal java thread, like yours, and it can fail as well. You have a bug in your test.

There are multiple reasons why we always will have such bugs in the tests.

How can we have a maintainable test product takung all this limitation into account? We need to learn how to live with occasional failures of the stress tests. This means, instead of fail, the test should better report how good it is on Harmony VM compared to RI:

Developers are better convinsed to fix "the worst issue" or "dergadation" instead of "some issue".

Several metrics for each test:

Back to top