The first objective was the development of a concrete instance of an evaluation package. A prototype tool, comprising various modules has been built, the framework model has been concretely instantiated for spelling checkers, and some data material has been produced. The evaluation package is not complete, but the result is an important contribution to such a package.
Secondly, feedback had to be given to the formal framework methodology. The experimentation has confirmed the viability of the formal approach to evaluation taken in the project, which has provided a sound basis for the construction of a collection of programs for carrying out evaluations. With respect to the underlying ISO model, the TEMAA framework has in fact gone further in the direction of a formalised view of evaluation by attempting to formalise what the ISO model calls 'stated or implied needs'.
The main body of the report outlines the approach taken, followed by a number of discussion sections. In appendices, testing reports on Danish and Italian spelling checkers are presented.
The primary responsibility for writing the various sections of the report is as follows:
* Overview of the PTB, tools for test suites and general notes: Steven Krauwer and Louis des Tombe (SST);
* Testing: Sandra Manzi (ISSCO);
* Test materials: Patrizia Paggio (CST);
* User profiles: Shona Douglas (LTG);
* Danish test report: Patrizia Paggio and Gurli Rohde (CST);
* Italian test report: Sandra Manzi (ISSCO).
Comments on all sections have been provided by all project participants. General editing was carried out by Shona Douglas (LTG).
The software programs described in the report have been developed by:
* Louis des Tombe (STT): ASCC, ET, errgen;
* Steven Krauwer (STT): PTB;
* Shona Douglas (LTG) and Gurli Rohde (CST): ER.