The need for Technology Evaluation as assessed by ELSE (Evaluation in Language
and Speech Engineering") will be presented. The following questions
will be addressed: Why does the field of Language Engineering needs to
have comparative evaluation? Where does Technology Evaluation appears in
the global development life-cycle of a product (from the research lab to
the end-user) ? What are the pitfall to avoid and the expected benefits
of implementing Technology Evaluation on a large scale ? What should we
conclude from the recent European experiments ? Why a quantitative black-box
approach is essential ? How is it complementatry to other approaches, like
for instance more user-oriented evaluation ? The conclusion will offer
for discussion a proposition for implementing quantitative black-box comparative
evaluation in FP5.