Back to EAGLES-II Home Page

EAGLES-II - Links to related sites

This page is devoted to links to other sites of interest to those involved in evaluation. If you have any suggestions for other useful links please send them to

EAGLES-I (EAGLES Evaluation Working Group). The final report: "Evaluation of natural language processing systems".

TEMAA (A Testbed Study of Evaluation Methodologies: Authoring Aids). A project building on the EAGLES-1 evaluation framework formalised and implemented as a Parameterisable Testbed (PTB) and applied to the concrete evaluation of Danish and Italian.

TSNLP (Test Suites for Natural Language Processing). A project developing guidelines to create test suites for evaluation and testing of NL systems, as well as substantial test suites in three languages: English, French and German.

DiET (Diagnostic and Evaluation tools for Natural Language Applications). A project to develop data, methods and tools for glass box evaluation of NLP components.

ELSE (Evaluation in Language and Speech Engineering). A project to provide a framework for semi-automatic quantitive black-box evaluation for the language and speech engineering industry.

(International Organization for Standardization)

TREC (Text REtrieval Conference) A programme to encourage research in information retrieval from large text collections.

MUC (Message Understanding Conference) performing evaluations of information extraction systems according to pre-established tasks.

GRACE (Grammaires et Ressources pour les Analyseurs de Corpus et leur Evaluation )

SENSEVAL (Evaluating Word Sense Disambiguation Systems)

LDC (the Linguistics Data Consortium) Creates collects and distributes speech and text databases, lexicons and other resources for research and development purposes.

NIST Spoken natural language processing Promotes the advancements of speech recognition and understanding by developing measurement methods, providing reference materials (speech corpora), coordinating benchmark tests within the R&D community and building prototypes.

NLPIR (Natural Language Processing and Information Retrieval) promotes the use of more efficitent techniques for maniupulating (largely ) unstructured textual information, organising TREC congernecs, creating test collections, developing improved evaluation methodology

SERCO (Serco Usability Services) Provides usability and user-centred design services and expert and user-based evaluation

Projects presented at the Brussels workshop:

INUSE (Information Engineering Usability Support Centres) an EU-sponsored project to set up a network of Usability Support Centres across Europe to assist both companies and projects within the EC Telematics Applications Programme.

MEGATAQ (Methods and Guidelines for the Assessment of Telematics Application Quality) The MEGATAQ objective is to provide Telematics Applications Projects (TAPs) with user-centred evaluation guidelines and consultancy.

LAUREATE BT's text-to-speech synthesis system

Back to EAGLES-II Home Page