Re: EAGLESEVAL: A question from George Doddington.
At 5:09 PM +0200 8/25/98, Maghi King wrote:
>Eduard Hovy wrote:
>> At 4:33 PM +0200 8/11/98, Maghi King wrote:
>> >A question from George Doddington:
>> >Several presenters have argued for moving evaluation from the technical
>> >(lower) level toward the application (user) level. If evaluation is
>> >viewed as a means of making progress, this implies that these presenters
>> >believe that it is more important (or easier?) to make progress at the
>> >application level than on the underlying problems. Is this a fair
>> >representation of the presenters' perspective?
>>
>> This is, of course, a loaded question. A counter is: Does Mr. Doddington
>> believe it is better to make progress on the underlying problems even if
>> they have no effect at the application level?
>
>And now I can't resist: what would it be like to make progress on an
>underlying level and NOT have an effect on the application level?
>I can imagine going in for something that is really overkill, like a
>full blown understanding system to deal with simple queries that could
>be dealt with just as well by very straight forward methods, but surely
>you'd still have an effect on the application level, if it was only the
>negative effect of slowing evrything down unnecessarily?
I can think of several places in which people make progress on underlying
levels that have no real effect on the application level. For example,
many groups have insisted that detailed syntactic parsing is the key,
without demonstrating significant increases in output quality (or a
significant decrease in speed). Ditto for the heavily semanticized
Interlingua-based approaches, who can spend years building ontologies
and fancy lexicons but still show no improvement on general text in the
large. (I happen myself to believe that semantics is the right direction
to go, but I do not believe that we will get adequately large semantic
knowledge bases and/or lexicons using current manual techniques.)
E
----------------------------------------------------------------------------
Eduard Hovy
email: hovy@isi.edu USC Information Sciences Institute
tel: 310-822-1511 ext 731 4676 Admiralty Way
fax: 310-823-6714 Marina del Rey, CA 90292-6695
project homepage: http://www.isi.edu/natural-language/nlp-at-isi.html