Annotation -- SpeakerB -- FacialDisplay

primary track
SpeakerB.FacialDisplay

Speaker B: Face track. All attributes are the same as for Speaker A.

Attributes
  ValueSet "Face-type" (4 tokens)    Face   Face attributes refer to general face expressions.
  ValueSet "Eyebrows-type" (3 tokens)    Eyebrows   Eyebrows attributes refer to eyebrows movements.
  ValueSet "Eyes-type" (5 tokens)    Eyes   Eyes refer to movements of the eyelids and not to gaze.
  ValueSet "Gaze-Direction-type" (6 tokens)    GazeDirection   Gaze direction is defined with respect to the subject's own body axis. Gaze and head movement are related. For instance, if head is up, gaze is also up if the subject does not move the pupils.
  ValueSet "Gaze-Interlocutor-type" (2 tokens)    GazeInterlocutor   GazeInterlocutor defines gaze direction with respect to the interlocutor. This feature is new wrt. MUMIN 3.3. GazeInterlocutor and GazeDirection are of course related. Which is the default value of GazeDirection when looking at the interlocutor depends of course on how the dialogue participants are positioned wrt. each other. In case of multi-party conversations, one may want to distinguish among several interlocutors.
  ValueSet "Mouth-Openness-type" (2 tokens)    MouthOpenness   Mouth openness can be combined with mouth lips values.
  ValueSet "Mouth-Lips-type" (5 tokens)    MouthLips   Mouth lips can be combined with mouth openness values.
  ValueSet "Head-Movement-type" (8 tokens)    HeadMovement   Head movements are defined similarly to gaze wrt. the subject's own body. Gaze and head movement are related. For instance, if head is up, gaze is also up if the subject does not move the pupils. The values are slightly different from MUMIN 3.3 in that repetition is now a separate feature.
  ValueSet "Head-Repetition-type" (2 tokens)    HeadRepetition   It indicates whether the movement is repeated or not. A repeated gesture is a sequence of similar gestures in rapid succession. See also HandRepetition. New feature wrt. MUMIN 3.3.
  ValueSet "F-Basic-type" (2 tokens)    FeedbackBasic   The basic feedback feature is used to distinguish between feedback giving and eliciting.
  ValueSet "F-Understand-type" (2 tokens)    FeedbackUnderstanding   Feedback understanding is used to specify whether basic feedback is accompanied by signs of understanding or non-understanding.
  ValueSet "F-Acceptance-type" (2 tokens)    FeedbackAcceptance   Basic feedback can also be accompanied by signs of agreeing or not agreeing.
  ValueSet "Emotion/Attitude-type" (13 tokens)    Emotion/Attitude   Emotions and attitudes can co-occur with any of the communicative features. They include i.a. Ekman's six basic emotions: happiness, surprise, anger, sadness, fear, disgust/contempt.
  ValueSet "Turn-type" (6 tokens)    Turn   Turn features refer to the way in which one of the dialogue participants gains, ends or holds the turn, and reflect whether this is done in agreement or not. In a more coarse-grained scheme, one could do with the three values TurnGain, TurnEnd and TurnHold (see MUMIN 3.3).
  ValueSet "Sequencing-type" (4 tokens)    Sequencing   Sequencing is concerned with speech act organisation and structuring into sub-dialogues. It should not be confused with turn-management, that models turn shifts among dialogue participants.
  ValueSet "InformationStructure-type" (1 tokens)    InformationStructure   The gesture has a function relating to information structure, i.e. the organisation of the information within a sentence or utterance. It could be underlining the topic, the focus or creating a contrast.
  ValueSet "Semiotic-type" (6 tokens)    SemioticType   Based on Peirce's theory, three types are defined: indexical, iconic and symbolic. Combinations are also allowed, which is new wrt. MUMIN 3.3.
  MultiLink    Addition   The gesture adds information to the corresponding speech segment, e.g. by making a concept more specific. See Poggi and Magno Caldognetto (1996). The feature was subsumed under Compatible in MUMIN 3.3.
  MultiLink    Reinforcement   The gesture has the same meaning as the corresponding speech segment. See Poggi and Magno Caldognetto (1996). It was subsumed under Compatible in MUMIN 3.3.
  MultiLink    Substitution   The gesture occurs in absence of words. See Poggi and Magno Caldognetto (1996).
  MultiLink    Contradiction   The gesture contradicts the corresponding speech segment e.g. to denote sarcasm or irony. See Poggi and Magno Caldognetto (1996). It corresponds to Incompatible in MUMIN 3.3.
  MultiLink    RelationOther   To be used if none of the other values are appropriate. A comment *must* be added.

 

Attribute Values

Face

Face attributes refer to general face expressions.

Smile   The facial expression shows pleasure, favour, amusement, or sometimes derision and scorn. Smile is characterized by an upturning of the corners of the mouth and usually accompanied by a brightening of the face and eyes.  
Laughter   The facial expression shows merriment, amusement, or derision, or nervousness and it is accompanied by an audible vocal expulsion of air from the lungs that can range from a loud burst of sound to a series of chuckles.  
Scowl   An angry expression, like a frown. The term "frown" is reserved for the attribute Eyebrows.  
FaceOther   To be used if none of the other values are appropriate. A comment *must* be added.  

Eyebrows

Eyebrows attributes refer to eyebrows movements.

Frown   The eyebrows contract and move towards the nose.  
Raise   The eyebrows are lifted.  
BrowsOther   To be used if none of the other values are appropriate. A comment *must* be added.  

Eyes

Eyes refer to movements of the eyelids and not to gaze.

X-open   Exaggerated Opening: the eyes are wide open as in the case of surprise.  
CloseBoth   The eyes are both closed, e.g. to underline focusing, but not in case of a biological blinking.  
CloseOne   One eye winks, that is opens and closes quickly.  
CloseRepeat   Both eyes wink, that is open and close quickly.  
EyesOther   To be used if none of the other values are appropriate. A comment *must* be added.  

GazeDirection

Gaze direction is defined with respect to the subject's own body axis. Gaze and head movement are related. For instance, if head is up, gaze is also up if the subject does not move the pupils.

GazeForward   The subject is looking forward wrt. own body.  
GazeBackward   The subject is looking backward wrt. own body (implies turning the head and naybe part of the torso).  
GazeUp   The subject is looking up wrt. own body.  
GazeDown   The subject is lookng down wrt. own body (the head may be bent or not).  
GazeSide   The subject is looking to the side of own body.  
GazeDirectionOther   To be used if none of the other values are appropriate, for instance for complex movements. A comment *must* be added.  

GazeInterlocutor

GazeInterlocutor defines gaze direction with respect to the interlocutor. This feature is new wrt. MUMIN 3.3. GazeInterlocutor and GazeDirection are of course related. Which is the default value of GazeDirection when looking at the interlocutor depends of course on how the dialogue participants are positioned wrt. each other. In case of multi-party conversations, one may want to distinguish among several interlocutors.

ToInterlocutor   The subject is looking at the interlocutor.  
AwayFromInterlocutor   The subject is looking at the interlocutor.  

MouthOpenness

Mouth openness can be combined with mouth lips values.

OpenMouth   The mouth is open or opens as in the case of surprise (that is not just to talk).  
CloseMouth   The mouth is close (relevant in combination with other values, e.g. lips retracted).  

MouthLips

Mouth lips can be combined with mouth openness values.

CornersUp   Corners up as in a smile.  
CornersDown   Corners down as in a scowl, sulk or sad expression.  
Protruded   The lips are rounded and protruded.  
Retracted   The lips are sucked-in, retracted in the mouth.  
LipsOther   To be used if none of the other values are appropriate. A comment *must* be added.  

HeadMovement

Head movements are defined similarly to gaze wrt. the subject's own body. Gaze and head movement are related. For instance, if head is up, gaze is also up if the subject does not move the pupils. The values are slightly different from MUMIN 3.3 in that repetition is now a separate feature.

Nod   A head movement down-up.  
Jerk   A quick head movement backwards up.  
HeadBackward   A movement of the head backward (up), this can either be a movement of the head only or can be movement of the whole trunk. This movement occurs often as a turn accepting signal.  
HeadForward   A movement of the head forward (down), this can either be a movement of the head only or can be a movement of the whole trunk. This movement occurs often as a turn elicit signal.  
Tilt   A movement of the head leaning on one side.  
SideTurn   A rotation of the head towards one side.  
Waggle   A movement of the head back and forth, side to side, like a mixture of shake and move backward or forward. Usually produced to show uncertainty, doubtfulness.  
HeadOther   To be used if none of the other values are appropriate. A comment *must* be added.  

HeadRepetition

It indicates whether the movement is repeated or not. A repeated gesture is a sequence of similar gestures in rapid succession. See also HandRepetition. New feature wrt. MUMIN 3.3.

Single   The movement is not repeated.  
Repeated   The movement is repeated.  

FeedbackBasic

The basic feedback feature is used to distinguish between feedback giving and eliciting.

FeedbackGive   The subject shows they have perceived the message and are willing to maintain contact and go on in the communication. In MUMIN 3.3, the two subtypes CPU and CP were distinguished.  
FeedbackElicit   The subject elicits willingness to maintain contact and go on in the communication, or signs that the interlocutor has perceived the message. In MUMIN 3.3, the two subtypes CPU and CP were distinguished.  

FeedbackUnderstanding

Feedback understanding is used to specify whether basic feedback is accompanied by signs of understanding or non-understanding.

Understand   The subject gives or elicits signs that the message is being understood.  
NonUnderstand   The subject gives or elicits signs that the message is not being understood.  

FeedbackAcceptance

Basic feedback can also be accompanied by signs of agreeing or not agreeing.

Accept   The subject gives or elicits signs of agreement.  
NonAccept   The subject gives or elicits signs of lack of agreement.  

Emotion/Attitude

Emotions and attitudes can co-occur with any of the communicative features. They include i.a. Ekman's six basic emotions: happiness, surprise, anger, sadness, fear, disgust/contempt.

Happy   --- 
Sad   --- 
Surprised   --- 
Disgusted   --- 
Angry   --- 
Frightened   --- 
Certain   --- 
Uncertain   --- 
Interested   --- 
Uninterested   --- 
Disappointed   --- 
Satisfied   --- 
Other   --- 

Turn

Turn features refer to the way in which one of the dialogue participants gains, ends or holds the turn, and reflect whether this is done in agreement or not. In a more coarse-grained scheme, one could do with the three values TurnGain, TurnEnd and TurnHold (see MUMIN 3.3).

TurnTake   The speaker takes a turn that wasn't offered, possibly by interrupting.  
TurnAccept   The speaker accepts a turn that is being offered.  
TurnYield   The speaker releases the turn under pressure.  
TurnElicit   The speaker offers the turn to the interlocutor.  
TurnComplete   The speaker completes the turn as well as the conversation. The turn is not passed on.  
TurnHold   The speaker wishes to keep the turn (usually done by rotating the head and the gaze away from the listener).  

Sequencing

Sequencing is concerned with speech act organisation and structuring into sub-dialogues. It should not be confused with turn-management, that models turn shifts among dialogue participants.

SeqOpen   Opening uence indicates that a new speech sequence is starting, e.g. a gesture occurring together with the phrase "by the way...".  
SeqResume   Resume sequence indicates that a preceding speech sequence is resumed after an interruption or a false start.  
SeqContinue   Continue sequence indicates that the current speech sequence is going on, e.g. a gesture occurring together with enumerative phrases such as "the first... the second... the third...".  
SeqClose   Closing sequence indicates that the current speech sequence is closed, e.g. a gesture occurring together with phrases such as "that's it, that's all".  

InformationStructure

The gesture has a function relating to information structure, i.e. the organisation of the information within a sentence or utterance. It could be underlining the topic, the focus or creating a contrast.

InfoStructure   --- 

SemioticType

Based on Peirce's theory, three types are defined: indexical, iconic and symbolic. Combinations are also allowed, which is new wrt. MUMIN 3.3.

IndexDeictic   Indexical gestures express a relation of cause-effect between the sign (the gesture) and its meaning. In particular, Indexical Deictics locate aspects of the discourse in the physical space (e.g. by pointing). They can also be used to index the addressee.  
IndexNon-deictic   Indexical gestures express a relation of cause-effect between the sign (the gesture) and its meaning. In Indexical Non-deictic the indexical relation is between the gesture and the effect it establishes. Batonic or beat gestures fall into this category.  
Iconic   Iconic gestures express a semantic feature by similarity or homomorphism. Examples are gestures that express size (length, height, etc.) of an object mentioned in the discourse. Included in this category are also gestures that are sometimes called metaphoric.  
Symbolic   Symbolic gestures (emblems) are gestures in which the relation between form and content is based on social convention (e.g. the okay gesture). They are culture-specific.  
IconicandIndexNon-deictic   Iconic and IndexNon-deictic. New type wrt. MUMIN 3.3.  
SymbolicandIndexNon-deictic   Simbolic and IndexNon-deictic. New type wrt. MUMIN 3.3.  

Addition

The gesture adds information to the corresponding speech segment, e.g. by making a concept more specific. See Poggi and Magno Caldognetto (1996). The feature was subsumed under Compatible in MUMIN 3.3.


Reinforcement

The gesture has the same meaning as the corresponding speech segment. See Poggi and Magno Caldognetto (1996). It was subsumed under Compatible in MUMIN 3.3.


Substitution

The gesture occurs in absence of words. See Poggi and Magno Caldognetto (1996).


Contradiction

The gesture contradicts the corresponding speech segment e.g. to denote sarcasm or irony. See Poggi and Magno Caldognetto (1996). It corresponds to Incompatible in MUMIN 3.3.


RelationOther

To be used if none of the other values are appropriate. A comment *must* be added.


Annotation -- SpeakerB -- FacialDisplay