Causal analysis of task completion errors in spoken music retrieval interactions

Sunao Hara, Norihide Kitaoka, Kazuya Takeda

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we analyze the causes of task completion errors in spoken dialog systems, using a decision tree with N-gram features of the dialog to detect task-incomplete dialogs. The dialog for a music retrieval task is described by a sequence of tags related to user and system utterances and behaviors. The dialogs are manually classified into two classes: completed and uncompleted music retrieval tasks. Differences in tag classification performance between the two classes are discussed. We then construct decision trees which can detect if a dialog finished with the task completed or not, using information gain criterion. Decision trees using N-grams of manual tags and automatic tags achieved 74.2% and 80.4% classification accuracy, respectively, while the tree using interaction parameters achieved an accuracy rate of 65.7%. We also discuss more details of the causality of task incompletion for spoken dialog systems using such trees.

Original languageEnglish
Title of host publicationProceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012
PublisherEuropean Language Resources Association (ELRA)
Pages1365-1372
Number of pages8
ISBN (Electronic)9782951740877
Publication statusPublished - Jan 1 2012
Externally publishedYes
Event8th International Conference on Language Resources and Evaluation, LREC 2012 - Istanbul, Turkey
Duration: May 21 2012May 27 2012

Other

Other8th International Conference on Language Resources and Evaluation, LREC 2012
CountryTurkey
CityIstanbul
Period5/21/125/27/12

Fingerprint

causal analysis
music
dialogue
interaction
communication technology
causality
Completion
Interaction
Causal
Music
cause
Tag
performance
Decision Tree

Keywords

  • Interaction parameters
  • Spoken dialog
  • Task incompletion

ASJC Scopus subject areas

  • Linguistics and Language
  • Language and Linguistics
  • Education
  • Library and Information Sciences

Cite this

Hara, S., Kitaoka, N., & Takeda, K. (2012). Causal analysis of task completion errors in spoken music retrieval interactions. In Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012 (pp. 1365-1372). European Language Resources Association (ELRA).

Causal analysis of task completion errors in spoken music retrieval interactions. / Hara, Sunao; Kitaoka, Norihide; Takeda, Kazuya.

Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012. European Language Resources Association (ELRA), 2012. p. 1365-1372.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hara, S, Kitaoka, N & Takeda, K 2012, Causal analysis of task completion errors in spoken music retrieval interactions. in Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012. European Language Resources Association (ELRA), pp. 1365-1372, 8th International Conference on Language Resources and Evaluation, LREC 2012, Istanbul, Turkey, 5/21/12.
Hara S, Kitaoka N, Takeda K. Causal analysis of task completion errors in spoken music retrieval interactions. In Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012. European Language Resources Association (ELRA). 2012. p. 1365-1372
Hara, Sunao ; Kitaoka, Norihide ; Takeda, Kazuya. / Causal analysis of task completion errors in spoken music retrieval interactions. Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012. European Language Resources Association (ELRA), 2012. pp. 1365-1372
@inproceedings{1854144f9c7b4f1fb11cdd173b85b358,
title = "Causal analysis of task completion errors in spoken music retrieval interactions",
abstract = "In this paper, we analyze the causes of task completion errors in spoken dialog systems, using a decision tree with N-gram features of the dialog to detect task-incomplete dialogs. The dialog for a music retrieval task is described by a sequence of tags related to user and system utterances and behaviors. The dialogs are manually classified into two classes: completed and uncompleted music retrieval tasks. Differences in tag classification performance between the two classes are discussed. We then construct decision trees which can detect if a dialog finished with the task completed or not, using information gain criterion. Decision trees using N-grams of manual tags and automatic tags achieved 74.2{\%} and 80.4{\%} classification accuracy, respectively, while the tree using interaction parameters achieved an accuracy rate of 65.7{\%}. We also discuss more details of the causality of task incompletion for spoken dialog systems using such trees.",
keywords = "Interaction parameters, Spoken dialog, Task incompletion",
author = "Sunao Hara and Norihide Kitaoka and Kazuya Takeda",
year = "2012",
month = "1",
day = "1",
language = "English",
pages = "1365--1372",
booktitle = "Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012",
publisher = "European Language Resources Association (ELRA)",

}

TY - GEN

T1 - Causal analysis of task completion errors in spoken music retrieval interactions

AU - Hara, Sunao

AU - Kitaoka, Norihide

AU - Takeda, Kazuya

PY - 2012/1/1

Y1 - 2012/1/1

N2 - In this paper, we analyze the causes of task completion errors in spoken dialog systems, using a decision tree with N-gram features of the dialog to detect task-incomplete dialogs. The dialog for a music retrieval task is described by a sequence of tags related to user and system utterances and behaviors. The dialogs are manually classified into two classes: completed and uncompleted music retrieval tasks. Differences in tag classification performance between the two classes are discussed. We then construct decision trees which can detect if a dialog finished with the task completed or not, using information gain criterion. Decision trees using N-grams of manual tags and automatic tags achieved 74.2% and 80.4% classification accuracy, respectively, while the tree using interaction parameters achieved an accuracy rate of 65.7%. We also discuss more details of the causality of task incompletion for spoken dialog systems using such trees.

AB - In this paper, we analyze the causes of task completion errors in spoken dialog systems, using a decision tree with N-gram features of the dialog to detect task-incomplete dialogs. The dialog for a music retrieval task is described by a sequence of tags related to user and system utterances and behaviors. The dialogs are manually classified into two classes: completed and uncompleted music retrieval tasks. Differences in tag classification performance between the two classes are discussed. We then construct decision trees which can detect if a dialog finished with the task completed or not, using information gain criterion. Decision trees using N-grams of manual tags and automatic tags achieved 74.2% and 80.4% classification accuracy, respectively, while the tree using interaction parameters achieved an accuracy rate of 65.7%. We also discuss more details of the causality of task incompletion for spoken dialog systems using such trees.

KW - Interaction parameters

KW - Spoken dialog

KW - Task incompletion

UR - http://www.scopus.com/inward/record.url?scp=85037376279&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85037376279&partnerID=8YFLogxK

M3 - Conference contribution

SP - 1365

EP - 1372

BT - Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012

PB - European Language Resources Association (ELRA)

ER -