Table of Contents Author Guidelines Submit a Manuscript
Advances in Human-Computer Interaction
Volume 2010, Article ID 319406, 21 pages
http://dx.doi.org/10.1155/2010/319406
Research Article

The SEMAINE API: Towards a Standards-Based Framework for Building Emotion-Oriented Systems

German Research Center for Artificial Intelligence DFKI GmbH, Stuhlsatzenhausweg 3, 66123 Saarbrücken, Germany

Received 25 March 2009; Accepted 11 October 2009

Academic Editor: Anton Batliner

Copyright © 2010 Marc Schröder. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, “A survey of affect recognition methods: audio, visual and spontaneous expressions,” in Proceedings of the 9th International Conference on Multimodal Interfaces (ICMI '07), pp. 126–133, ACM, Nagoya, Japan, 2007. View at Publisher · View at Google Scholar
  2. M. Pantic and L. J. M. Rothkrantz, “Automatic analysis of facial expressions: the state of the art,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424–1445, 2000. View at Publisher · View at Google Scholar
  3. S. V. Ioannou, A. T. Raouzaiou, V. A. Tzouvaras, T. P. Mailis, K. C. Karpouzis, and S. D. Kollias, “Emotion recognition through facial expression analysis based on a neurofuzzy network,” Neural Networks, vol. 18, no. 4, pp. 423–435, 2005. View at Publisher · View at Google Scholar · View at PubMed
  4. A. Batliner, S. Steidl, B. Schuller et al., “Combining efforts for improving automatic classification of emotional user states,” in Proceedings of the 1st International Language Technologies Conference (IS-LTC '06), Ljubljana, Slovenia, 2006.
  5. B. Schuller, D. Seppi, A. Batliner, A. Maier, and S. Steidl, “Towards more reality in the recognition of emotional speech,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '07), vol. 4, pp. 941–944, 2007. View at Publisher · View at Google Scholar
  6. C. Peter and A. Herbon, “Emotion representation and physiology assignments in digital systems,” Interacting with Computers, vol. 18, no. 2, pp. 139–170, 2006. View at Publisher · View at Google Scholar
  7. P. Gebhard, “ALMA—a layered model of affect,” in Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS '05), pp. 177–184, Utrecht, The Netherlands, 2005.
  8. N. Tsapatsoulis, A. Raouzaiou, S. Kollias, R. Cowie, and E. Douglas-Cowie, “Emotion recognition and synthesis based on MPEG-4 FAPs,” in MPEG-4 Facial Animation—The Standard, Implementations, Applications, I. S. Pandzic and R. Forchheimer, Eds., John Wiley & Sons, Hillsdale, NJ, USA, 2002. View at Google Scholar
  9. E. Bevacqua, M. Mancini, R. Niewiadomski, and C. Pelachaud, “An expressive ECA showing complex emotions,” in Proceedings of the AISB Annual Convention, pp. 208–216, Newcastle, UK, 2007.
  10. F. Burkhardt and W. F. Sendlmeier, “Verification of acoustical correlates of emotional speech using formant synthesis,” in Proceedings of the ISCA Workshop on Speech and Emotion, pp. 151–156, Newcastle, UK, 2000.
  11. M. Schröder, “Approaches to emotional expressivity in synthetic speech,” in The Emotion in the Human Voice, K. Izdebski, Ed., Plural, San Diego, Calif, USA, 2008. View at Google Scholar
  12. G. Castellano, R. Bresin, A. Camurri, and G. Volpe, “Expressive control of music and visual media by full-body movement,” in Proceedings of the 7th International Conference on New Interfaces for Musical Expression, pp. 390–391, ACM, New York, NY, USA, 2007.
  13. ISO—International Organization for Standardization, “ISO 261: ISO general purpose metric screw threads—general plan,” 1998, http://www.iso.org/iso/iso_catalogue/catalogue_ics/catalogue_detail_ics.htm?csnumber=4165.
  14. ISO—International Organization for Standardization, “ISO/IEC 26300:2006: Information technology—Open Document Format for Office Applications (OpenDocument) v1.0,” 2006, http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=43485.
  15. D. Raggett, A. Le Hors, and I. Jacobs, HTML 4.01 Specification, 1999, http://www.w3.org/TR/html401/.
  16. K. van Deemter, B. Krenn, P. Piwek, M. Klesen, M. Schröder, and S. Baumann, “Fully generated scripted dialogue for embodied agents,” Artificial Intelligence, vol. 172, no. 10, pp. 1219–1244, 2008. View at Publisher · View at Google Scholar
  17. P. Piwek, B. Krenn, M. Schröder, M. Grice, S. Baumann, and H. Pirker, “RRL: a rich representation language for the description of agent behaviour in NECA,” in Proceedings of the AAMAS Workshop Conversational Agents, Bologna, Italy, 2002.
  18. B. Kempe, N. Pfleger, and M. Löckelt, “Generating verbal and nonverbal utterances for virtual characters,” in Proceedings of the 3rd International Conference on Virtual Storytelling (ICVS '05), vol. 3805 of Lecture Notes in Computer Science, pp. 73–76, 2005. View at Publisher · View at Google Scholar
  19. M. Löckelt and N. Pfleger, “Multi-party interaction with self-contained virtual characters,” in Proceedings of the 9th Workshop on the Semantics and Pragmatics of Dialogue (DIALOR '05), pp. 139–142, Nancy, France, 2005.
  20. R. Aylett, A. Paiva, J. Dias, L. Hall, and S. Woods, “Affective agents for education against bullying,” in Affective Information Processing, J. Tao and T. Tan, Eds., pp. 75–90, Springer, London, UK, 2009. View at Google Scholar
  21. A. Ortony, G. L. Clore, and A. Collins, The Cognitive Structure of Emotion, Cambridge University Press, Cambridge, UK, 1988.
  22. P. Gebhard, M. Schröder, M. Charfuelan et al., “IDEAS4Games: building expressive virtual characters for computer games,” in Proceedings of the 8th International Conference on Intelligent Virtual Agents (IVA '08), vol. 5208 of Lecture Notes in Computer Science, pp. 426–440, Springer, Tokyo, Japan, 2008. View at Publisher · View at Google Scholar
  23. Mystic Game Development, “EMotion FX,” http://www.mysticgd.com/site2007/.
  24. Luxand, Inc., “Luxand—Detect Human Faces and Recognize Facial Features with Luxand FaceSDK,” http://www.luxand.com/facesdk/.
  25. N. Dimakis, J. K. Soldatos, L. Polymenakos, P. Fleury, J. Curín, and J. Kleindienst, “Integrated development of context-aware applications in smart spaces,” IEEE Pervasive Computing, vol. 7, no. 4, pp. 71–79, 2008. View at Publisher · View at Google Scholar
  26. US National Institute of Standards and Technology (NIST), “NIST Data Flow System II,” 2008, http://www.nist.gov/smartspace/sf_presentation.html.
  27. N. Hawes, J. L. Wyatt, A. Sloman et al., “Architecture and representations,” in Cognitive Systems, H. I. Christensen, A. Sloman, G. Kruijff, and J. Wyatt, Eds., pp. 53–95, 2009. View at Google Scholar
  28. M. Henning, Choosing Middleware: Why Performance and Scalability do (and do not) Matter, ZeroC, 2009, http://www.zeroc.com/articles/IcePerformanceWhitePaper.pdf.
  29. “Semaine Project,” http://www.semaine-project.eu/.
  30. M. Schröder, R. Cowie, D. Heylen, M. Pantic, C. Pelachaud, and B. Schuller, “Towards responsive sensitive artificial listeners,” in Proceedings of the 4th International Workshop on Human-Computer Conversation, Bellagio, Italy, 2008.
  31. G. Banavar, T. Chandra, R. Strom, and D. Sturman, “A case for message oriented middleware,” in Proceedings of the 13th International Symposium on Distributed Computing (DISC '99), p. 846, 1999.
  32. “Apache ActiveMQ,” http://activemq.apache.org/.
  33. M. Hapner, R. Burridge, R. Sharma, J. Fialli, and K. Stout, Java Message Service (JMS) Specification Version 1.1, Sun Microsystems, 2002, http://java.sun.com/products/jms/docs.html.
  34. “The Apache Xerces Project—xerces.apache.org,” http://xerces.apache.org/.
  35. A. Le Hors, P. Le Hégaret, L. Wood et al., Document Object Model (DOM) Level 3 Core Specification, 2004, http://www.w3.org/TR/DOM-Level-3-Core/.
  36. “Behavior Markup Language (BML) Wiki,” 2008, http://wiki.mindmakers.org/projects:BML:main.
  37. M. Johnston, P. Baggia, D. C. Burnett et al., “EMMA: Extensible MultiModal Annotation markup language,” February 2009, http://www.w3.org/TR/emma/.
  38. D. C. Burnett, M. R. Walker, and A. Hunt, “Speech Synthesis Markup Language (SSML) Version 1.0,” 2004, http://www.w3.org/TR/speech-synthesis/.
  39. M. Schröder, P. Baggia, F. Burkhardt et al., Elements of an EmotionML 1.0, World Wide Web Consortium, 2008, http://www.w3.org/2005/Incubator/emotion/XGR-emotionml-20081120/.
  40. S. Kopp, B. Krenn, S. Marsella et al., “Towards a common framework for multimodal generation: the behavior markup language,” in Proceedings of the 6th International Conference on Intelligent Virtual Agents (IVA '06), vol. 4133 of Lecture Notes in Computer Science, pp. 205–217, 2006.
  41. D. Heylen, S. Kopp, S. Marsella, C. Pelachaud, and H. Vilhjálmsson, “Why conversational agents do what they do functional representations for generating conversational agent behavior,” in Proceedings of the Workshop on Functional Markup Language at the 7th International Conference on Autonomous Agents and Multiagent Systems (AAMAS '08), Estoril, Portugal, 2008.
  42. D. Becket and B. McBride, RDF/XML Syntax Specification (Revised), 2004, http://www.w3.org/TR/rdf-syntax-grammar/.
  43. T. Bray, J. Paoli, C. Sperberg-McQueen, E. Maler, and F. Yergeau, Extensible Markup Language (XML) 1.0 (Fifth Edition), 2008, http://www.w3.org/TR/xml/.
  44. M. Mancini and C. Pelachaud, “The FML-APML language,” in Proceedings of the Workshop on Functional Markup Language at the 7th International Conference on Autonomous Agents and Multiagent Systems (AAMAS '08), Estoril, Portugal, 2008.
  45. B. De Carolis, C. Pelachaud, I. Poggi, and M. Steedman, “APML, a markup language for believable behavior generation,” in Life-Like Characters, H. Prendinger and M. Ishizuka, Eds., pp. 65–86, Springer, New York, NY, USA, 2004. View at Google Scholar
  46. M. Schröder, M. Charfuelan, S. Pammi, and O. Türk, “The MARY TTS entry in the Blizzard challenge 2008,” in Proceedings of the Blizzard Challenge, Brisbane, Australia, 2008.
  47. M. Schröder, M. ter Maat, C. Pelachaud et al., SEMAINE deliverable D1b: 1st integrated system, 2008, http://semaine.sourceforge.net/SEMAINE-1.0/D1b%20First%20integrated%20system.pdf.
  48. D. M. Wolpert and J. R. Flanagan, “Motor prediction,” Current Biology, vol. 11, no. 18, pp. R729–R732, 2001. View at Google Scholar
  49. F. Eyben, M. Wöllmer, and B. Schuller, “OpenEAR—introducing the Munich open-source emotion and affect recognition toolkit,” in Proceedings of the Affective Computing and Intelligent Interaction, IEEE, Amsterdam, The Netherlands, 2009.
  50. M. Wöllmer, F. Eyben, S. Reiter et al., “Abandoning emotion classes—towards continuous emotion recognition with modelling of long-range dependencies,” in Proceedings of the 9th Annual Conference of the International Speech Communication Association (Interspeech '08), Brisbane, Australia, 2008.
  51. E. Douglas-Cowie, R. Cowie, I. Sneddon et al., “The HUMAINE database: addressing the collection and annotation of naturalistic and induced emotional data,” in Proceedings of the 2nd International Conference on Affective Computing and Intelligent Interaction (ACII '07), vol. 4738 of Lecture Notes in Computer Science, pp. 488–500, Lisbon, Portugal, September 2007. View at Publisher · View at Google Scholar
  52. D. B. Paul and J. M. Baker, “The design for the wall street journal-based CSR corpus,” in Proceedings of the Workshop on Speech and Natural Language, pp. 357–362, Association for Computational Linguistics, Harriman, NY, USA, 1992. View at Publisher · View at Google Scholar
  53. J. Carletta, “Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus,” Language Resources and Evaluation, vol. 41, no. 2, pp. 181–190, 2007. View at Publisher · View at Google Scholar
  54. B. Schuller, R. Müeller, B. Höernler, A. Höethker, H. Konosu, and G. Rigoll, “Audiovisual recognition of spontaneous interest within conversations,” in Proceedings of the 9th International Conference on Multimodal Interfaces, pp. 30–37, ACM, Nagoya, Japan, 2007. View at Publisher · View at Google Scholar
  55. E. Douglas-Cowie, R. Cowie, C. Cox, N. Amir, and D. Heylen, “The sensitive artificial listener: an induction technique for generating emotionally coloured conversation,” in Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC '08), pp. 1–4, Marrakech, Morocco, May 2008.
  56. J. Ostermann, “Face animation in MPEG-4,” in MPEG-4 Facial Animation: The Standard, Implementation and Applications, I. S. Pandzic and R. Forchheimer, Eds., pp. 17–55, John Wiley & Sons, London, UK, 2002. View at Google Scholar
  57. “SEMAINE sourceforge page,” http://sourceforge.net/projects/semaine/.
  58. “SEMAINE-1.0 wiki documentation,” http://semaine.opendfki.de/wiki/SEMAINE-1.0.
  59. CMLabs, “Psyclone,” 2007, http://www.mindmakers.org/projects/Psyclone.
  60. R. Niewiadomski, E. Bevacqua, M. Mancini, and C. Pelachaud, “Greta: an interactive expressive ECA system,” in Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, vol. 2, pp. 1399–1400, 2009.
  61. A. Heloir and M. Kipp, “EMBR—a realtime animation engine for interactive embodied agents,” in Proceedings of the 9th International Conference on Intelligent Virtual Agents (IVA '09), pp. 393–404, Springer, Amsterdam, The Netherlands, 2009.
  62. “GNU Lesser General Public License, version 3,” http://www.gnu.org/licenses/lgpl.html.
  63. “GNU General Public License, version 3,” http://www.gnu.org/licenses/gpl-3.0.html.