An interaction-embedded HMM framework for human behavior understanding: With nursing environments as examples

Chin De Liu, Yi-Nung Chung, Pau Choo Chung

Research output: Contribution to journalArticle

33 Citations (Scopus)

Abstract

This paper presents an interaction-embedded hidden Markov model (IE-HMM) framework for automatically detecting and classifying individual human behaviors and group interactions. The proposed framework comprises a switch control (SC) module, an individual duration HMM (IDHMM) module, and an interaction-coupled duration HMM (ICDHMM) module. By analyzing the relative distances between the various participants in each scene, and monitoring the duration for which these distances are maintained, the SC module assigns each participant to an individual behavior unit (comprising a single participant) or an interaction behavior unit (comprising two or more participants). The individual behavior units are passed to the IDHMM module, which classifies the corresponding human behavior in accordance with the pose, motion, and duration information using duration HMM (DHMM). Similarly, the interaction behavior units are dispatched to the ICDHMM module, where the corresponding interaction mode is classified using an integrated scheme comprising multiple coupled-duration HMM (CDHMM), in which each state has an embedded coupled HMM (CHMM). The validity of the IE-HMM framework is confirmed by analyzing the human actions and interactions observed in a nursing home environment. The results confirm that the atomic behavior unit concept embedded in the SC module enables the IE-HMM framework to recognize multiple concurrent actions and interactions within a single scene. Overall, it is shown that the proposed framework has a recognition performance of 100% when applied to the analysis of individual human actions and 95% when applied to that of group interactions.

Original languageEnglish
Article number5482142
Pages (from-to)1236-1246
Number of pages11
JournalIEEE Transactions on Information Technology in Biomedicine
Volume14
Issue number5
DOIs
Publication statusPublished - 2010 Sep 1

Fingerprint

Nursing
Hidden Markov models
Switches
Monitoring
Nursing Homes

All Science Journal Classification (ASJC) codes

  • Biotechnology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

@article{cfa69491a64946fb99585cc97f1d8b09,
title = "An interaction-embedded HMM framework for human behavior understanding: With nursing environments as examples",
abstract = "This paper presents an interaction-embedded hidden Markov model (IE-HMM) framework for automatically detecting and classifying individual human behaviors and group interactions. The proposed framework comprises a switch control (SC) module, an individual duration HMM (IDHMM) module, and an interaction-coupled duration HMM (ICDHMM) module. By analyzing the relative distances between the various participants in each scene, and monitoring the duration for which these distances are maintained, the SC module assigns each participant to an individual behavior unit (comprising a single participant) or an interaction behavior unit (comprising two or more participants). The individual behavior units are passed to the IDHMM module, which classifies the corresponding human behavior in accordance with the pose, motion, and duration information using duration HMM (DHMM). Similarly, the interaction behavior units are dispatched to the ICDHMM module, where the corresponding interaction mode is classified using an integrated scheme comprising multiple coupled-duration HMM (CDHMM), in which each state has an embedded coupled HMM (CHMM). The validity of the IE-HMM framework is confirmed by analyzing the human actions and interactions observed in a nursing home environment. The results confirm that the atomic behavior unit concept embedded in the SC module enables the IE-HMM framework to recognize multiple concurrent actions and interactions within a single scene. Overall, it is shown that the proposed framework has a recognition performance of 100{\%} when applied to the analysis of individual human actions and 95{\%} when applied to that of group interactions.",
author = "Liu, {Chin De} and Yi-Nung Chung and Chung, {Pau Choo}",
year = "2010",
month = "9",
day = "1",
doi = "10.1109/TITB.2010.2052061",
language = "English",
volume = "14",
pages = "1236--1246",
journal = "IEEE Journal of Biomedical and Health Informatics",
issn = "2168-2194",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "5",

}

An interaction-embedded HMM framework for human behavior understanding : With nursing environments as examples. / Liu, Chin De; Chung, Yi-Nung; Chung, Pau Choo.

In: IEEE Transactions on Information Technology in Biomedicine, Vol. 14, No. 5, 5482142, 01.09.2010, p. 1236-1246.

Research output: Contribution to journalArticle

TY - JOUR

T1 - An interaction-embedded HMM framework for human behavior understanding

T2 - With nursing environments as examples

AU - Liu, Chin De

AU - Chung, Yi-Nung

AU - Chung, Pau Choo

PY - 2010/9/1

Y1 - 2010/9/1

N2 - This paper presents an interaction-embedded hidden Markov model (IE-HMM) framework for automatically detecting and classifying individual human behaviors and group interactions. The proposed framework comprises a switch control (SC) module, an individual duration HMM (IDHMM) module, and an interaction-coupled duration HMM (ICDHMM) module. By analyzing the relative distances between the various participants in each scene, and monitoring the duration for which these distances are maintained, the SC module assigns each participant to an individual behavior unit (comprising a single participant) or an interaction behavior unit (comprising two or more participants). The individual behavior units are passed to the IDHMM module, which classifies the corresponding human behavior in accordance with the pose, motion, and duration information using duration HMM (DHMM). Similarly, the interaction behavior units are dispatched to the ICDHMM module, where the corresponding interaction mode is classified using an integrated scheme comprising multiple coupled-duration HMM (CDHMM), in which each state has an embedded coupled HMM (CHMM). The validity of the IE-HMM framework is confirmed by analyzing the human actions and interactions observed in a nursing home environment. The results confirm that the atomic behavior unit concept embedded in the SC module enables the IE-HMM framework to recognize multiple concurrent actions and interactions within a single scene. Overall, it is shown that the proposed framework has a recognition performance of 100% when applied to the analysis of individual human actions and 95% when applied to that of group interactions.

AB - This paper presents an interaction-embedded hidden Markov model (IE-HMM) framework for automatically detecting and classifying individual human behaviors and group interactions. The proposed framework comprises a switch control (SC) module, an individual duration HMM (IDHMM) module, and an interaction-coupled duration HMM (ICDHMM) module. By analyzing the relative distances between the various participants in each scene, and monitoring the duration for which these distances are maintained, the SC module assigns each participant to an individual behavior unit (comprising a single participant) or an interaction behavior unit (comprising two or more participants). The individual behavior units are passed to the IDHMM module, which classifies the corresponding human behavior in accordance with the pose, motion, and duration information using duration HMM (DHMM). Similarly, the interaction behavior units are dispatched to the ICDHMM module, where the corresponding interaction mode is classified using an integrated scheme comprising multiple coupled-duration HMM (CDHMM), in which each state has an embedded coupled HMM (CHMM). The validity of the IE-HMM framework is confirmed by analyzing the human actions and interactions observed in a nursing home environment. The results confirm that the atomic behavior unit concept embedded in the SC module enables the IE-HMM framework to recognize multiple concurrent actions and interactions within a single scene. Overall, it is shown that the proposed framework has a recognition performance of 100% when applied to the analysis of individual human actions and 95% when applied to that of group interactions.

UR - http://www.scopus.com/inward/record.url?scp=77956357907&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956357907&partnerID=8YFLogxK

U2 - 10.1109/TITB.2010.2052061

DO - 10.1109/TITB.2010.2052061

M3 - Article

C2 - 20529751

AN - SCOPUS:77956357907

VL - 14

SP - 1236

EP - 1246

JO - IEEE Journal of Biomedical and Health Informatics

JF - IEEE Journal of Biomedical and Health Informatics

SN - 2168-2194

IS - 5

M1 - 5482142

ER -