9/16 Effects of Modality – Talia Feinberg

In this research article, a system was developed in order to create a natural human-computer interaction that integrates many of the human capabilities such as face, gesture, and speech. The goal of this platform is to provide a vivid and multi-modal system for the users that creates a more natural humanistic environment without needing extra equipment. This idea has been developed and used before in different scenarios, mostly regarding learning. It is called a Cognitive Immersive Room (CIR). The system has three large projectors that pick up on human behavior such as recognizing both static and dynamic motion patterns and poses. 

Systems such as this will likely be more frequently used in the future given the new nature of online learning in the post-pandemic world. Therefore, it is important that people continue to study and further develop this type of integrative technology. In this particular machine, there is a three level hierarchy ranging from the most basic recognition of human behavior to actually integrating it and putting out output regarding the new understanding of personalized human behavior.  This system “takes raw visual and audio signals as input and generates a proper response to interact with the user”. This new type of human-computer interaction could be groundbreaking for learning and working, allowing people to continue to do such activities from home while creating a much more vivid and realistic environment. 


3 thoughts on “9/16 Effects of Modality – Talia Feinberg

  1. It’s interesting to see how much technology is advancing in our lifetime. Reading your discussion, you point out this system that works to create human-computer interaction, integrating human capabilities like the face, gesture, and speech. This type of technology would’ve been interesting to research during the pandemic when students were interacting with technology everyday for class. While working remotely feels isolating in one’s room, technology like this has the potential to replicate what feels like an office or classroom.

  2. This is such an interesting system that you described, and can be so helpful in so many different fields, it almost reminds me, in the smallest of ways, of simulation technology, ones you would see in Disney World or even the glasses you can buy that take you to a different place from your living room. But for the population as a whole this technology could be helpful for those unable to be in a classroom setting, whether it is physical or emotional, as well as other things that hinder the experience of being in an everyday environment. However, I do find that there may be consequences to that, such as increasing the amount of laziness of the individual as they wouldn’t even need to stand up to be in a classroom or office. It can also create a serious sense of isolation, and from what we have learned with Covid-19, it can be detrimental to the mental health of many.

  3. I like the point you make about how convenient this type of technology will be as things continue to be held remotely. As you mentioned, CIR would have been extremely helpful during the pandemic as students transitioned to taking classes online. For many students it was quite difficult to be so isolated and many craved social interaction from their classmates. This type of technology could give them just that. My only doubt about this type of technology is that it could create a divide between those students who can afford this technology and those who do not have access to it. It can put those students who are not able to use CIR at a disadvantage.

Leave a Reply

Your email address will not be published. Required fields are marked *