To check out my podcast, click here!

To check out my proposed changes to Course Site, click here!

An Analysis of Course Site from the Perspective of Usability

Every student that attends Lehigh University will have to interact with Course Site at some point during their academic career. As explained by Lehigh’s Library and Technology Services (LTS) webpage, “Course Site is Lehigh University’s course management system,” which is “primarily used by faculty to provide online delivery of content to students and promote 21^st century learning through online collaboration.” (Course Site). According to LTS, the site “allows students to have constant access to course content, collaborate with peers, submit course work, as well as communicate with faculty and fellow students.” (Course Site). From the student perspective, Course Site is the online hub for all information pertaining to most, if not all, of one’s classes.

As with most sites, the user interface of Course Site has aspects that could be improved. In order to determine these specific aspects, a survey constructed in Qualtrics was sent to current and past Lehigh students via GroupMe and Facebook. In total, 36 people participated in the survey, with approximately equal distribution across year at Lehigh (freshman, sophomore, junior, senior, and 5^th year +) and colleges at Lehigh (Arts & Sciences, Business & Economics, Engineering, and Integrated Programs such as IDEAS, IBE, and CSB). If this study were to be replicated, it would be desirable to include professors, as well, in order to gain their perspectives.

The survey included blocks of questions from two different sources: The System Usability Scale, as found on usability.gov, (SUS), and the personal questionnaire of Thomas S. Tullis and Jacqueline N. Stetson, as presented at the UPA 2004 Presentation. (Tullis et al.). Participants were asked to rate their agreement with both blocks of questions on a Likert scale of 1 = Strongly Disagree to 7 = Strongly Agree.  Beyond the SUS and Tullis et al. questions, participants were asked to answer free response questions about their likes, dislikes, and suggestions for improvements to the site.

The survey questions point to students being relatively satisfied with Course Site. From the SUS scale, most answers were generally neutral. Participants specifically disagreed with the need of support from a technical person to be able to use Course Site, but only slightly disagreed with it being unnecessarily complex, cumbersome to use, and that they needed to learn a lot of things before being able to use the site. They agreed that they felt confident while using Course Site, but only slightly agreed that they liked to use it frequently, that it is easy to use, and that people would be able to learn how to use it quickly. Participants were neutral on the various functions of Course Site being well integrated, as well as there being too much inconsistency in Course Site.

The Tullis et al. scale produced similarly neutral results: on average, participants slightly disagreed with Course Site being visually appealing but were neutral about it being easy to move from one page to another and that individual pages are well designed. They slightly agreed with the overall organization of Course Site being easy to understand, that terminology used is clear, and that its content meets their expectations. Participants agreed that they were able to complete tasks on Course Site in a reasonable amount of time.

The open-ended response questions, however, received answers that were not as neutral: features that participants thought worked best ranged from “none of them” to “all” of them. Among other aspects, participants mostly noted that they like the ability to view, upload, and receive notifications about assignments, grades, and course materials, as well as the clear organization of classes and similar organization of all classes within the site; however, one participant noted that they think the user experience “honestly partially depends on the professor’s ability to use Coursesite and organize their pages.” Participants also stated that they disliked the way grades were structured, how to upload files, and trying to navigate through different pages. Additionally, they disliked discussion forums, the “old news” on the homepage of the site, and not knowing whether clicking on a file would cause it to download automatically or not. The likes, dislikes, praises, and complaints of the site were distributed across years at Lehigh as well as individual colleges; for example, some seniors strongly disliked the site, whereas some freshman liked it a lot, and vice versa. There were no noticeable differences in responses between members of different colleges.

From a technical standpoint, participants were relatively neutral, as previously discussed. From a social psychological perspective, the opportunity for comparison in class discussion forums has the ability to either improve or degrade one’s self-esteem. Cognitively, the abundance of information on individual Course Site pages can lead to attentional overload and subsequent missing of information; additionally, if students associate the site with academic stress or academic success, they may create a mental model of Course Site which is inherently negative or positive, based on the experiences they associate with it. From a behavioral standpoint, the presence of grades and assignments on Course Site may lead to obsessive checking of grades, and the formation of an unhealthy relationship with the site.

Ultimately, participants in the study were neither enamored with or disgusted by Course Site. However, the site has the potential to create either positive or negative user experiences based on its nature, the type of information it provides to the user and how the user interacts with and interprets that information.

References

Course Site. (2018). Retrieved from https://lts.lehigh.edu/services/course-site

System Usability Scale (SUS). (2013, September 06). Retrieved from

      https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html

Tullis T. S., & Stetson, J. N. (2004). A Comparison of Questionnaires for Assessing Website

Usability. Usability Professionals Association (UPA) Conference.