Usability Evaluation of a Virtual Educational Laboratory Platform

Chang, Y., Aziz, E.-S., Zhang, Z., Zhang, M. & Esche, S. K.
Computers in Education Journal, Vol. 7, No. 1, pp. 24-36, 2016.

Abstract

Taking advantage of videogame technology, game engine-based virtual laboratories are able to offer promising immersive and collaborative learning experiences. Research indicates that such virtual laboratories can be viable alternative forms for laboratory learning activities with special advantages in distance education applications. Various researchers also evaluated whether students learned target knowledge via virtual laboratory exercises. However, several questions emerged during these evaluations: Can students complete this new form of laboratory exercises, which they have not encountered before, in an effective way? Can they collaborate in the virtual world like in the real?

This article tries to answer these questions by assessing the students’ performances in two videogame-based virtual gear train laboratory exercises. Simple and planetary gear train scenarios were designed and implemented on the basis of Garry’s Mod, a sand-box 3D game utilizing the Source game engine. 94 undergraduate students taking a course on machine dynamics and mechanisms were assessed right after completing the lecture and homework of the gear design chapter. Most of these students were randomly divided into laboratory groups of 2 while the remaining students conducted the laboratory experiments alone. In order to simulate a remote learning scenario, the students in each group were physically separated into two rooms so that they could not communicate directly with each other but could do so only by text-chatting within the virtual laboratory environment. A teaching assistant was present in each room to help the students.

In order to evaluate the usability of this game engine-based laboratory, a data set containing the students’ videogame playing background and a game log, which tracks the students’ activities, were collected and analyzed. The result shows that all students were able to complete the laboratories regardless of their prior videogame playing experience. Also, it was discovered that from the students’ laboratory operation perspective, most students made mistakes before completing all tasks. From a collaboration perspective, most students in a group did not evenly distribute the tasks amongst them.