Ergonomic design of a virtual proctor system with reliable face recognition and tracking

Zhang, Z., Zhang, M., Chang, Y. & Esche, S. K.
Submitted for publication in International Journal of Human-Computer Studies, 2017.

Abstract

The function of proctors is to detect academic improprieties during examinations and laboratory experiments. Virtual proctor (VP) systems that replace continuous human supervision with video surveillance are starting to become popular in distance education, despite their most common shortcomings, which include their vulnerability to pose translations of the learners and their de-pendency on the illumination conditions of the environment. As a result, virtual proctor systems impose extra constraints on the movements of the learners being monitored, and thus they may feel uncomfortable when being subjected virtual proctoring. At present, in the design of VP systems, sufficient consideration to the human factors and ergonomics is typically not given.
To overcome these shortcomings, a video-based VP system with a reliable two-stage face recog-nition and tracking method is proposed here. First, the face region is detected and cropped out from the video frames by a combination of eye, mouth and face detection. After that, in order to render the usage of the VP system more comfortable to the learners being monitored, a modified face recognition method based on a proposed improved real-time stereo matching algorithm is employed to track the learners’ movements. The VP system is capable of identifying a limited set of pre-defined suspicious behaviors that may represent cheating. In order to evaluate the efficiency of the proposed methods, two benchmark analyses with respect to the effect of pose translations and varying illumination conditions on the effectiveness of the VP system are presented. It is asserted that the proposed VP system is simpler to use compared with prior solutions and realizes an er-gonomic design that makes the learners being monitored more comfortable.