INTRODUCTION
Usability testing is the method of measuring the ease of use and efficiency of a user interface (i.e., a web page, software, document or device). It involves conducting a controlled experiment where users perform certain tasks using the interface. Usability measures such as: task accuracy, time on task, perceived task difficulty, user satisfaction, and user response (verbal and non-verbal comments via the think aloud protocol) are recorded and evaluated to suggest recommendations for improvement.
In the early days of usability testing, recording these measures was tedious and involved lots of different equipment (i.e., scan converters, cameras, logging tools). Researchers were also limited as to how much onscreen and user information they could capture. Capturing user reaction required an external video recording device which was both expensive and required a special lab setup. Onscreen activity recorders were non-existent. Remote testing, (evaluating an interface from a location outside of the actual test) was unheard of because of the technological limitations. More importantly, these limitations shifted the researchers' focus from understanding the user's experience to worries of accurately logging the experimental measures. Results of the usability studies were also limited because researchers could only report their observations and not readily show video footage of participants' behavior. The process of recording and editing the videos to show this information required special skills and was very tedious, time consuming, and expensive.
Ergosoft introduced ErgoBrowser (2001) which brought some relief to researchers by exempting them from hand recording time on task, mouse clicks and number of pages browsed by the participants. However, ErgoBrowser was limited in the sense that it did not boast any over-the-network features like remote viewing and did not tap into the audio-visual and screen capture possibilities (Figure1).
Figure 1. ErgoBrowser's setup window.
TechSmith's Morae was introduced in 2004. It features an on-screen activity recorder that is capable of recording an entire session of a usability test. The session timeline can be divided into numerous fragments (tasks and/or comments) simply by placing markers on the timeline. Data such as time, mouse clicks, and total number of pages view can be extracted from any fragment of the recording session. Also, numerous fragments from different recording sessions can be easily combined into a highlight-video without the knowledge of video editing. Morae is also picture-in-picture (PIP) capable, so the users' expressions can be recorded simply and inexpensively with the use of a web camera.
Morae has three components to it: the Recorder, Remote Viewer, and Manager. The Recorder runs on the computer where the usability test is being conducted. The location-free Remote Viewer can connect to the Recorder via the IP address of the computer where the Recorder is running, or by simply connecting to the Recorder computer if they are both on the same network. The Manager is the post-experiment application that is used to place/edit markers, extract data, and create highlight videos from the recorded sessions.
Morae's robust features ensures ease of use, data reliability, but above all, the onscreen recorder and the PIP feature empowers the researcher to convey the results along with the actual clip of the user's onscreen activity. The current release, Version 1.3, is a big improvement over the previous versions. The following outlines some areas that could be further enhanced.
1. No editing in the Remote Viewer. When the Recorder and Remote Viewer are communicating, observers can flag the start and end of each task (using markers) as the user progresses and flag the comments that are task-relevant. This saves the researcher from going through the recording of the entire session again just to mark the tasks. The Remote Viewer, however, does not visually show the markers placed in real time and does not allow the observer to edit the markers. It is not always possible to accurately place the markers in real time and hence they need editing. Because of this, some researchers may choose not to place markers in real time at all.
2. Playback in the Manager. After a session is recorded, a Morae project is created in Manager and the recordings are imported (Figure 2). Manager is a very powerful program that can create, edit, and delete markers. It has an integrated video player to play the recordings and allows the researcher to add markers. These markers are displayed on the timeline. The playback controls of this integrated player are very limited. The play button does nothing but play; there is no Play/Pause toggle feature as seen in many video playback programs. Instead, to pause playback, the user has to move the mouse and click on the stop button. Also, the spacebar cannot be used to play or pause, which is a feature commonly used in other video playback software. Additionally, a researcher may need to "scan" through a recording which is easy to do using a fast forward or rewind button. Morae lacks both a fast forward and rewind button; both of these would be very helpful in streamlining the playback and marking process (Figure 3). Shortcut keys for these functions would also be beneficial.
Figure 2. Morae Manager in the Analyzer view shows the marker flags on the timeline.
Figure 3. Morae Manager playback controls.
3. Zooming the timeline. Just above the playback controls is the timeline. This new feature in version 1.3 allows the timeline to be zoomed which is helpful when selecting a specific time frame or flag (Figure 4). It can either be zoomed in by clicking on the zoom-in button (which is intuitive) or by dragging a "bar"just below the timeline (which is not so intuitive). One frustrating feature of the timeline is that it does not maintain its zoom setting when another task within the same recording is selected. The zoom is also not quantified (200% or 400%) or selectable and hence the zoom-in button needs to be clicked several times to attain the desired zoom level.
Figure 4. Morae Manager timeline showing the zoom feature.
4. Color Coding of Markers. Just above the timeline are the markers (Figure 2). The markers are default color coded: green for start task, red for end task, and purple for comments. On mouse-over, all of them turn green which defeats the purpose of having the coding or mouse-over effect when they can no longer be differentiated. The flags represent an anchor in the timeline but its placement cannot be changed by dragging it. It can only be moved by giving it a specific time in its edit window. The ability to Lock and Unlock Markers would be convenient.
5. Title Clip Window. The title clip window is an undockable element but when undocked and moved around it can sometimes get hidden behind the Windows Taskbar. Then, it becomes impossible to find unless the Taskbar is set to auto-hide. There is no "restore"button for the docked elements of the program. Even shutting down and restarting the program does not restore the undocked elements.
6. Creating Highlight Videos. With the Manager, the user has the ability to create video clips and place those clips on a storyboard to produce highlight videos. Title clips can also be added to the storyboards. When creating a title clip caution must be taken to ensure that the text used appears as expected. When the text size is large, it may appear adequate in the Title Clip Details window but appears cut off when the clip is placed on the storyboard. This is very frustrating because sometimes highlight videos have to be re-rendered if the bad title clip goes unnoticed during creation. Also, the text on the title clip appears highly aliased and visually unappealing. The storyboard offers a transition effect but offers no choices. More choices of transition effects would allow the creation of more interesting videos.
The process of creating the highlight clips is also very tedious. The clips need to be created using the segment markers but this seems redundant if the segment covers the entire task. For example, to create a highlight clip of an entire task for 10 participants, 40 markers (task markers + segment markers) have to be set before the video can be produced. If there are 10 clips to produce in a particular study it totals 400 marker placements. There is also a software delay when switching between the recordings of different participants within the study, which results in a longer time to place the redundant markers. With this in mind, the latest version of Morae offers an 'automatic' segment creation function (Figure 5) that creates segments based on markers that are already placed on the timeline. It would be useful if this wizard could be enhanced to create clips and place those clips on the storyboards without requiring the user to individually drag and drop each of them to the storyboard. Alternatively, Morae could introduce a clip creation feature that would create clips based on existing markers and store them in the clip bin (Figure 6). Here, multiple clips could be selected at one time so the individual dragging and dropping process is diminished.
Figure 5. Morae Manager showing segment creation wizard.
Dragging and dropping is a laborious process in Morae because multiple selections of segments from different recordings cannot be done, unlike many other windows-based programs that allow the "Control Left-Click" combination to perform multiple selection.
Figure 6. Morae Manager stores video clips in a clip bin from which the user can drag to the storyboard.
7. Calculating Time on Task. The Manager has two tabs: Presenter and Analyzer. The Analyzer tab offers options to gather the number of page views and time on task data (Figure 7). Time on task, however, is displayed in minutes, seconds, and milliseconds. It would be convenient to be able to choose the unit of measurement and for example, display the time in seconds only. In addition, the time on task and page view data is not accurate when websites with frames are used.
Figure 7. Morae Manager analyzer view showing time on task and number of pages viewed.
8. Connecting through a VPN. The Remote Viewer connects to the Recorder remotely, giving clients and researchers the opportunity to watch the user go through the tasks in real time. Morae, however, has difficulty connecting to the Recorder running inside a VPN. For such scenarios the Recorder and Viewer should have the ability to communicate via a direct connection if they are located in the same facility. This would allow the audio-visual transmission to the Viewer to be virtually bandwidth independent and time-lag free.
CONCLUSION
Overall, Morae 1.3 is a great tool to have for usability testing. Improvements to the process of producing highlight videos, playback control, zooming, setting flags, and Remote Viewer capability would improve its overall utility even more.
