2017 TestNav User Study

TEXT PASSAGE. In this set of items, students read a story and then answered questions. In the first two items, the story was presented on the left side of the page with the questions on the right; in the third item, the story occupied the main part of the page with the questions beneath it; and in the fourth item, the story was contained in an “Exhibits” link. In addition, the story in Item 1 was in its own scrollable window and in Item 2, the story and questions were in a single window.

Even though all of the students initially expected the questions to be beneath the passage, the students who didn’t use screen magnification liked the questions on the right because they were immediately visible. The student who used ZoomText never noticed the questions on the right and when they were pointed out, they were difficult to reach because they required a lot of horizontal scrolling. For the students using browser zoom, the page reflowed, forcing the questions beneath the passage where these students expected them to be.

None of the students noticed or liked the passage when it was contained in the Exhibits link.

The students using screen magnification (ZoomText or browser zoom) found reading the story in Item 1 difficult because they had to scroll the passage window as far as they could and then scroll the main window to read the bottom of the passage.

The other difference among these items was the length of the lines in the story. In Item 3, in which the passage occupied the full width of the computer screen, the lines of text were longer than in the previous questions. Because of the increased line length, students had more difficulty tracking and reading, especially with screen magnification.

MATCH QUESTIONS. There were two items in which students had to match two answers to a math expression. One required a drag and drop interaction and the other used checkboxes. The students using screen magnification had difficulty with the drag and drop item because the drag items and drop bays weren’t visible on the screen at the same time, plus they had to hold their mouse button down while the page scrolled enough to bring the drop bays into view. The match interaction also required students to be very precise when placing a dragger into a drop bay. Even though the drop bay changed color as the mouse got close to it, students had to pull the dragger all the way over the drop bay before they could drop it. It was confusing to them that they could not simply drop the dragger when the drop bay changed color.

USING EACH ANSWER ONCE.These items were similar to the match questions except that each answer in a group of answers was to be selected only once. The first item was a drag and drop interaction, the second used radio buttons, the third used combo boxes, and the fourth was presented as a table.

The students using screen magnification had difficulty with the drag and drop and table interaction due to horizontal scrolling or reflow issues. All students performed well with the radio buttons and combo boxes. Students asked why the combo boxes didn’t eliminate answer choices that had already been used like the radio button interaction. Many students preferred to see the answers eliminated once they were used, but they liked the combo boxes because they were more compact and occupied less space on the screen. Using combo boxes requires less scrolling, especially with magnification and this was helpful to the students. However, students were bothered that their answer choices were not completely visible in the combo boxes after selection because the boxes weren’t wide enough to show the entire answer.

VIDEO CONTROLS. In this item, the controls for adjusting the speed of the video and turning closed captions on and off were contained in a “Settings” link. All students either chose Settings immediately or they adjusted to the location of these controls quickly. The only possible usability issue was that one student noted that the slowest speed is at the top of the list and the fastest speed is at the bottom. This was counterintuitive and even after noticing it, she still automatically went to the bottom of the speed options when she was trying to reduce the rate of speed.

COLOR FILTERS. Two items tested the use of color filters – one blue and one yellow. Most of the students said they didn’t need and wouldn’t use the filters, but one said the yellow filter made the screen warmer and not as bright. All of the students could easily move the filter but they struggled a bit to change the size of the filter. It is important to note that we did not have students in this study with photophobia, which is the primary use case for the filters. Students with photophobia often prefer reduced contrast and are used to using filters. One student noted that she uses filters on her phone, but she uses a pink filter, rather than the blue and yellow options that were available for this study.