2017 Equation Editor User Study

Executive Summary

The user study consisted of a usability test of the Accessible Equation Editor conducted with 18 middle school, high school, and college students who are blind. Testing took place during the 77th National Federation of the Blind annual convention held July 10 – 15, 2017 in Orlando FL.


The results indicated that many of the issues identified during the usability test conducted at the convention in July 2016 have been resolved, including:.

  • Voicing of the math as students enter it on the braille display.
  • Allowing students to read the tutorials using only braille, i.e., without the screen reader.
  • Clarifying the Equation Editor’s entry area.
  • Fixing bugs.

As identified in the 2017 study, there are still some issues that need to be addressed. These are categorized below as Equation Editor Issues, Voicing Issues, and Tutorial Issues.

Future content will include:

  • Technical resources
  • Testing tools
  • Additional external resources

Need assistance?

As identified in the 2017 study, there are still some issues that need to be addressed. These are categorized below as Equation Editor Issues, Voicing Issues, and Tutorial Issues.

Equation Editor Issues

Nemeth symbol palettes. Students who were shown the palettes appreciated having the Nemeth symbols available but there was some concern that they may not know which category contains some of the symbols.

Recommendations:

  • Modify the quick bar at the top of the palettes to provide the symbols that are needed for a given expression.
  • Provide a search function.
  • Provide the practice expressions in Nemeth to help students learn the symbols.

Erasing content. Students were divided in their expectations and preferences for using backspace to erase content. Many were confused by the current behavior of using backspace to navigate into the expression and to erase. These students expected to erase either one character or one braille cell at a time. Other students, however, understood and appreciated using backspace for navigating and erasing, and they liked that the context of fractions and mixed fractions was maintained until it too was erased.

Recommendations:

  • Conduct a competitive analysis of other equation editors to determine how content is erased (MathHere, Learnosity).
  • Conduct additional research with students (including students with cognitive disabilities) to further define expectations for backspace (including whether the context of fractions should be left intact), and determine whether the backspace issue can be addressed through tutorials and practice.
  • From an architecture/Math ML standpoint, explore the possibility of erasing content character by character.

Space. Most students expected spaces they entered in braille to be represented on the braille display. In addition, some students were confused about when to enter a space, for example, before signs of inequality and equals, but not after.

Recommendations:

  • Conduct a competitive analysis to determine how spaces are handled in other products.
  • From a technical standpoint, explore how to represent spaces on the braille display (i.e., Sam has ideas!).
Voicing Issues
  • Speech output. The voicing of the math as it is entered was an enhancement to the Accessible Equation Editor, and it paid off by alerting students to errors, including some who said they would do their work without speech.
    • In the instructions, encourage students to work with speech when learning the Equation Editor so they can determine whether or not speech output will be valuable when taking high stakes assessments.
    • Implement verbosity settings so students can select the amount of speech output that works for them.
    • Conduct research with students who are deaf and blind to determine if braille-only output conveys enough information for them to use the equation editor efficiently.
  • Spoken expressions. Even though students read the practice expressions with the screen reader (and some additionally followed along on the braille display), all students referred to the embossed braille of the expressions because when they got into the Equation Editor, they couldn’t remember what they were supposed to enter.

    • Recommendations:

    • Make it very easy to navigate between the practice expression and the Equation Editor by making the expression a heading and maintaining focus in the partially entered expression.
    • Consider additional “external supports” students may need to enter the expressions or work the problems, such as embossed braille.
  • Voicing of multi-cell Nemeth symbols. Most students indicated that, for them, the voicing of multi-cell Nemeth symbols was clear the way it is (e.g., “decimal-equals” or “exponent-decimal-degrees”), but they could see that it might be confusing to other students, especially those who are new to Nemeth. When asked about announcing the dot patterns of the initial cells instead of its symbol, most students agreed that this might be helpful.

    • Recommendations:

    • Explore the possibility of announcing the dot pattern of each braille cell as it is entered, and then announcing the symbol when a braille cell is entered that completes the symbol.
    • Conduct user research with this implementation to make sure students understand it.
  • Read Selection. When using the Read Selection command, students expected to be able to select the part of the expression they wanted to read.

    • Recommendations:

    • Implement cursor routing so students can select parts of the expression.
  • Screen reader preferences. Some students didn’t like the way fractions were read in the tutorial and exercises. A long-term solution would be to consider implementing verbosity settings so students can choose how fractions and other symbols/expressions are read.

    • Recommendations:

    • Explore possibility of copying the expression into the EE so we can control verbosity. Long-term look at controlling verbosity of Alt text.
    Tutorial Issues
    • Dual purpose. Most of the tutorials introduce 2 concepts – an Equation Editor concept (e.g., turn speech on/off, read expression, read selection, erase content) and a problem type. These should be separated so that each tutorial introduces a single concept with enough practice for students to become proficient with that concept.
    • Focus on palettes. Most students needed help remembering Nemeth symbols. Those who were shown the palettes very much appreciated having the Nemeth available. The next revision of the Equation Editor should focus on the palettes by writing a tutorial that covers palette navigation and symbol selection.
    • Remembering expressions. As mentioned above, students were very dependent on the embossed braille when entering the practice expressions. In addition to continuing to make this available, it will be important in high stakes test environments for students to be comfortable going into and out of the Equation Editor’s entry area and easily navigating to the problems, while maintaining focus in the equation they are entering.
    • Dot 5, Dot 8, and closing symbols. Many students didn’t understand the purpose of the Dot 5 to indicate return to baseline after an exponent, nor the purpose of the Dot 8 that shows the last character entered, including grouped characters/symbols. Some additionally misinterpreted the Dot 8, saying it looked like an “x” or a subscript. In addition, many students were confused because closing symbols, such as end root and end fraction are prepopulated on the braille display, so they didn’t realize or didn’t remember from the tutorial that they needed to enter them as they normally would, e.g., on a Perkins brailler. These need to be more thoroughly covered in their respective tutorials, with sufficient practice for students to get comfortable with the concepts.