Take me to…

• Background behind tools like these in clinical speech research
• Other tools besides Praat that may have been just as good for the job
• Tutorial for using vas.praat
• The Praat source code for vas.praat

Speech intelligibility ratings

Many people with speech disorders find it difficult to be understood by others. In the clinical speech world, the term speech intelligibility refers to the ability of a typical listener to understand a person’s speech (Yorkston et al., 1996). This is often referred to in percentage form and provided with the context in which it was measured. For example, “Bob was 85% intelligible during his reading of a short passage” means that people who listened to Bob reading some sentences aloud were able to understand approximately 85% of what he said. That could change depending on what Bob’s doing (reading vs. having a conversation), who’s listening to him (people who know him, people who don’t), the environment (in a quiet vs. noisy environment), and many other factors.

There are a few ways we can gather speech intelligibility estimates. One way is transcription; listeners write down exactly what they heard, and their responses are compared to what was actually said. Another method is estimation; listeners rate along a scale with anchors (e.g., “low” to “high”) how intelligible a spoken utterance was to them. One type of scale is what’s called a Visual Analog Scale (VAS), which consists of a line with anchors, along which the listener makes a tick mark to indicate their response. This post is about using tools for estimation, but the next post in this series will discuss transcription.

#🛠 The search for the right tool {#tools}

In our lab, we’ve frequently presented listeners with printed forms that they mark manually as they listen to audio clips. A goal of mine this year was to come up with a system to do the same thing but through an automated, computerized system (I do love me some semi-automation). Automating this kind of task takes a big chunk out of the time required to conduct analysis, and reduces the risk of manual input error (e.g., me messing up and entering the wrong number into a spreadsheet when my eyes are going blurry from entering hundreds of other numbers). In looking into my options, I found several viable routes:

Demo

Note: There’s no audio in the clip, but it’s demonstrating what the process would be for two audio clips.

What vas.praat does

• Iterates through a directory of .wav files and plays them one at a time, with options to randomize the presentation and replay a subset of the files for reliability purposes
• Presents the listener with a visual analog scale (VAS)
• Allows the listener to click anywhere along the visual analog scale as they listen to the audio clip, and adjust their rating until they’re ready to move on
• Allows the listener to advance at their own pace, but not before the clip has finished playing, and not without making a mark on the line
• Prohibits the listener from advancing to the next audio clip if they haven’t marked the line, or if they try to advance before the audio is finished playing
• Logs the results to a .csv file for later analysis
• Logs the listener order and progress, allowing the listener to stop and pick up where they left off later (and the possibility to run other listeners in between)

Using the Praat demo window, the script essentially sets up an interface for the listener by defining the boundaries of the visual analog scale and “buttons”, and registers whenever the user clicks anywhere within those boundaries. If the click occurs along the line, the screen subtly updates and refreshes the mark. If the click occurs within the button to advance, the script checks whether that’s okay to do, and either advances or alerts the user as to why that can’t happen. It does all this through use use of a few core procedures, i.e., user-defined Praat functions which I’ll walk through in the next post in this series.

The code for the original core functions came during my search for any kind of Praat functionality for this kind of application. The code provided in José Joaquín Atria’s Praat scripting forum post about building a “scroll bar” proved to be the most important piece, which I was able to adapt for my purposes. The Praat forum is exceptionally helpful for finding useful code snippets in Praat and understanding more about how they work³.

In the next post, I’ll walk through the building blocks of how the VAS script works from a technical standpoint. For now, here’s what it’s like to run it.

Assumptions

For the script to run, the following criteria must be fulfilled:

• Audio files in .wav format are stored in one directory. All audio files will be played.
• The script, vas.praat, is stored in this same directory

User options

The form defined at the beginning of the script (and is what pops up when you run the script) contains a bunch of fields for the user to fill out if they wish.

The user options can be grouped into the following categories:

Session info [Listener ID, Session number]

• Here the user (researcher) can enter important information for the session at hand. Most important is the listener ID, and whether this is their first session (i.e., the first time setting up a playlist for this listener). If it’s not the first session, the researcher can indicate which session it is on the next line.

Experiment info [Output filename, anchor labels]

• Things like the name of the output file where the results are logged and the anchor text are likely to be constants for a given experiment.

Other experimental options [Reliability, Sentence repetition]

• The user can also decide whether to randomize the presentation files; if they don’t, they will be played in alphanumeric order (as they would appear in the folder where they are stored). If they’d like a subset of the files to be repeated, they can specify the percentage of files to repeat (the script will round down to the nearest integer). There is also the option to permit that each audio clip play twice. If unclicked, the script will still give the option for the user to repeat the file, but it won’t do this by default.

Sanity check [VAS output visible on screen]

• The last option I included mainly as a sanity check to myself. If checked, the script will show the value that the listener has selected according to where they clicked along the line. This is probably not something you’d actually want a listener to see, but can be helpful in absolutely being sure that the script is doing what you want, and can be cross-checked with the output files.

Once satisfied, the user clicks Okay and the next page pops up for the listener.

Instructions to listener

At this point, the researcher may wish to pass control off to the listener. This text can be modified in the script to suit whatever the instructions are you wish to convey. Once the listener presses “Begin”, the VAS screen will appear and the audio will begin playing.

VAS screen

The user won’t be able to advance until they have made a mark somewhere along the scale, and the audio has finished playing. When they are finished with the last file, a pop-up informs them they’re all done.

Output

The main output is vas_responses.csv (or whatever you’ve chosen to name the response file), and looks like this:

## 
## ── Column specification ────────────────────────────────────────────────────────
## cols(
##   filename = col_character(),
##   listener = col_double(),
##   session = col_double(),
##   order = col_double(),
##   percentage = col_double()
## )

“percentage” refers to the percentage along the line where the user clicked for that file. In this example it refers to “percent intelligible.”

Each time you run a new listener for the first time (i.e., first session), two other files are created (the following is assuming the listener ID is 01):

• listener_01_order.txt: logs the original order created during the randomization
• logfile_01.csv: logs the progress made by the listener

Multiple sessions for a listener

If the listener needs to leave before finishing all the audio files, you can just exit the script. The next time you run that particular listener, type their same ID into the form (make sure you label it the same way, i.e., 01 or 1, etc.), uncheck “First session” and type the accurate session number into the next field.

In this case, the script looks up their order and their progress, the two files that were generated during their first session. It looks up the last file they did in their logfile, finds it in the order file, and starts with the next file on the list in the order file.

⚠️ Important: At the present time, there’s no check built in to make sure you haven’t already run that listener. Here you just have to be diligent. Future versions will be diligent for you. It also doesn’t technically matter what you enter as a session number; all that really matters is that you have or haven’t clicked “First session”; the session number is just logged for your reference.

Outstanding additions I’d like to make but haven’t yet

• Should run from any permanent location
• Instructions should be more readily modifiable for user
• Should check whether listener ID/session combo has already been entered
• Progress bar/indicator/some output relating how many files are left to do
• Make as Praat plugin
• Other things I haven’t thought of yet? Quite possibly…

Hopefully somebody somewhere finds this useful. In any case, hopefully at least future me finds it more difficult to forget what I did as a result of this and the subsequent posts in this series. We have successfully used the script for a couple of listening experiments in our lab, and I’m pretty happy with how it’s turned out. Praat is clunky, sure, but I love it anyways, and I have to say it did just fine for the task at hand (without taking years off my life to figure out 🙌)