Readability formulas part 1: what are they really measuring?

We get asked about readability formulas all the time. And we get it, people want tools to help them know if their writing is appropriate for their audience. It might feel like a simple question, but there’s not a simple answer.

While readability scoring formulas might feel like a concrete way to quickly assess content, they can also give an incomplete picture of the usability and accessibility of a text.

This blog is part one of two exploring readability formulas. Here, we’ll dig into what readability formulas are and what they actually measure. In part 2, we’ll talk about how to use them effectively in your work.

What are readability formulas?

Readability formulas are mathematical tools used to measure the complexity of a text.

There are many readability formulas, both digital and hand scored. The formulas count specific text features and use that data to provide a score. Scores are often correlated to grade levels, so you might get a result that says “4th grade” or “8.2.”

Many digital tools combine formulas. For example, ReadablePro uses 17 different readability formulas to evaluate texts. The Automatic Readability Checker gives an average score from 7 formulas. Microsoft Word uses two formulas: Flesch Reading Ease and Flesch-Kincaid Grade Level.

What do readability scores mean?

There are generally two types of scores you get from readability tests.

Some tests produce a number that doesn’t relate to anything else. For example, the Flesch Reading Ease score gives you a number between 0 and 100. Higher scores are considered easier to read. These numbers do connect with a specific grade level.

Other tests give the text a grade level. What does it mean for something to score at a 3rd grade level? The researchers who designed readability formulas used textbooks and other educational materials to assign grade levels.

A 3rd grade level means the text shares features with texts written for 3rd graders. It doesn't tell you anything about your reader's life experiences or familiarity with the content, which are important for reading comprehension.

What do you notice about this text written at a 3rd grade level?

Do you see the crunch? I love the smell of purple at lunchtime. There are so many bunnies in my shoes. Let's get a snack! We can play soup in the ocean and watch penguins juggle. Yesterday Tomas and Amina ate the new aquarium.

It doesn’t make any sense! Yet none of the readability formulas I used pointed that out. If a program can score a text like this without identifying the lack of cohesion and logic, what is it scoring?

What do readability formulas measure?

When you use a digital readability formula, the computer scans the text and counts things like:

• Sentences per paragraph

• Words per sentence

• Syllables per word

• Difficult words

• Complex grammar

Most formulas only measure one or two elements, like sentence length and syllables per word. "Difficult words" come from lists created by whoever made the formula. These can look very different from formula to formula.

Given that each method is a bit different, it's not uncommon for the same text to be scored at different levels.

Let’s use this post as an example. I used four programs to generate a grade level score for everything before this paragraph. I got four different results, ranging from 7th grade to 10th grade. That seems like a pretty big jump!

When I add periods to the list above, the scores drop 1 to 4 grade levels. Why? Because adding periods reduces the average sentence length. Fewer words per sentence produces a lower score. And the change in levels is not consistent across formulas.

Without periods at the end of the list, this text scores 8.2 in Microsoft Word.

With periods at the end of the list, the text scores 6.2 in Microsoft Word.

What can we do with readability formulas?

Readability formulas can help us understand our writing better. We might find patterns of long sentences or complex vocabulary that can make writing more difficult to understand. Readability formulas and grammar checkers can be a tool to help you in your own writing process, but they won’t tell you what works for your audience.

In order to really meet our audiences’ needs, we need to consider their perspective and knowledge. The organization, layout, and design of written communication is just as important as the words we’re using.

Check out part 2 of this series, to learn how we can use readability formulas in our work.

The Lab Report is Clear Language Lab’s blog and includes posts on various topics related to clear, effective communication. Questions or errors you want to report? Contact Tiara Whitlock, Program Manager, at tiara@litworks.org.

© 2024 Literacy Works