How (not) to design a checklist

Thursday, 28 June 2012

This guest post by David Baxter originally appeared on the One2Many blog.

Last week the Royal Ontario Museum (ROM) and some other local organizations coordinated a Bioblitz in Rouge Park, an urban wilderness park in Toronto. In a sentence, a bioblitz is an event where expert biologists, naturalists, and volunteers all come together in an effort to survey all the different species in a natural area, usually constrained to a 24 hour period. I was appointed to be data manager/data enterer for this event, and I was all too happy to oblige. I was given species lists based on previous surveys, and while someone else set up the data entry system in a Google Doc spreadsheet, I tasked myself with designing checklists that people could take out in the field with them to mark off the species they saw.

I created a rough draft in MS Excel, met with the plant team leader to discuss the types of data we would like people to record in the field, and sent a draft out to the other taxon leaders. Further changes needed to be made as different identification methods are used for different taxa (e.g. botanists collect samples of tricky plants to bring back to the lab for identification, so they need to record collection numbers, while birders need to record whether they identified a bird by seeing it or by hearing its song).

The field checklist I ended up making for the plant team was a table containing columns for scientific name, common name, year last sighted, as well as three columns where field workers could check off species that were seen, initial that the ID was confirmed by an expert, and record a collection number if a specimen was taken. General information about the location and the team members was to be recorded at the top of the page. The final product looked like this:

Does it look okay? Maybe? Maybe not? From the get go the main concerns I heard were about the length of the checklist. With over 1000 plant names, the whole list was 28 pages long (14 sheets if printed double sided). I probably could have managed to make the list in portrait instead of landscape, but then the long names and data collection fields on the right would not have fitted on the page.

The form turned out to be rather unpopular. Usage ranged from adoption with difficulty…

… to outright rejection and resorting to taxon codes.

And I know first-hand how bad the form was. I went out in the field with one of the teams on the first day and I offered to record. Finding all the names as the botanists called them out was a nightmare.

The point of this whole story is that the reason why this checklist was a flop is because while I was designing it I didn’t realize that I was designing an information system. I guess that is “lesson zero”: that not all information systems are digital.

Here are some information system design pointers that would have resulted in a much better field checklist.

1. Beware that designers tend to design for themselves

In this case, its quite obvious that the form was designed for the comfort and benefit of the person compiling the results for data entry, with very little regard for the users in the field (remember that this checklist was 28 pages long).

2. Know your users

The form shown was arranged by taxonomic family, although one sorted by genus/scientific name was also available. In order to quickly determine the families for these 1000+ plant names I plunked them all into the Taxonomic Name Resolution Service and pasted in what was spat out.

It turned out, however, that nobody preferred the form sorted by family. This is largely because plant taxonomy is always in flux: plants jump back and forth between families, new families are created while old ones disappear, and the result is that different botanists have different sensibilities for plant families depending on the time(s) and place(s) that they learned their plant classification. The original species list I was given contained some out-of-date names, so by using TNRS results without modification I ended up making a checklist of old scientific names arranged into new families. Very frustrating for all involved.

3. Don’t blithely follow design principles

I’m taking an information architecture class this summer, so I’m excited to apply the principles I learn whenever I can. One of the things we learned about when talking about managing the user’s cognitive burden is “chartjunk”. Chartjunk is “extra ink on the page”, or extra stuff on your chart that doesn’t give any information. We were taught to minimize this to avoid confusing the user. One of the examples was to remove unnecessary lines from tables to make it easier to read.

Excited about applying this concept, I hid the grid on my tables. It made it look sleek, but in practice it ended up making it much more difficult to use. The columns were very wide because of a couple of long plant names, so wide in fact that it was hard to tell if you were checking off the correct box (as illustrated in the second image above, where the user drew a line across from the name to the checkbox). Later when I was entering the data I had to use a ruler to line up the names and the checkboxes, so this design really didn’t do favours for anyone.

Despite dealing with a poorly designed form, the blitz was a success for the botanists and the response to the event was generally positive. The botany team spotted over 500 different plant species in 24 hours, including 14 new ones that were not spotted in previous surveys. The botanists also were not shy to give me their feedback and suggestions, which I’ll use to design a much better field checklist for future events, but I think we could have skipped this awkward first time if I consulted with the intended users in the first place.