Digitizing the Visual Records: AAS Plays Metadatagames

Last week, about twenty AAS catalogers, research fellows, curators, and other staff members gathered to discuss the challenges that come with transforming the visual code of an image into a written code. The creation of metadata in the form of indexing images is an inexact science, and it is one challenge that faces us as we, like many similar institutions, look to make our graphic arts collections as searchable and accessible as possible. In preparation for our conversation, we read Wanda Klenczon and Pawel Rygiel’s “Librarian Cornered by Images, or How to Index Visual Resources,” detailing the difficulties that come with assigning “high quality semantic metadata to the non-textual materials.” Though the development of automated image recognition software is underway, Klenczon and Rygiel explain, “there are layers of meaning that can be indexed only with human knowledge, experience, and intuition. Properties of an image such as shape, texture, and color contribute to our understanding of an image, but do not define it. Text-based search techniques remain the most efficient and accurate methods for image retrieval” (43). Our  graphic art catalogers shared their own experiences of creating this metadata, and we discussed how  we think about the 94,000 visual objects represented in the AAS’s online digital image archive, GIGI. As detailed in Klenzon and Rygiel’s articles, crowdsourcing or social tagging present considerable disadvantages, such as inconsistency and subjective expression, while at the same time we have institutional limits to how much indexing we can do ourselves.

We then turned our attention to one attempt to address the disadvantages of social tagging while still taking advantage of the many people willing to assist in the herculean task of providing metadata to the vast digitized, but not searchable, visual record. Recently developed at Dartmouth College with the support of NEH and ACLS, Metadatagames is a digital gaming platform that entices players to contribute metadata while playing games, either against a clock or against other anonymous participants on the site. The goal of Metadatagames is to refine the crowd’s tagging efforts by testing how many others would associate the same word or phrase with a given image. To simulate the games one plays in Metadatagames, we turned online games into group activities and then discussed our findings.

First, we looked at an image that is near and dear to us all at AAS:

Image 1 AntiquarianIn silence, we each wrote down five words or short phrases we would use identify this image, and then we compared our results.  Our tags varied considerably, and we quickly realized how many of our tags were subtle acts of interpretations, rather than just descriptions. Our resident experts in nineteenth-century American graphic arts added valuable context that made many of us rethink some of our tags. For example, in reaction to many people’s use of the word “man” to describe the image, one participant asked if we can know that the figure depicted is in fact male. One of our curators responded that because the painting dates to the 1830s and that is too early for women to wear bloomers, we can be pretty sure that the figure is male. We discussed how such knowledge is specialized and would not necessarily be present in an unmitigated social tagging experiment.

We then tried another adopted version of Metadatagames. Half of us left the room, and the remaining half looked at this image:

Image 2 Bird

Those of us in the room had to come up with one word or short phrase to describe this image. We then asked the other half of the room to come back into the room, and we showed them these six images:

Image 3 Many Birds

Based on one descriptor, “birds of prey,” the group that had been out of the room had to determine which image we were describing. After much debate among themselves, they chose correctly: number one. We then talked about how it took an expert curator, who was part of the describing group, to convince the group to use “birds of prey” and not just “birds,” the tag others suggested. Had we just chosen “birds,” the group that had been out of the room would have been lost: all six images either have birds in them, or, as is the case in #3, have a “bird’s eye view.” We then discussed the need to be specific when tagging and searching images. In these ludic activities, we came to see the ways in which tagging is an act of representation, and how it grows much more precise as more people collaborate to generate the tags.


List of images above:

Image 1: The Antiquarian. Lithographed Senefelder Lithorgraphy Company. c. 1830.

Image 2: Goshawk. Drawn from nature by J.J. Audubon.  Lithographed and printed by J.T. Bowen. Between 1840 and 1844.

Image 3:

1.) Goshawk. Drawn from nature by J.J. Audubon.  Lithographed and printed by J.T. Bowen. Between 1840 and 1844.

2.) My Little Favorite. Lithographed and published by Currier and Ives. Between 1857 and 1872.

3.) New Bedford, Mass. Lithographed by J.F.A. Cole. Between 1857 and 1860.

4.) The young mother. Lithographed and published by Currier and Ives. Between 1857 and 1872.

5.) [Woman with birds]. H. Thomas 64 [signed on stone]. Published by Kimmel & Forster. 1864 or 1865.

6.) Buff-breasted sandpiper Tringa rufescens Vieill. Chromolithography by J. Bien. 1860.

Published by

Molly O'Hagan Hardy

Molly O’Hagan Hardy is AAS Director of Digital and Book History Initiatives. She shares news on digitization and cataloging efforts at AAS, coverage of digital humanities projects using AAS materials, and ideas for such projects. Stay current with all things DH at AAS by checking out the “Digital AAS” section of our website.

One thought on “Digitizing the Visual Records: AAS Plays Metadatagames”

Leave a Reply

Your email address will not be published. Required fields are marked *