Process

Working with a statistical consultant, Prof. Grant Ritter of Brandeis University, and advised by the EAST Validation Working Group, the EAST team developed the following methodology for the study:

  • Provided a sample of 6,000 items for each of the 52 libraries for a total of 312,000 items to be sampled across the two cohorts
  • The sample was drawn from the data extracts the library had provided as part of the collection analysis
  • Developed a data collection tool that supported easy and efficient collection of data by workers [typically student workers] in the library
  • Provided documentation on how to check the library's sample against its local circulation system to eliminate items known to be checked out before items were checked on the shelves
  • Developed an easy to implement mechanism to provide a cursory check of the physical condition of the item as it was being handled by the worker
  • Provided documentation to the libraries to facilitate training of their local workers.

Working closely with the Validation Working Group, which had been appointed by the EAST Executive Committee, as well as OCLC’s  Sustainable Collection Services®, the vendor working on the EAST collection analysis, the EAST team completed preparation for the VSS by mid-February 2016 with the data collection taking place in early spring of 2016 for Cohort 1 libraries, and in the fall of 2017 for Cohort 2 libraries.

The Data Collection Tool for the Validation Sample Study

To support efficient data collection for the first sample validation study, the EAST Data Librarian, Sara Amato, worked closely with members of the Validation Working Group to design a web-based tool.  The tool supports download of the random lists of titles selected from the holdings records for each of the libraries participating in the VSS.  Prior to using the tool, librarians ran the randomized list of 6,000 titles against their current ILS to mark any items currently in circulation.  Accounting for checked out volumes from the beginning ensured only items which are expected to be found present will be downloaded to the tool.

To use the tool, a student or staff worker specifies the number of items they wish to check and those are loaded in location and call number order.  Using a tablet, laptop or workstation with an attached barcode, the worker then scans the first title in the list. If the item is present based on the barcode read, the tool automatically updates the status as present.  The worker is then asked to do a brief condition check of the item to determine whether its condition is poor, good/acceptable or excellent. Both documentation and a short video were developed by the EAST team to provide training to the workers to simplify this check.

Although access to wifi is required for the initial download of the items to be checked and to upload the list once the checking is complete, the actual barcode scanning and condition checking can be completed offline.  The tool is also designed to support multiple users simultaneously checking items in the stacks.

Since the workers were handling each item to perform the check, they were also asked to perform a check of the condition of the physical item based on a three point scale: 1 - Poor 2 - Average 3 - Excellent.  This data was entered into the data collection tool following the scanning of the barcode and required only minimal additional time on the part of the workers. A training video for the workers was made available to assist and to better ensure consistency across the libraries.   

Because the data collection tool reported back results immediatly as groups of items checks were submitted, the EAST team had up-to-date access to the availability metrics across the EAST libraries.  Upon completion of the data collection, results [anonymized except for reporting to the local library] were shared across the EAST membership.

How the Validation Data Collection Tool Works

The data collection tool allows workers to check the shelves to determine whether or not items expected to be available on shelf are present.  Items which had been identified as being checked out or missing were not presented by the tool for shelf validation. For items that were not found on the shelf  but were expected to be present, the worker can perform a manual check against the catlaog using the tool. Whenever possible, the data downloaded to the tool included barcodes so that the workers could use a barcode reader attached to the laptop or tablet and simply scan the item's barcode to confirm presence.

 

As shown above,  once a worker can scanned the barcode, the tool will record it as "Present".  At that point, the worker performs a cursory examination of the physical item and selects the description that best reflects its condition: Excellent, Acceptable/Good or Poor.   Further details on the tool, including access to the open source software are available on GitHub