What is Validation?

Confidence that retained titles will, in fact, be made available and useable to scholars across the EAST libraries is is critical to building trust among the Retention and Supporting Partners as well as local stakeholders.  

Validation efforts for shared print initiatives span a spectrum of goals with increasing levels of effort and cost:

  • Basic inventory confirmation – is the title actually on the shelf or otherwise accounted for?

  • Condition review  – is the physical volume in good enough condition to support ongoing use?

  • Bibliographic review – is the volume actually as described in the cataloging metadata?

With over 16 million holdings from Maine to Maryland,  a complete item by item validation of the EAST member libraries’ collections was clearly impossible, both in terms of time and money. However, with grant funding and support, EAST has developed a sample based approach to validation.  Two validation sample studies, one completed in the winter/spring of 2016 and the other in planning for completion in late 2016, are focused on developing an "avilability metric" for EAST retention partner libraries and investigating characteristics of the print collections such as age and usage that might impact the likelihood of a scholar being able to access a retained title and could influence the future development of retention models and programs.

Validation Sample Study #1

 

Working with a statistical consultant and the Validation Working Group,  the EAST team designed a sample-based validation study across the 40 EAST collection analysis participant libraries that:

  • Determined an “availability metric” for each of the libraries
  • Provided insight into the condition of titles at the libraries.

Using a random sample of 6,000 titles (determined to be statistically valid assuming at least  90% availability at most libraries with a 1% margin of error) drawn from the collection analysis data, each library used a data collection tool designed by the EAST Data Librarian to validate the presence of titles on shelf and perform a cursory condition review of titles present.  The methdology for the availability check included an initial comparison of the sample data to the local library's circulation system to exclude items known to be not on shelf due to their circulation status.  Following this, each of the 40 libraries sent workers to the stacks to "touch" the almost 6,000 titles in their local sample.  The data collection tool provided an easy to use interface that required only that the worker scan the barcode on the item to account for its presence and perform a brief 3-point condition assessment of the physical item.  Upon completion of a session in the stacks, the data were uploaded and made available to the EAST project team in near realtime allowing easy oversight of the data collection.

Each library was compensated for this work at a standard rate, paid for through grant funds.  This first validation study commenced in February, 2016 and was completed in late April.  Further information on the methodology and access to the open source tools used for Validation Sample Study #1 are available on Github.  While staffing for the study varied from libray to library, most used student workers to scan items in the stacks using the data collection tool. 

The aggregated results of the review of 240,000 holdings across the 40 libraries showed:

  • 97% of monographs in the sample were accounted for: mean: 97%, median: 97.1%, high of 99.7% and low of 91%.  (Note: “accounted for” includes those items previously determined to be in circulation based on an automated check of the libraries’ ILS’s.)
  • 2.3% of titles were in circulation at the time of the study
  • 90% of the titles were deemed to be in average or excellent condition with 10% marked as in poor condition.

This data provide a high level of confidence that students, faculty, and researchers at the member libraries can count on titles that are retained at other EAST libraries being available for their use.  

The EAST project team and statistical consultant are further analyzing the data from this initial study to shed additional light on characteristics that might influence availability and condition.

Validation Sample Study #2

 

Building on the learnings of the first sample validation study, EAST will undertake a second phase of validation later in 2016.  Details of the second validation study are still to be determined and will be informed by a further analysis of the data collected in the first study by the EAST statistical consultant.