Status Line
- Collections & Technical Services
- Jul-Aug 08
- Issue: 223
Reimplementing Duplicate Detection and Resolution (DDR)
From 1991 through the migration of WorldCat to the new system platform in 2005, OCLC had software that allowed the WorldCat database to be matched against itself to find and merge duplicate records in the books format. Sixteen runs were completed prior to mid-2005, resulting in the elimination of a total of 1.6 million duplicate records. Prior to the last runs, a project was started to reimplement the DDR software to work in the new environment and to enhance it to work with all types of bibliographic records. Progress has been slow with other projects often taking precedence. However OCLC recently announced that the project has been reinvigorated in the past few months, and significant progress has been made. Enhanced matching software, which will be part of the new DDR, was installed last month for use in the batchloading process. This, for the first time, brings the two processes into close alignment as a part of a two-pronged attack on the problem of duplicate records. Current plans call for deduplicating a subset of WorldCat beginning in the 3rd quarter of next fiscal year with additional work on the full database to follow that.
In the meantime, OCLC’s Quality Control Section staff have continued to merge duplicate records in response to reports from member libraries and based on their other WorldCat maintenance activities. Thus far in the current fiscal year, they have completed more than 200,000 merge transactions resulting in the elimination of more than 300,000 records. Special projects have also been undertaken to reduce backlogs of member library-reported duplicate records (most recently, sound recordings). For all formats except books, newly received reports are processed in the month they are received; for books, there is currently a four-month turnaround time.
[OCLC – edited]
Collections & Technical Services Contact Information
Please contact Lauren Pinsley (pinsleyl@nylink.org) or Mary Edgerton (edgertonm@nylink.org) at Nylink with any questions. They are available via email or phone (518-443-5444/800-342-3353).