Cross-evaluation of metrics to estimate the significance of creative works
Wasserman M, Zeng XHT, Amaral LANProc. Natl. Acad. Sci. U. S. A. 112(5), 1281 - 1286 (2015)
Abstract
In a world overflowing with creative works, it is useful to be able to
filter out the unimportant works so that the significant ones can be
identified and thereby absorbed. An automated method could provide an
objective approach for evaluating the significance of works on a
universal scale. However, there have been few attempts at creating such
a measure, and there are few “ground truths” for validating the
effectiveness of potential metrics for significance. For movies, the US
Library of Congress’s National Film Registry (NFR) contains American
films that are “culturally, historically, or aesthetically significant”
as chosen through a careful evaluation and deliberation process. By
analyzing a network of citations between 15,425 United States-produced
films procured from the Internet Movie Database (IMDb), we obtain
several automated metrics for significance. The best of these metrics is
able to indicate a film’s presence in the NFR at least as well or better
than metrics based on aggregated expert opinions or large population
surveys. Importantly, automated metrics can easily be applied to older
films for which no other rating may be available. Our results may have
implications for the evaluation of other creative
works such as scientific research.