The use of alternative metrics is a way to determine the social value and impact of publications.
The traditional method of measuring the importance of scholarly publications is based on citation measurement in other scholarly journals. But social impact can also be determined by figures about how publications are shared, downloaded, bookmarked, mentioned, liked, retweeted and cited on social platforms. This data reveals more information about how the information and knowledge contained in the publication are being actively used. For the academic world this type of measurement can be a valuable addition to well-established Journal Impact Factors (JIF).
For governmental publications there is a big difference. These publications vary in form and format: they can be articles but also news items, websites, blogs, reports, book chapters, conference presentations, Prezis and so on. These publications don’t have a Digital Object Identifier (DOI) and for the most part the authors do not appear in an author database like ORCID and lack a persistent digital author identifier.
Governments publish their information for a number of reasons: they have to be accountable, they are responsible for sharing their knowledge, they want to provide information to support decision-making processes and sometimes they want to influence public opinion. Although government agencies use a broad repertory of means to bring something out into the limelight, they often fail to measure the social impact of their actions.
Alternative bibliometrics, article-level metrics (ALM) or altmetrics have been in use since 2012 to measure the impact of a specific article, rather than the impact of a journal as a whole. They can be used for measuring the social impact of non-scholarly publications like governmental ones. Several applications are used for this new way of measuring, like PlumX and Altmetrics. They measure the use of the information and the social impact by monitoring and measuring social platforms like Facebook, Twitter, YouTube, Vimeo, Slideshare, but also Mendeley, Reddit, Wikipedia, Delicious, Bitly, Scopus, WorldCat - and many others.
Measuring by alternative metrics is a quick method and the results are easy reusable for visual representation.
For this reason Rijkswaterstaat, the executive governmental organisation of the Dutch Ministry of Infrastructure & Environment, performed a case study. After consulting some key figures in the organisation, a selection of authoritative publications was selected for a trial. The publications dealt with a broad variety of governmental topics, like water (floods, risk, dunes, dikes, water plan, sand nourishment), infrastructure (system engineering, land-use), traffic (management, i-mobility), technology (durability, reinforced concrete), law (legislation, dilemmas, mandate, decisions), information (big data, library sciences, online strategy). A test set of 33 publications was compiled; all the publications were freely available on the internet.
With PlumX (Plum Analytics by Ebsco) an account was set up for Rijkswaterstaat and all publications were measured for their social impact. We announced the first preliminary results at Internet Librarian International 2015 in London.
Some of the publications with a general scope, like a YouTube movie with a public relations purpose or a news article in the English daily newspaper The Guardian, gained enormous impact compared to the more scholarly book chapters and reports. The more digital the scope of a publication was, the more impact it seemed to gain.
Based on the first results of this case study, the future for social impact measuring of governmental publications looks bright!
Peter Nieuwenhuizen is Senior Consultant Information Management at Rijkswaterstaat in the Netherlands.