Given the high value placed on research excellence by legal professionals and consultants, I am surprised that stories continue to be reported about the lack of rigour exercised in the creation of work product by these professional groups. In addition to the ongoing stories of professional sanctions placed on lawyers for including incorrect citations and other issues associated with the use of generative AI, there have been regular stories about the high values for government report contracts and and the use of AI to create them. Here are some articles on a report prepared by Deloitte for the Province of Newfoundland and Labrador on health-care worker staffing that was prepared for the price of $1.6 million:
- Matt Barter, “GOV NL Spent Over $1.5 Million on Health Human Resource Plan,” via his blog.
- Justin Brake, “Major N.L. healthcare report contains errors likely generated by A.I.,” via The Independent.
- Justin Brake, “N.L. government pledges ‘strict review’ on AI use after more false citations found in reports,” via The Independent.
- Garrett Barry, “Deloitte breaks silence on N.L. healthcare report,” via CTV.
In the CTV article includes the following:
“AI was not used to write the report; it was selectively used to support a small number of research citations,” a spokesperson for Deloitte said in a statement. “We are revising the report to make a small number of citation corrections, which do not impact the report findings.”
I notice that the quoted text uses the Unicode character “narrow no break space” with the code <0x202f> on either side of the word “write”. This character allows for control over how text is displayed by showing a space but forcing applications like web browsers to not allow the text to run to a new line at that spot, it is used in places like before a colon in French typesetting. It has been found to be a common artifact of generative AI systems, and, as there is no apparent reason for it to be used there, I infer this indicates that a similar system was used in the drafting of the statement (to check for this you can copy and paste the section from the site into a text editor such as NotePad or TextEdit). I interpret this to mean that generative AI is used in a widespread way at Deloitte, which conforms to the way they discuss their work processes in their marketing material on their website.
I recognize that generative AI can be useful, but to me the particular type of situation outlined above indicates lack of interest in important parts of research and writing non-fiction (using generative AI in writing fiction is another discussion). This is concerning to me, as it seems to indicate that the authors followed one of two processes, either they have written something and then used an AI system to generate citations retroactively. Or they were revising their citations using an AI system. To me this is implicitly saying that the body of the text is important but the citations matter less, which is a problematic perspective for research integrity.
At the risk of mounting one of my hobby horses, citation practice is a core component of writing substantive content like this. It is not put there to be decorative or to give a document the appearance of gravitas. In The Independent, Justin Brake reported that “Those citations reference research articles which don’t exist but were used to support claims related to virtual care, monetary recruitment and retention incentives, recruitment strategies, and impacts of the COVID-19 pandemic on healthcare workers. In at least two cases, the citations also named actual researchers who did not author the fabricated articles.” Though it is time consuming to verify whether citations exist, it is more so to verify if the referenced material actually says what it is asserted to say, especially in a situation where a system has been used that is designed to provide text that seems like the kind of thing it would say.
One of my foundational professional memories is teaching a class on research to first year students at the University of British Columbia Library when I was still a student and having one of the students respond on the evaluation form that the most important thing they learned in the session was “that works cited has a real purpose.” When she read it, my supervisor looked me in the eyes very intently, asking: “What exactly did you say?” My answer as I recall was that it places your writing into a wider dialogue with what others have written, gives credit to others for their ideas, and increases your credibility by showing that you have researched the topic and are not simply writing your own thoughts.
I was recently told that developers of legal information online have been exploring ways to better integrate their content into the emerging environment of AI generated snippets in online search. At the Law Via the Internet Conference in November, Craig Newton, co-director of the Legal Information Institute at Cornell Law School, said that providing text in a format that is suitable for this use can mean that online reach is significantly inflated over site visits, with a potential audience of millions. However, the information at the bottom of the snippets that references websites as support for the content is misleading: it appears that these are intended to be citations, but in fact they are AI generated lists of sites that include the kind of information included in the snippet. It is impossible to know from this display where the actual text came from.
Breaking down the network of citations and treating it as an afterthought to research and writing is a concerning trend, though it didn’t start with the launch of widely available generative AI platforms. In response to this, I would encourage you to learn and teach others how to use citation management software. These are mature products that work well, and which have many attractive options (my personal favourite is Zotero). These applications will allow you to manage your research for immediate use, but also over time, and they integrate well with word processors to allow you to avoid the chore of manually inserting and updating your references. There are excellent ways to approach how to manage tracking the sources of the ideas, and I encourage you to learn how to use them before you start a big project. I’m sure you can find a library with people who will help you with this if you need it. It can avoid significant difficulties.
— I would like to thank Jen Brubacher, Katarina Daniels, and Annette Demers, who discussed this with me on the Canadian Association of Law Libraries member forum before I wrote this column.
The post Hallucinated References, Government Reports, and Managing Your Citations appeared first on Slaw.