Artificial Intelligence

To interact effectively with AI, it is important to provide prompts that convey the results you are seeking. It is also important to evaluate the information from generative AI tools due to the sometimes lack of transparency regarding the sources with which they were trained. Just as you would evaluate any piece of information before incorporating it into your research, you should also evaluate the output of GenAI and the tools you are using. Remember the saying "Garbage-in, garbage-out" (GIGO), and be wary of any information presented as fact by a generative AI tool unless you are certain of the credibility of its training sources or can corroborate the information through other information sources.

The quality of the responses you receive from a generative AI tool may depend on the prompts you use to elicit those responses. For this reason, the art of prompt engineering has developed as a way to guide a generative AI tool to give you the clearest responses with the most helpful features, e.g. cited sources.

This University of Calgary guide offers information on prompt generation, the act of forming an explaining effective request for an LLM. 

MIT offers three key suggestions for creating prompts:

• Provide Context
• Be specific
• Build on the Conversation

OpenAI Tips on Prompt Engineering.

Example Prompt and Response from Microsoft Copilot

Response: Certainly! 📚 Here are the major themes in Mohsin Hamid’s novel “Exit West”:

1. Migration: The novel explores how people immigrate to other countries in search of new possibilities, whether for safety, opportunity, or connection. The mysterious doors serve as portals for instantaneous migration, emphasizing the global nature of this theme [12].
2. The Danger of Nationalism: Hamid questions the concept of belonging based on nationality. Characters who cling to ideas of purity or nationalism often turn toward violence. The novel suggests that the country of origin plays only a small role in a rapidly globalizing world [1].
3. Mortality: Despite the violence depicted, the theme of mortality highlights life’s fragility and unpredictability. It emphasizes the commonalities of humanity and the preciousness of life [1].

For further interpretation and analysis, you can explore the following secondary sources:

• SparkNotes: Provides detailed summaries, themes, and character analyses [34].
• LitCharts: Offers visualizations, quotes, and explanations of themes in "Exit West" [5]

Note: The sources the chatbot provides for further analysis are not, in fact, scholarly at all. This is especially true of the blog site “booksthatslay.com”. Notice the phrase, “AI-generated content may be incorrect”. Take this into consideration when drawing material from a chatbot session.

Fact-Checking AI Output

It's important to think about the differences between Generative AI tools and search engines, such as Google. Because the sources upon which an LLM was trained are not always openly shared, and the frequency with which a generative AI tool can produce hallucinations, it’s important to keep a critical mindset when attempting to glean factual information from such a tool. We recommend fact-checking apparent claims made by a generative AI tool by using the strategy of lateral reading — that is, having a separate browser tab open where you can check claims against Wikipedia, popular fact-checking sites, open web sources, and, of course, library resources.

Evaluating AI Tools

In addition to evaluating AI's output, you should also critically evaluate the AI tools you choose to use. There are several evaluation frameworks to help guide your evaluation, including the SIFT Method (Stop, Investigate, Find, Trace).

One to start with is the ROBOT Test, which is a tool you can use when reading about AI applications to help consider the legitimacy and usefulness of the technology. ROBOT stands for:

• Reliability
• Objective
• Bias
• Ownership
• Type

Reliability

• How reliable is the information available about the AI technology?
• If it’s not produced by the party responsible for the AI, what are the author’s credentials? Bias?
• If it is produced by the party responsible for the AI, how much information are they making available? 
  • Is information only partially available due to trade secrets?
  • How biased is they information that they produce?

Objectivity

• What is the goal or objective of the use of AI?
• What is the goal of sharing information about it?
  • To inform?
  • To convince?
  • To find financial support?

Bias

• What could create bias in the AI technology?
• Are there ethical issues associated with this?
• Are bias or ethical issues acknowledged?
  • By the source of information?
  • By the party responsible for the AI?
  • By its users?

Ownership

• Who is the owner or developer of the AI technology?
• Who is responsible for it?
  • Is it a private company?
  • The government?
  • A think tank or research group?
• Who has access to it?
• Who can use it?

Type

• Which subtype of AI is it?
• Is the technology theoretical or applied?
• What kind of information system does it rely on?
• Does it rely on human intervention? 

It's important to note that no one evaluation framework or checklist provides a foolproof test--but regularly asking yourself these types of questions when encountering new AI applications (or new information in any format) will help you build your research and AI literacy skills.

The ROBOT Test is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.