New Book

Finding Reliable Information

If you want to improve your own life, the lives of those around you, and the quality of life online, read (the wonderfully readable) “Finding Reliable Information Online.” –Howard Rheingold

Finding Reliable Information Online: Adventures of an Information Sleuth. Rowman & Littlefield.

Chapter 3 is now available Open Access courtesy of the publisher and the author.

 

finding reliable information webinarNEW: An American Library Association/Booklist Webinar on the book is now available free. 

 

 

An Instructor’s Guide for using the book Finding Reliable Information Online is available below. The Guide is free and uses the threshold concepts from the new Framework for Information Literacy for Higher Education just published by the Association of College & Research Libraries. For a word .doc of the guide please email the author LeslieStebbins at gmail dot com.

Instructor Guide Finding Reliable Information Online  (pdf)

 

——————————————————————————————————

Here is an overview. This is the draft Introduction to my new book  that is pre-edit & pre-copyedit. Some full text is also available on GoogleBooks here.

Introduction

I just think that people seem less and less concerned about where their information comes from at a time when I think they should be more and more concerned about it.                                                                                                      –David Carr[i] 

My teenage son comes down for breakfast and reports that Adam Sandler has Ebola. He saw it trending on Facebook.

Slide1

 I sigh inwardly and ask if he has looked at the source of the information. He says yes: CNN.COM.[ii]

While educators used to teach a checklist of skills –currency, relevance, authority, accuracy, purpose — we have moved way past that now. Finding reliable information online is beyond challenging. The process has to start with where you choose to look for information. This book is based on a conceptual approach to finding and evaluating information based on the understanding that research is a creative, reflective, iterative, and most of all a messy process.[iii]

Google works magically for some types of information seeking, but in other cases it leads us to information that is unreliable and in some cases costly or even dangerous. The enormous amount of user-generated content combined with powerful commercial interests clamoring for attention calls for more savvy and nuanced strategies for tracking down reliable information.[iv] These strategies are essential in an age when we are bombarded with information and every answer feels like only a Google search away. This doesn’t mean we need to abandon our Google habit altogether, it just means we need to be strategic in how we search, and understand what types of questions cannot be answered with a simple Google search.

Finding Reliable Information Online uses stories to illustrate strategies for finding and evaluating different types of information. Health, vacation planning, restaurant and product reviews, workplace needs, and scientific claims serve as examples to demonstrate the different purposes of a piece of information: as a commodity, as a means of education, as a means to influence, and as a means of negotiating and understanding the world.[v]

The stories serve as a way to explore many questions about how we can judge the credibility of a piece of information. What percentage of Yelp restaurant reviews are fake? Can we learn to spot these fake reviews? What does 5 stars really mean? Are WebMD and Mayo Clinic comparable and trustworthy health information sites? When can we trust the “wisdom of the crowd” and when should we seek out an expert? When can an amateur give us good advice? Do professional critics still play a role? What’s the difference between a blog and a newspaper site? Is there a difference? When do we need to use scholarly research to solve an information need? Can we trust what we read in National Geographic?

Chapter Overviews

Each chapter investigates a research question from its initial formulation to the final analysis of the best sources that can provide credible and reliable information to answer the question.

Chapter One investigates a seemingly simple health question about red wine and longevity. Over 80 percent of people in the United States who search the web search for health information, and a significant majority of searchers report that they have used the information found online to make a decision about their own health or the way they cared for someone else.[vi] Because research on health topics is so popular, thousands of pages of content are generated daily to attract viewers. Much of the content churned out is unreliable, but some is incredibly useful and credible. This chapter looks closely at user-generated Q&A sites, popular sites such as WebMD, scholarly sites like PubMed, and commonly used sites such as Wikipedia. It also looks at the search process itself and how biases we bring to the table can greatly influence our search results. The chapter also delves deeply into the scholarly research process in health and medicine and unpacks both the challenges and opportunities in finding and evaluating the latest research studies.

Chapter Two focuses on restaurant reviews within the larger arena of user-generated and professional reviews for products, movies, the theater, and books. This chapter traces the origins of restaurant reviews and picks apart the differences between a professional critic, a blogger, or a crowd-sourced review site such as Yelp. Along the way concepts are introduced related to the psychology of search such as our overconfidence in our ability to spot fake reviews, our bias toward thinking a top hit in Google confers a high reliability score, and the problem of source amnesia. Examining when crowd-sourced information can be useful, and uncovering issues with rating systems such as positive data skew; herd behavior; and selection bias are explored in order to understand how to evaluate online restaurant information. This chapter also looks at the different types of knowledge that define expertise and whether professional training is essential to conferring expert status. The ethics of different types of reviewers are examined and the transformed world of restaurant criticism is analyzed.

Chapter Three investigates a work question from the CEO of a successful startup company that is designing a new office building and wants to know if an open-plan office design will increase productivity. The process begins with a simple Google search and then digs heavily into social science research. The concept of the Deep Web is explained and a contextual approach to research is conducted. Research is messy and a certain amount of poking around is needed to uncover reliable information. Formulating a search query and understanding the notion of “satisficing,” confirmation bias, our strong need for closure, and other heuristics – shortcuts — are explored along with the perils of echo chambers. Developing a need for healthy skepticism without becoming cynical, and avoiding “fast thinking” and our tendency to substitute an easier question to avoid doing the work of answering a more challenging question are examined. [vii] The chapter illustrates the need to evaluate arguments based on evidence rather than getting swept up in the easy fix.

This chapter also looks more closely at research processes within the social sciences including the invisible college, the idea of scholarship as an ongoing conversation, the role of research review articles, and techniques and shortcuts for digesting the contents of a research article and evaluating its worth and importance. Patience, persistence, and resilience are needed to fully understand the research on a complex subject. This chapter also looks closely at how to comprehensively research authority: both publishing body and author.

Chapter Four explores the challenge of planning a vacation and deciding what information is reliable in an area where marketing sites dominate. User-generated rating and review systems are unpacked to understand how they should be interpreted and the current research on popular travel sites such as TripAdvisor and Airbnb is explored. The rise of the collaborative sharing economy and its counterpart the “pseudo sharing economy” are also investigated.

Figuring out which sites best match your “travel tribe” and the pros and cons of relying on amateur travel bloggers, professional travel writers, and traditional travel guidebooks are compared. What is expertise in the world of travel? If someone travels a lot are they an expert? What is a parachute artist? Is Google good at selecting the most reliable travel information? This chapter explores bounce rates, negative search engine optimization attacks, and whether it is sometimes ethical to write a fake review. Should travel bloggers disclose free trips? How good are people at spotting false or biased information online?

Chapter Five delves deeply into the problematic link between science journalism and scientific research in looking at the question of whether dogs possess some rudimentary form of empathy. The chapter also illustrates the use of many of the heuristics we use when we search for information: reputation, bandwagon, consistency, persuasive intent, and other shortcuts are examined. These heuristics are a double-edged sword because on one hand they can reduce the amount of cognitive effort used in decision-making, but using heuristics can also lead to systematic biases or errors in judgment.[viii]

The scientific research process and the way that this clashes with the demands of journalism are reviewed and Q&A sites, White Papers, and magazines such as National Geographic and Discovery – that tend to come up first in Google searches — are analyzed to see if they contain reliable science information. Research on the different roles of science blogs and their use in the research process and in communicating science information to the public is also explored. The parallel universe of scientific research that is separate from journalism, the challenges from the lack of open access journals, and the hyping of science by journalists, university press offices, and even the researchers themselves is examined in terms of the roles these play in finding reliable science information. Finally, tracking, filtering, and evaluating the research on a single science topic is demonstrated to uncover the current state of the research on dogs and empathy.

More, Faster, Better?

What started as an exciting experiment – all the world’s information at our finger tips – is evolving into a complex information environment where competing interests vie for our attention. We used to consume our information after it had been filtered for us: by publishers, editors, booksellers, libraries, and others. While there are significant downsides to having an elite group decide what we see and don’t see, we have gone to the other extreme. The gatekeepers are gone, the gates have been flung open, and those with the most strategic marketing plans and search engine optimization strategies are often the ones that win our attention. Who do we want in charge of answering our information queries and giving us advice?

There is a groundswell of thinkers sounding alarms about the dangers of our shoddy information diet, our loss of focus, and our inability to take time for contemplative thinking and long-form reading and analysis. [ix] We celebrate our ability to participate in creating content and we enjoy the vast access we now have to information without questioning the quality and reliability of what we are consuming. There is a great deal of well-founded joy in the wisdom and promise of user-generated crowd-sourced content. Crowd-sourced projects –such as Galaxy Zoo where amateurs help classify millions of galaxies, or grass roots flu-tracking by smart phone app –are expanding our research capabilities. But sometimes we conflate all the positive uses of user-generated data with activities where user data is not useful for generating reliable information or is being gamed by commercial interests. Jaron Lanier has said: “there is an odd lack of curiosity about the limits of crowd wisdom.”[x]

Professor David M. Levy talks about how American society is succumbing to an ethic of “more-faster-better” and is losing the ability for reflection. Levy notes that while there is great value in the variety of information online, people are feeling:

…overloaded, overwhelmed, and oppressed by the amount of information they find themselves sifting through; by the fragmentary nature of the info–bits they are consuming and their own fragmented states of mind.”[xi]

In terms of information “more” is often not “faster” or “better.” Ironically, the more information available on a topic the less likely we are able to find the most reliable sources.

Writer Leon Wieseltier laments the constant disruption from disintermediated pieces of information and the focus on content creation that values quantity over quality:

Journalistic institutions slowly transform themselves into silent sweatshops in which words cannot wait for thoughts, and first responses are promoted into best responses, and patience is a professional liability. As the frequency of expression grows, the force of expression diminishes: Digital expectations of alacrity and terseness confer the highest prestige upon the twittering cacophony of one-liners and promotional announcements.[xii]

The push to constantly churn out new “content” at the expense of creating valuable knowledge is probably the biggest threat to this wonderful opportunity we have of harnessing the web as a tool for knowledge sharing and education. Instead we have pharmaceutical companies directly providing us with health advice by coming up first in Google searches, and gaming platforms giving us relationship or travel advice in order to pull in new customers.

Psychologist Daniel Kahneman encourages us to engage in more “Slow Thinking.” He argues that our brains use two systems to process information. Thinking Fast involves instantly fitting what we see into our preconceived notions of how the world works, whereas Thinking Slow involves making a conscious effort to question the information we are viewing.[xiii] Taking Kahneman’s work one step further: We need to slow down and question how we search for information and where we look for it. We need to become more proactive in our selection of sources.

Media expert Dominque Brossard has conducted research showing that many of us are getting sucked into a self-reinforcing information spiral in which how we search for a topic subsequently influences the algorithms that a search provider like Google uses to weigh and retrieve content. Broussard questions whether we are moving toward a communication process in which knowledge that people acquire is greatly influenced by what links search engines pull up and how they direct traffic to us. Rather than expanding our information universe, search engines and Facebook feeds may be narrowing our options.[xiv]

Avoiding the Drool Bucket

In the Air Force the expression “drool bucket” is used to describe the massive amounts of data provided to pilots who experience such overload that they find themselves lost and staring unblinkingly into massive data systems. While multiple data sources greatly enhance tactical capabilities for pilots, too much data results in an overload that causes this channelized attention that is one of the most common human factors in flight accidents.[xv] The military is currently studying ways to increase the capacity of pilots to focus on important information, while at the same time improving and limiting the flow of data that is provided to these pilots in order to avoid the “drool bucket” phenomenon.

With Twitter, Facebook feeds, and the cacophonous collection of videos, news, blogs, apps, and entertainment services surrounding us we are prime candidates for drool bucket syndrome as well – unless we become more engaged in how we search for information and where we look. Google has come a long way toward effectively delivering search results that are targeted to our requests, but unlike the military that controls and chooses which data goes into the feed a pilot views, Google searches are combing through billions of pages of data, much of it data from sources we would not voluntarily select. At the same time, Google is not able to get to billions of web pages beyond the shallow web including many scholarly databases and journal articles.[xvi]

We are at a crucial tipping point in the evolution of search engines, and in particular in the evolution of Google, Bing, and Yahoo!. Every day Google is searched more than 3.5 billion times. Google is not able to keep on top of the search engine optimization marketers that are gaming the system, and Google itself is engaging in questionable practices by manipulating search results for commercial gain by pointing to Google-owned sites first.[xvii] Many factors contribute to why we get the results we get when searching Google or other search engines. Hitting the top five in the search list sometimes has little correlation with quality and credibility, though research has shown that many people equate link order with reliability.[xviii]

As we become more reliant on the web for every-day-life information, as well as for our research and workplace needs, it is essential that we develop ways to home in on the most useful and trustworthy information, rather than letting other interests control the information that we find. In this book we peek under the hood and kick the tires of such well-known players as Wikipedia and Answers.com. Sites like Wikipedia have become a mainstay for those searching for information via Google, but the early hubris of the Wikipedia community is falling away to reveal a bureaucratic and increasingly small number of predominantly male editors who are heavily controlling “all the world’s knowledge,” while the quality of that information declines.[xix] At the same time, the stakes are so high for sites like TripAdvisor and Yelp — with millions of dollars in play — that questionable marketing tactics are now being used to compete effectively for our increasingly short attention spans.

In a study on the information competencies of recent college graduates in the workplace, employers indicated that their newly hired students were adept at quickly finding an answer, but lacked the skills needed to find the best answers to solve problems in the workplace; they lacked persistence and patience and relied on information found on initial search screens rather than moving beyond Google and using more sophisticated strategies.[xx] This book seeks to address this issue.

And this is Your Brain on Information

New research on the brain suggests that we are increasingly depending on external sources like the web to serve as an extension of our brain.[xxi] A recent study on memory from Harvard and Columbia confirms this phenomenon and demonstrates that having easy access to information on the Internet results in people retaining fewer facts, but instead remembering how to find those facts.[xxii] Not surprisingly, people have adapted to the smart phone in their pocket by no longer trying to remember an address or the hours their favorite coffee shop is open, because they know with a few taps they can retrieve this information.

Scientists suggest that this move toward reaching outside of our brains is inevitable, and what we should focus on is not whether we should be doing this, but on ways to improve this brain-Internet connection. One area of focus should be making sure that the information we reach out and use is reliable. Using the web to augment our brains is fantastic, but not if what we reach out and retrieve is biased or of poor quality. Finding coffee shop hours and basic factual information about the world around us has become straightforward –one tap on a smartphone — but researching more complex questions often cannot be approached with a simple Google search if we want a credible answer.

It would be wonderful to think of the Web as an extremely sophisticated system that could retrieve the highest quality information for every question asked, but basic questions like “how do I get a strawberry jam stain out of my shirt?” are very different from research questions like “Is wind power a threat to birds?” or consumer questions like “What are the least expensive hotels in safe areas of Bangkok?” We are reasonably good at judging the information we get in person from people we know, we weight information with varying degrees of reliability based on who it comes from. We are far less experienced at judging the validity and authority of disintermediated information and too prone to be satisfied with a quick answer. Our vast social networks also provide us with product recommendations and information, but in many cases these “likes” may have hidden motives such as giving five stars to a cousin’s sushi restaurant without ever having eaten there. 

The Six Strategies

The six strategies for finding reliable information are not just another checklist to pop through when reading a piece of information. The strategies start before the search question is even defined, well before the piece of information is viewed. By developing a deeper understanding of how information is created, constructed and packaged; how search engines and websites function; and the many different types and purposes of information available, we can approach search armed with the knowledge needed to strategically uncover the most trustworthy information about a topic.

The six strategies are:

  1. Start at the Source. This seemingly simple but complex idea involves conceptually changing the focus of the search: To hunt for a source for the information first and then the information itself. By taking control of the sources viewed, half the job of evaluation is completed up front, before even reading a piece of information. This conceptual approach is indebted to the work of Marc Meola and his radical notion of “chucking the checklist,” as well as the many librarians involved in the new Framework for Information Literacy for Higher Education.[xxiii] Searching is strategic and this new framework has driven the development of this book:

Information searching is often nonlinear and iterative, requiring the evaluation of a broad range of information sources and the mental flexibility to pursue alternate avenues as new understanding is developed.[xxiv]

Starting at the source might involve a simple decision such as starting the search with Google Scholar instead of Google, or making a few tweaks in how a search query is constructed, but it sometimes involves a lengthier more sophisticated strategy.

  1. The Psychology of Search. This strategy focuses on the other side of the search equation: the psychological baggage we bring to the search. We have a tremendous impact on our search results: the potential for bias in how we ask questions, how easily we are satisfied or willing to settle, the influence of “name” brands and reputations, and social factors that influence our choices. In investigating the psychology of search we look at the search habits of Supreme Court Justices, the information decision-making strategies of burglars deciding which houses to rob, and why tourists using TripAdvisor thought they were going to a resort and ended up at a homeless shelter. The psychology of search draws on the work of Metzger and Flanigan who have investigated the “heuristics”—shortcuts — we rely on when searching for and evaluating information, as well as the work of Daniel Kahneman on Thinking Fast and Slow, and many other researchers who investigate how people search for and evaluate information. The psychology of search also includes the ways we can develop metacognitive habits — thinking about how we are thinking — in order to adjust our search behavior.
  1. Expert, Amateur, Crowd. This strategy involves making a conscious decision as to when to use an expert and when an amateur or crowd-sourced wisdom can be valuable. Some researchers suggest that there is a paradigm shift taking place in how we define credibility as we weigh expectations about expertise, accuracy and absence of bias with the desire for interactivity, transparency, and identification. [xxv] There are not always easy answers: For high stakes searches often a combination of types of information is desirable, but there are also times when expertise provides richer, more complex, and more reliable information. This strategy looks closely at the definition of an expert in different arenas: restaurant and travel information, health advice, scientific research, and workplace information needs. The ACRL Framework underlines the idea that authority can be fluid: for a medical decision we may want a doctor, but for dating advice a peer might be the superior source. At the same time, Howard Gardner’s recent work provides an anchor and inspiration in an information world that has been turned upside down:

…best antidote to the attack on truth is understanding the methods that expert people use to determine the truth. …you have to understand the methods that human beings have developed over hundreds if not thousands of years, of separating the truth from the chaff… And I think it can be done. Indeed, I’d even go further … . I think at this point in human history … if you’re willing to work at it, if you’re willing to be skeptical without being cynical, the chances of establishing what’s really true are greater than ever before.[xxvi]

  1. Context, Motivation, and Bias. Context relates to everything surrounding the information including the context of the search you are embarking on: Bar bet? Research paper? And context also includes determining the purpose of a piece of information, understanding how it was constructed, whether it was vetted, edited, peer reviewed, published, and commented on. By figuring out context we can better understand the motivation and potential bias of a piece of information.

Many of us rely on a few trusted sources for many of our information needs, but we need to be both flexible and skeptical and adjust our strategies according to the importance of a particular search. Sometimes we want to take charge and control the context ourselves: if we are researching bee colony collapse we might want to go onto a biology or science database such as PLOS.org, rather than rely on what comes up first on Google: a four paragraph article from Forbes suggesting a simple cause of colony collapse and surrounded by advertisements.

  1. Comparison and Corroboration. In scholarly research there is an ingrained process of basing new research on previous findings. Research is not conducted in a vacuum. When carrying out a comprehensive literature review on a health, social science, or science topic, comparison and corroboration are built into the process. But for many every-day-life searches we pop in and look for a quick answer. If we pop in to an extremely trustworthy source and the information topic is a low stakes issue, we may need to use only a bare bones comparison and corroboration process. But even confirming a simple piece of information can be challenging because so much information is rebroadcast and regurgitated across the web. The comparison process needs to involve using independent resources and the corroboration process involves unpacking the factual claims and confirming that they hold up.
  1. Going Deep, or Not. Searching for information is an iterative process fraught with continual choices: where to look, how far to go, how many sources to read, what types of sources to go after. Taking a piece of information and uncovering the back-story –and every piece of information has a back-story—can be a lengthy process. How was it created? Who is this author? What is the purpose? Do others agree? What is it based on?

An important component of the search process is recognizing when a quick piece of information will suffice and when it is going to be a long night. A simple bar bet can turn into a “Who killed JFK?” type of search, and what appears at first glance to be a complex science search can sometimes lead easily to the perfect review article at the top of a Google Scholar results list. Stay open, bend your knees, be flexible.

The American Library Association recently stated that:

Increasingly, information comes to individuals in unfiltered formats, raising questions about its authenticity, validity, and reliability… The uncertain quality and expanding quantity of information pose large challenges for society. The sheer abundance of information will not in itself create a more informed citizenry without a complementary cluster of abilities necessary to use information effectively.[xxvii]

 This book is about developing that cluster of abilities needed to find reliable information online. Thinking strategically and controlling where you look for information and how you evaluate it are pieces of a much larger ongoing challenge for how we build the web in the years to come. There is no reason why shoe stores, doctors, restaurants, libraries, and schools cannot co-exist on the web, but we need to have better markers of quality and reliability. We need to know when we are in a shoe store and when we are in a library when we are online. We need many new tools, beyond the six strategies discussed in this book, to help filter and curate the web. Crowdsourcing “likes” is not a sufficient strategy.

———————————–

[i] Hicks, Jesse. “The Verge Interview: David Carr on the Curation, Crowdsourcing, and the Future of Journalism.” The Verge. (April 3, 2012). http://www.theverge.com/2012/4/3/2912487/david-carr-interview-dnp

[ii] A Facebook user shared this on their Facebook feed and then described how to hack the text and photos on a news outlet’s site and then share the results so that the image and text link back to the official site, even though once you go to the official site the hacked story and image do not exist. In this case the site hacked was CNN.com.

[iii] The Framework for Information Literacy for Higher Education. Draft 3. Association of College and Research Libraries. (November 2014). http://acrl.ala.org/ilstandards/wp-content/uploads/2014/11/Framework-for-IL-for-HE-draft-3.pdf

[iv] This book uses the word “reliable” to mean trustworthy, true, valid, authentic, and accurate. It is used to apply to all types of information: factual and subjective, to the degree that that is possible. For a good discussion of the differences between words such as credibility, accuracy, trust, and authority see: Flanagin, Andrew and Miriam Metzger. “The Credibility of Volunteered Geographic Information.” GeoJournal 72 (2008). doi 10.1007/s10708-008-9188-y

[v] Framework for Information Literacy for Higher Education. Association of College and Research Libraries (Februrary 2015). http://www.ala.org/acrl/standards/ilframework

[vi] Fox, Susannah. Health Topics Report. Pew Internet and American Life Project. (February 2011). http://www.pewinternet.org/Reports/2011/HealthTopics/Summary-of-Findings.aspx

[vii] Kahneman, Daniel. Thinking Fast and Slow. New York: Farrar, Straus & Giroux (2011).

[viii] Materska, Katarzyna. Information Heuristics of Information Literate People. [Preprint] (2014). http://eprints.rclis.org/23919/

[ix] Johnson, Clay. The Information Diet: A Case for Conscious Consumption. Sebastopal, CA: O’Reilly (2012); Lanier, Jaron. You are Not a Gadget: A Manifesto. New York: Vintage (2011); Levy, David. “More Better Faster: Governance in an Age of Overload, Busyness, and Speed.” First Monday 7 (2006). http://firstmonday.org/ojs/index.php/fm/article/view/1618/1533

[x] Lanier, Jaron. You are Not a Gadget: A Manifesto. New York: Vintage, (2011).

[xi] Levy, David M. More Better Faster: Governance in an Age of Overload, Busyness, and Speed. First Monday 7 (2006). http://firstmonday.org/ojs/index.php/fm/article/view/1618/1533

[xii] Wieseltier, Leon. Among the Disrupted. New York Time Book Review. (January 7, 2015). http://www.nytimes.com/2015/01/18/books/review/among-the-disrupted.html?_r=0

[xiii] Kahneman, Daniel. Thinking Fast and Slow. New York: Farrar, Straus & Giroux, (2011).

[xiv] Brossard, Dominque and Dietram A. Scheufele. “Science, New Media, and the Public.” Science 339, no. 6114 (2013); Mitchell, Amy. State of the News Media 2014. PewResearch Journalism Project. (March 26, 2014). http://www.journalism.org/2014/03/26/state-of-the-news-media-2014-overview/; Miller, Claire Cain. “Why BuzzFeed is Trying to Shift its Strategy.” The New York Times. (August 12, 2014); Goel, Vindu and Ravi Somaiya. “With New App, Facebook Aims to Make Its Users’ Feeds Newsier.” The New York Times (Februrary 3 2014). http://www.nytimes.com/2014/02/04/technology/with-new-app-facebook-aims-to-make-its-users-feeds-newsier.html

[xv] Shanker, Thom and Matt Richtel. “In New Military, Data Overload Can Be Deadly.” The New York Times. (Jan 17, 2011). http://www.nytimes.com/2011/01/17/technology/17brain.html?pagewanted=all&_r=0#

[xvi] Goshen College. The Invisible Web. Viewed February 1, 2014. http://libraryguides.goshen.edu/beyond_googling

[xvii] Efrati, Amir. “Rivals Say Google Plays Favorites: Search Giant Displays Its Own Health, Shopping, Local Content Ahead of Links to Competing Sites.” Wall Street Journal. (December 12, 2010). http://online.wsj.com/news/articles/SB10001424052748704058704576015630188568972

[xviii] HargittaI, Eszter, Lindsay Fullerton, Ericka Menchen-Trevino, and Kristin Yates Thomas. “Trust Online: Young Adults’ Evaluation of Web Content.” International Journal of Communication 4: 27 (April 2010). http://ijoc.org/index.php/ijoc/article/view/63

[xix] Simonite, Tom. “The Decline of Wikipedia.” MIT Technology Review (October 22, 2013). http://www.technologyreview.com/featuredstory/520446/the-decline-of-wikipedia

[xx] Head, Alison J. How College Graduates Solve Information Problems Once They Join the Workplace. Project Information Literacy Research Report. The Passage Studies. Institute Of Museum and Library Services and The Berkman Center For Internet and Society at Harvard (October 16, 2012). http://projectinfolit.org/pdfs/PIL_fall2012_workplaceStudy_FullReport.pdf

[xxi] Zimmer, Carl. “How Google Is Making Us Smarter: Humans are ‘natural-born cyborgs,’ and the Internet is our giant ‘extended mind.’” Discover Magazine (Februrary 2009).

[xxii] Sparrow, Betsy, Jenny Liu, and Daniel M. Wegner. “Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips.” ScienceExpress Report (2011). http://www.wjh.harvard.edu/~wegner/pdfs/science.1207745.full.pdf

[xxiii] Meola, Marc. “Chucking the Checklist: A Contextual Approach to Teaching Undergraduates Web-Site Evaluation.” Portal: Libraries and the Academy, 4 (3) 2004. For a review of approaches see: Hjorland, Birger. “Methods for Evaluating Information Sources: An Annotated Catalogue.” Journal of Information Science 38 (June 2012). Doi: 10.1177/016555151439178

[xxiv] Framework for Information Literacy for Higher Education. Association of College and Research Libraries (Februrary 2015). http://www.ala.org/acrl/standards/ilframework

[xxv] Carroll, Brian and R. Randolph Richardson. “Identification, Transparency, Interactivity: Towards a New Paradigm for Credibility for Single-Voice Blogs.” International Journal of Interactive Communication Systems and Technologies. 1, no.1 (January 2011). doi>10.4018/ijicst.2011010102

[xxvi] Gardner, Howard. Howard Gardner on Truth, Beauty, and Goodness. Richard Heffner’s Open Mind. Interview. Open Mind. (July 2, 2011). http://www.thirteen.org/openmind/media/howard-gardner-on-truth-beauty-and-goodness/2310/

[xxvii]Association of College and Research Libraries. Information Literacy Competency Standards. (2000). http://www.ala.org/acrl/standards/informationliteracycompetency