In this chapter, you will be given guidance on how to prepare and perform a literature review on your chosen research topic. This includes learning about scholarly networks and sub-networks, as well as recognizing some of the common pitfalls that happen when you start gathering academic secondary source materials to scan and subsequently read. This guide to reviewing secondary literature also includes step-by-step instructions on how to start the literature review process, how to organize materials into different lists and how to read and take notes on the most relevant and interesting sources that you find.
1. Understanding Secondary Materials
- Secondary materials may include academic books and journal articles as well as investigative reporting and backgrounder pieces in newspapers, magazines and blogs
- While there are many different kinds of secondary source materials out there, this chapter will deal primarily with articles and books written by academics for an audience of other academics
- The DEFINING CHARACTERISTIC of any secondary source is its goal, which is to explicate and explain information
- In terms of bias, secondary materials can and often do overtly display a point of view, and this material is influenced in part by the ideological, philosophical and personal predilections of its authors
- That said, some fields (e.g. the hard sciences) make every effort to keep the values of the authors in check, while others (e.g. humanities) provide more room for authors to openly display their commitments to certain points of view
- Academic secondary sources often make a contribution to a theory, which is a proposed explanation about a particular situation
- In some cases, their theoretical work will be narrowly construed, in which case the research is located in context of others’ work in the same field, or broadly construed, which is more abstract and offers a more all-encompassing point of view including defined concepts and methodologies for future research (e.g. Marxism, political economy, legal pluralism)
- For your research project, you will need to perform a high level literature review of academic secondary sources which will involve carefully reading and critically assessing the articles and books selected for inclusion
- This is in contrast to a low level literature review which usually involves a quick survey as opposed to thorough review of materials
2. Selecting Secondary Materials
- Unlike a primary documents search which involves collecting many materials and not discarding anything until the near end of the research process, with secondary materials, discarding will come EARLY in the in the research process
- This means you will have to decide which materials to select and how to categorize them based on the discipline or field of study which it came from
- Everything you choose has been shaped by a debate within the discipline that is already in progress, and any new research is simply contributing to this debate as well
- You need to ID these debates in order to understand what the authors are saying, and this will not happen if you just select items you already know or simply find first
- Having a STRATEGY is therefore necessary to create a manageable literature review that has relatively few items but ones that are both interesting to you and relevant to your particular public issue, jurisdiction and time frame
3. Scholarly Networks and Sub-Networks
A. Scholarly Networks
- One excellent way of understanding disciplinary debates is to seek and identify different scholarly networks
- Scholarly networks are the loose connections between and among academics and/or scientists
- Sometime these networks can be large (thousands of people) yet usually, they are small, with as few as 50 or 60 people being part of this network
- A scholarly network is evident when academics write in the same journals, cite each other’s work, attend the same conferences and most importantly, focus on the same issues for debate
- In other words, their research is contributing to a conversation or debate they and their colleagues are having about the same issue
- In order to do this, those within a network have to AGREE on some basic principle or perspective (i.e. Marxists will normally be in the same network as hard-core capitalists), yet this doesn’t mean they won’t DISAGREE about things like what topic area to focus on, how to best approach or solve a problem, research methodology, and even definitions of key concepts in the field
- Within a scholarly network, such disagreements may result in the creation of sub-networks which you also need to look out for.
What is the difference between a network and a sub-network?
Think of a sub-network as a group of academics who share the same perspective about their particular topic that may be different from others within the greater scholarly network. For instance, there is a currently a listserve on regulation and governance that has 2000 members. Yet while this might constitute a network of academics who are all interested in the field of regulation, there are different sub-networks which represent scholars who are interested a particular topic or share a particular point of view. For example, some scholars may be more interested in regulatory compliance as opposed to EU governance policies. In addition, others may belong to a sub-network of conservative writers who critique the idea of regulation as a whole, or come from “the regulation school” where a broader definition of regulation is adopted to reflect social relations in general.
Don’t forget however, that while disagreements within and among networks and sub-networks are very common, what holds a scholarly network or sub-network together is a shared sense amongst its participants that a particular issue, perspective, method or contention is important enough to be debate thoroughly.
C. Evolving and Shifting Networks and Sub-Networks
- Please remember that scholarly networks and sub-networks are not stable and regularly change over time
- Sometimes, this is just part of the normal evolution of a network or sub-network, but at other times the change is more radical
- There are many reasons for this, including the fact that new issues command attention, the debate is exhausted or people just don’t care anymore about the issue, new people join the network or a new article emphasizing something different such as new concepts or terminology attracts attention
- Academic conference documents are valuable in a literature review because they help us understanding network shifts since their topics, agendas and participants provide an indication of where new networks are developing or how existing ones are evolving
- This is also why you should pay attention to when a book or article is written, as by following publication dates, you can following the evolution of thinking about the issue of concern
Now that I know what networks and sub-networks are, how do I find them?
Although scholarly networks are sometimes clearly defined, most times a network or sub-network is an implicit arrangement. Even so, you can locate these networks and sub-networks fairly easily! For example, you will know you have uncovered the basic outlines of a network when the same authors’ names keep appearing in the citations and bibliographies of the articles and books you find.
4. Doing a High-Level Literature Review
A. Research Pitfalls
There are four problems that arise in any literature review, and it is good to know what they are before you get started:
1. Dated Materials
- Many references you find will be to material that is dated, and this often happens since the time-lag from final draft to publication of an article or book is so long
- Be aware that this can be a problem because academic research and debates move quickly, sometimes changing within as little as three years
- If this happens, see if the author has written anything more recently about your topic
- Still, don’t dismiss dates sources entirely as some materials remain both relevant and interesting long after publication
2. Lack of Materials
- Sometimes if you have a narrowly defined research questions it may appear that very few books or articles have been written about it
- If this happens to you, the best way to include material on your list is to choose secondary sources you find both RELEVANT and INTERESTING
- It is not enough to search for items that are simply about the same topic, you need to search around your question, not just on it
- Once you find one or two books or articles that are truly relevant and interesting, you should not have any problem finding material directly related to your topic
3. Know Background and Context
- While a good research question should be fairly narrow in scope, you still have to know about the background and context of your topic to make sure you are interpreting things correctly
- That said, your research will quickly become unmanageable if you spend too much time examining background information which could result in many new literature reviews
- To avoid this, conduct small-scale literature reviews for background info and use other sources such as interviews, encyclopaedias and even Wikipedia
4. Know Norms and Conventions of Your Field
- You need to know the norms and conventions associated with your field or discipline, or you will not be able to appreciate the nuances of what you read, whether a particular book or article fits within a network or whether there is an audience for this material
- One good way to knowing the field norms is to consult with experts within that particular area
5. Research Steps (as they relate to your literature review)
A. Get a good research question
- While your research question may change and take shape during the literature review, there needs to be a clear starting point in order to define the initial parameters of your search
- For further info on good research questions, refer to Chapter II
B. Locate a couple of gems
- A good literature review requires finding one or two books or articles that seem pertinent to what you what to know about
- Once you find these sources quickly scan them and look for which authors are mentioned in the text and notes
- Also scan the bibliography to find items that appear both relevant and interesting
- Scanning should take no more than 10 minutes of your time, and your focus should be on the abstract, first paragraph as well as notes, references and bibliography
- Careful reading will come later, as if you start now, you will NEVER get through your literature review
- Next, take these materials and do the following: collect the abstracts, record the author(s)’ names and citation information
- Doing this will allow you to quickly compile a list of many books and articles which will only continue to grow and you continue to scan and search for more relevant and interesting sources.
You keep saying I should look for relevant and interesting materials. I get the relevant part, but I am unclear by what you mean by “interesting.” What does an interesting source look like and why is this important for my lit review?
It is usually self-evident why an item is placed on your list when it seems relevant. But relevance is not the only criterion. Not only should an article or book be relevant, it must also look interesting to you. Otherwise, you are not likely to read it! This is also a handy way of pairing down sources, as while there may be many books and articles relevant to your topic, only a small proportion of them will look interesting enough to provoke you to follow up. DON’T PUT SOMETHING ON YOUR LIST IF IT DOESN’T COMPEL YOU ENOUGH TO READ IT.
Also, remember that deciding whether something looks interesting is not really a rational choice. It is more a matter of instinct, personal tastes, goals, and points of view. A better term might be “gut feeling.” Sometimes, making decisions on items that look interesting results in the literature being a reflecting your real preoccupations. Deciding that something looks interesting will go a long way to helping you shape your research question into something that you really want to answer!
C. Internet Author Search
- Once you have a good list going, look at the authors of the books and articles on it and do an internet author search to see if they have written anything else about the topic
- Here, you are looking for more recent and relevant materials that you can scan and quickly examine
D. Retrieve Materials
- Locate and retrieve the books and articles on your list
- Scan them and look for names of authors and other books and articles you want to add to your list.
Couldn’t a computer search accomplish all of this much faster than doing it manually?
There are some similarities between a manual literature search and computer search, as both processes involve seeing authors as being part of a network or sub-network of scholars writing about the same things. Yet the big difference between these search methods is that with a manual search, YOU decide what is relevant and interesting, not the machine! Everything on your list is important because you have chosen it – unlike a computer which simply looks for materials in the same topic area or with the same keywords.
E. Discarding Items
- Your reading list is going to get long very quickly, so after some time start paring down your list by discarding materials that once seemed relevant and interesting but are now less so in comparison to other items on your list
- Once you have pared down your list, keep shaping and re-shaping the list by removing one item for each source you add to the list
- A 30 (thirty) item limit can be adopted to ensure hard choices are made and that you have truly created a worthwhile literature review
- Don’t forget that just because you remove something from your list now doesn’t mean you can’t return to it later on! Since you have all the citation information, simply put these sources in a file and revisit if required
F. Make 3 (three) Lists
- Based on this decision making exercise of choosing relevant and interesting sources, you will create three lists: the A list, B list and C list
i.Your “A” List
- This list will include only those articles or books that you really think are worth taking the time to read carefully and to annotate (take notes) thoroughly
- This will likely be only a portion of the books and articles on your list
ii. Your “B” List
- This list will contain useful background information
- Make sure to keep accurate citation and locator information, as these are articles and books you will return to later on in the research process
- Here, a short list of the kinds of information available in the book or article is essential, including tables and stats
iii. Your “C” List
- This list will include everything you thought was interesting and relevant, but on closer reflection, is not so compared to what is on your “A” list
- Like your “B” list, make sure to keep accurate citation information
- In addition, you might want write a sentence about the source and to cut and paste an abstract or page from the introduction for future reference
G. Follow-up “A” List Items
- Once you have an “A” list to work with, follow up on these items by doing things such as finding the journals that the articles are published in
- These journals may now become a new source of inspiration for your “A” list, as are the call numbers of the journal or book which can also be used to perform a manual shelf search
- Don’t forget that for every item you add to your “A” list, something else must be discarded
H. Online Search
- At this point, you can perform an internet key word search which may help you find sources that were not captured in your original search
- Doing a key word search is much easier following these earlier steps as you will now know which terms yield the best results
I. Read Carefully
- Start reading your “A” list items carefully and take notes (see next step)
- Often, once you start reading you will find that something which seemed relevant and interesting is not so after all – if so, you can safely discard this source
- If your list appears to be growing quite small as a result of careful reading, simply start scanning for sources once more which should take far less time now since you know what to look for in terms of relevance and interest.
I am getting tired just thinking of doing all of these steps. Aren’t there any shortcuts I can take for my literature review?
We understand that all the prep work for creating your “A”, “B” and “C” lists will take a lot of time, and that you may want to start reading items right away before your “A” list has been properly pared down. Yet the time saved by taking any shortcuts is illusionary since you will likely read and take too many notes on sources then end up being irrelevant and/or uninteresting. Careful reading and note-taking will consume a huge amount of your time, and it is better to ensure these tasks are devoted to material that is truly worth the effort!
- The goal for taking notes is that once your source has been read and the notes are complete, you need never return to the original material again
- There is no right length for an annotation, as some materials may be captured in a few pages while other sources may require reams of notes – the key of course, is to do it properly so that you don’t need to look at the book or article again
- When annotating your sources, make sure to include the following:
- Citation (complete with page numbers)
- Locator Information (e.g. URL, library call number, person who recommended reading, original source where reference was found)
- Source abstract or table of contents (cut and paste if necessary)
- Key ideas that are relevant and interesting for your purposes
- Main arguments about these key ideas
- Methodology (description of what was done to arrive at findings)
- Concepts used by the author(s), including specific definitions
- Discussion of connections between this source and others on your “A” list
- Other authors referred to in the text, and their relevance to your research
- Concepts used by these other authors, including their definitions
- Quotes that may be useful for your research, including page numbers
- Comments on how the material might be used later
Why do I need to include all this information when taking notes?
Steps 1 and 2 are critical to keep track of your materials in case you need to retrieve these items later on. Steps 3, 4 and 5 simply recap the contents of the article or book but focus exclusively on aspects of the content that are relevant to your own research. Steps 6, 7, 8, 9 and 10 are ESSENTIAL since they provide the basis for discussion of this material once the literature review is complete and/or in your analysis of your own research. This is where you should spend the bulk of your time taking notes. Steps 11 and 12 are simply for own your convenience.
My thesis focused upon tort theory relating wrongful death damages. The question that I sought to answer is this: why is loss of human life worth so little monetary compensation in Canada, the United States and Britain? Although I was well versed in the law and history of wrongful death, I knew nothing about legal theory. I had hoped, at the beginning, to find a wide variety of discussions about the theoretical implications of wrongful death law. I believed that wrongful death was a fascinating topic that had surely captured the fancy of many academics. I was, unfortunately, dead wrong.
It was recommended that we begin our literature review by finding two books or articles that seemed relevant to our topic. I decided that my research would instead be better served by an Internet keyword search. My plan was to scan a wide variety of sources before choosing my two initial favorites. After two weeks of reviewing a seemingly endless stream of online books and articles, I realized that I had made a mistake. I found many secondary sources that mentioned wrongful death law in greater or lesser detail, but they were numerous and unorganized. Most of these sources contained only brief and rudimentary references to wrongful death; and almost none of them discussed wrongful death’s theoretical implications. In fact, no secondary source that I encountered even mentioned the legal theory that I subsequently chose for my thesis’ analysis. I may have never found most of the books and articles that are currently listed in my annotated bibliography had I continued my Internet searches.
So I belatedly followed the advice, by perusing textbooks that surveyed contemporary tort law theory. I found my preferred tort theory within an hour: something called “Aristotelian corrective justice theory.” Contemporary exponents of corrective justice believe – to varying degrees – that tort cases should be guided only by factors that are relevant to the disputes between litigating parties. External concerns such as political, economic, and social factors should play no role in tort law. My previous experience as a tort litigator predisposed me towards this theory. So I next reviewed these textbooks’ relevant footnotes and bibliographies in order to find my two initial sources (textbook footnotes are particularly useful for this task because they list only the most important books and articles). My two initial sources comprised of a short book – Ernest Weinrib’s The Idea of Private Law, and an essay from a printed collection – Bruce Chapman’s “Wrongdoing, Welfare, and Damages: Recovery for Non-Pecuniary Loss in Corrective Justice.” I chose Weinrib’s book because I was intrigued by his purist argument that only common law is correctively just, and that all statutes should accordingly be eradicated from our tort legal system. I selected Chapman’s essay for two reasons: 1) He combines Weinrib’s theory and compelling argument to reach a conclusion about tort damages with which I strongly disagree, and 2) his article was written in response (and therefore contained references) to articles that corresponded more closely with my biases toward tort damages. Neither of these sources discussed wrongful death. I reviewed my two sources’ footnotes and bibliography to find more books and articles of interest. I then turned to these new sources’ footnotes and bibliographies, and so forth.
The full text of most journal articles and other academic commentary is available online. Many books and collections of essays, on the other hand, can only be found in print. Nonetheless, citations for and references to most print books and essay collections can still be found online. I found that full-text databases (such as HeinOnline) were very useful for scanning legal journals’ footnotes. Not only did these databases contain large collections of material that was easy to scan, several of them offered a reference citations feature. With the click of a button, I could see a list of all subsequently written articles and academic commentary that referenced the article I was reviewing. This feature thereby provided me with abundant links to other authors. In addition, the citations feature gave me a quick and easy way to measure an article’s relative importance; as high impact journal articles tend to generate more responses. Whenever I found an author’s work to be particularly relevant, I checked to see whether that author had changed his or her opinion in subsequent publications (this is not unusual). I did so by entering authors’ names into online index databases such as ICLL and Legaltrac, and they provided me with typically exhaustive lists of authors’ publications. I soon possessed a rather large list of potentially useful books and articles.
It was only now that I repeated my initial (and at that time mistaken) step of conducting online keyword searches. I carried out this search in order to catch secondary sources that my reviews of footnotes, bibliographies, reference citations and authors’ names may have missed. I used such online search engines and databases as Google Scholar, ICLL, Legaltrac, HeinOnline, SSRN, Westlaw, Quicklaw, Lexis-Nexis and both my university’s library websites. Keyword searches were unreliable because authors used many different words, phrases and jargon to describe things. I accordingly made my search terms as short and vague as was practicable (using text searches such as “’wrongful death’ and damages” or “tort /7 damages and discrim*” or “life AND [economic or monetary] AND value”). I once again encountered the same avalanche of data that I had previously. However, this information was now much easier to navigate because I had already found my research focus. I could accordingly evaluate the efficacy of books, articles and other academic commentary much more quickly and efficiently. I ended my literature reviews of various research topics after I repeatedly encountered the same articles.
Articles that made my C list included those that discussed wrongful death legal history, tort law theory, historic and current valuations human health and life, and the socioeconomic impact of current tort damages law. Whenever I found potentially relevant articles or books online, I cut and pasted their summaries and/or important excerpts. If my sources were only available in print, I typed brief summaries and excerpts. I admit that I initially suspected that this exercise was a waste of my time and effort. However, my C list has since proved itself to be an indispensible resource. First, it served as a historical record of my literature search, so it greatly reduced my incidence of duplicate searches. Second, my bibliography continued to function as an important research reference tool. I, like most graduate students, periodically redirected my thesis’ focus. Now I consulted my C list first whenever I embarked upon a new search, and I almost always found some useful articles. For instance, I reviewed my C list after I decided that my thesis would discuss socioeconomic implications of wrongful death damages law. I found, in my cursory fifteen-minute search, four heretofore seemingly unimportant articles and a book that ended up on my final A list. Both Martha Chamallas’ article “The Architecture of Bias: Deep Structures in Tort Law” and Tsachi Karen-Paz’ article “An inquiry into the Merits of Redistribution through Tort Law: Rejecting the Claim of Randomness;” proved valuable because they each explain how tort law’s gender bias devalues womens’ lives and labour. I selected Viviana A. Zelizer’s book, Pricing the Priceless Child: the Changing Social Value of Children, because she examined the history of young children’s economic, emotional and legal worth in British and American cultures. I was particularly fascinated by David K. Rees’ article “Blind Imitation of the Past: An Analysis of Pecuniary Damages in Wrongful Death Actions,” because he claimed that wrongful death laws were a relic of the nineteenth century, and that they continued to devalue children’s lives in the same outdated manner. David Rees’ theory about wrongful death law’s obsolescence so caught my interest that I briefly scanned my C list for another article that elaborated upon his argument. Within two minutes I had found what I sought.
My B list contained about twenty titles by the time I had completed my literature review. Titles in my B list changed with the focus of my thesis research. For instance, I removed four articles that discuss wrongful death law’s ancient history after changing my focus to wrongful death’s socioeconomic history over the past one hundred and sixty years.
I began selecting sources for my A list bibliography after I had completed all of my C list scanning. I accordingly began reading – for the first time. I chose eleven items that I thought were particularly relevant to my thesis for my A list. These sources include the four articles and one book I have already mentioned. I additionally selected two articles by authors who disagreed with Chapman, and whose opinions about tort damages were similar to my own. Aristotle’s The Nicomachean Ethics also made my A list because it is universally recognized as the original source of Aristotelian corrective justice theory. I felt that I should have a first-hand knowledge of The Nicomachean Ethics if I were to effectively evaluate the many academic articles that debate its meaning. Now I reviewed my entire C list to make sure that I had not forgotten any other important sources. This exercise netted me one more article. My final A list therefore contained twelve titles.
My last step was an annotated bibliography of my A list sources. Once again, I suspected that compiling such a detailed annotated bibliography was a time-waster; and, once again, I was proven wrong. I had only a general notion of the shape that my thesis would take when I started annotating my bibliography. By the time I had finished the annotations, however, I had a very specific idea of how my thesis would look. My annotated bibliography will now become the literature review section of my thesis.
Once you have read this chapter, you should be able to:
- Locate different scholarly networks and sub-networks related to your research question
- Recognize the pitfalls that derail many researchers in their literature review search
- Distinguish between carefully reading a source versus scanning it
- Follow the ten steps necessary to create a manageable and workable literature review
- Decide what constitutes a source that is both relevant and interesting
- Make judgement calls about where to place your sources in either “A”, “B” or “C” lists
- Take proper notes (annotations) of your various sources