Onora O’Neill (2002) contends that advocates of media freedom have erroneously equated the citizen’s right to information and expression with press freedom. They have claimed for journalists and media organizations what is essentially an individual right reserved for citizens. A free media, according to O’Neill, “is not an unconditional good… Good public debate must not only be accessible to but also assessable by its audiences.”
Accessibility is often measured through indicators that quantify access to various media, such as newspaper circulation or the number of TVs, radios, and computers per thousand people in the population (e.g., UNESCO, World Bank). Assessability, on the other hand, is driven by normative standards and can be carried out on at least two levels.
First, assessment can be conceptualized as “diagnosis” of the current state of the media, given a framework of enabling components (e.g., laws and regulation), characteristics, and opinions regarding the present state of media systems, including problems that need to be addressed. Second, assessment can be conceptualized as Monitoring and Evaluation (M&E), guided by a clear view of what media systems ought to look like and whether they fulfill the functions they ought to perform. According to Lee Becker, et al. (2007), there are more than 100 organizations worldwide that carry out diagnosis and performance assessment of media outlets, media support organizations, and environments in which media operate.
Some cross-national indices of press freedom widely used by the policy and media development communities include Freedom House’s Freedom of the Press (FH) , Reporters without Borders’ Worldwide Press Freedom Index (RSF) , Committee to Protect Journalists’ Attacks on the Press , and International Research and Exchanges Board’s Media Sustainability Index (IREX).
In terms of global scope, the FH and RSF indices are the most extensive. They have been used as empirical tools for diagnosing media environments, policy dialogue, and decision-making. This strength in application, however, is countervailed by a key weakness: the lack of a clear process that moves from theory, to conceptualization, to operationalization. To fill this gap, one study makes an attempt to draw out conceptualizations, post hoc. Lee Becker, Tudor Vlad, and Nancy Nusser (2007) claim that the concept that FH operationalizes and measures is “freedom of the media…defined as the legal environment, political pressures that influence reporting, and economic factors that affect access to information” (p. 9). RSF, say the same scholars, uses the concept “respect for media freedom, defined as ‘the amount of freedom journalists and the media have in each country and the efforts made by government to see that press freedom is respected’” (p. 11).
A criticism one might level against these indicators is that they are elite-driven, taking into account expert views, but not public opinion. In response, the United Nations Educational, Scientific, and Cultural Organization  (UNESCO) recommends the inclusion of public opinion data through its media indicator framework (Puddephatt, 2008). An initial stab at this, done by Becker and Vlad (2009), finds positive associations between elite and citizen evaluations of press freedom in their own countries, comparing public opinion data from WorldPublicOpinion.org and elite measures from FH, RSF, and IREX.
These indicators have also been criticized for reflecting a strong Western bias. Treating these indices as barometers of success or failure of media systems in different country contexts runs the risk of advocating standards that are not locally owned, especially since the largest organizations which generate and promote the use of these indicators are based in the West. Christina Holtz-Bacha (2005) writes that “press freedom is understood differently in the various parts of the world… even established democracies do not interpret press freedom in exactly the same way” (p. 2).
Holtz-Bacha questions the view that political party ownership (e.g., Scandinavia) and state support to media outlets (e.g., France) are necessarily opposed to media freedom. In Freedom House’s Freedom of the Press, low scores on these dimensions reduce country rankings. Another example of disagreement regarding the definition of media freedom is the U.S. preference for free market safeguards versus most Western European countries’ provision of state financing to public service broadcasters in support of diverse programming (ibid.).
Becker et al. (2007), however, found that even though FH and RSF are headquartered in different countries and continents, the U.S. and France, respectively, their “… measures all seem to be measuring the same things” (p. 16) with positive correlations between recent waves of the two indices being quite strong (r=.81 in 2002 and .84 in 2003). In 2005, the positive relationship between the two indices was still high (r=.84, p<.01).
Still, definitions of media freedom are not universal and there are serious dangers of overemphasizing ideological positions and their corresponding policy preferences. This underscores the importance of carefully deliberating the priorities of the media located in different country and subnational contexts. This also means carefully theorizing, conceptualizing, and operationalizing indicators of success and choosing appropriate research methods. What we mean by "evaluation" ought to include cittizen opinions, and not be limited to consulting elites. For these reasons, and more, the global community of practice and relevant policy networks should consider revisiting how media systems are diagnosed as well as monitored and evaluated via broad-based consultation and deliberation.
Photo credit: Flickr user Hong Kong dear Edward