Double blind, randomized, drug trials are considered the “gold standard” of drug testing despite many conceptual problems. Over time, the legal requirement in the United States for “adequate and well controlled investigations” (1962) was interpreted by the FDA to mean “randomized” and “controlled” trials. While the public believes the clinical trial to be a model of scientific method, the profession’s attitude may be more realistic. Although this method was introduced as an improvement on physician’s “testimonials,” close investigation shows that it may not have done away with the problems of the former, whimsical method for ensuring a drug’s safety and efficacy. In a 1987 survey of several hundred specialists, 70% agreed that RCT were corrupted by too many purely commercial trials and that trial protocols are often inadequate. Over 50% felt that the results of RCT are often ambiguous and uninterpretable.
The (seemingly) intractable problem of human variety highlights one of the greatest pitfalls of RCT, which do not account for this basic medical phenomenon. Just focusing on one point of difference between individuals – – metabolism – – we see how difficult it is to factor diversity into a randomized study. Human metabolism, which is critical to a drug’s effectiveness changes with age, sex, race, diet, season, time of day, blood type, etc. According to Harris Coulter (PhD), in The Controlled Clinical Trial; An Analysis, human variety is a fundamental issue in the “controlled clinical trial,” which will be scientifically valid “only if it can accommodate, and allow for, the heterogeneity of human sickness. If this heterogeneity is ignored or assumed out of existence, the “controlled clinical trial” must be judged a failure” (13). The paradox is that if heterogeneity of human sickness were accommodated, the applicability of RCT would be obsolete: “The more heterogeneous the sample, the harder it is to distinguish the outcome in the test group from that in the controls, and the more difficult it is to demonstrate efficacy of the new treatment. The well- conducted trial, which recognizes patient heterogeneity and accommodates it by a high degree of stratification, may yield much knowledge of the biology of the given disease but have little or no impact on practice” (52).
Harris goes on to argue that the physician’s way of coping with the problem of heterogeneity in medical practice – – by categorizing individuals according to various disease entities – – does not provide a firm foundation for a structure of scientific medicine. Even though disease names are in steady use, they correspond to nothing in nature other than vague and shifting similarities among patients. He finds it unwise to base the country’s whole system for vetting new medicines upon such an inherently unstable concept as the “disease entity” (27).
The unfounded assumption that the material of a RCT is reasonably homogeneous renders these trials completely unsound by any scientific standards. Robert Platt summarizes this view: “We are inclined to underestimate the extent of biological variation, which is such that a controlled trial is not always possible” (6). Even if the problem of heterogeneity could be overcome and it was possible to find a generalizable sample, the question of who makes these evaluations can be raised. It is difficult to deny the irreducible (sometimes legitimate) differences of medical opinion, reflecting the physician’s particular area of specialization and other factors. Observer bias is a well-known factor and the comparative element of diagnoses is difficult to eliminate. No scientific method can be better than its raw data, yet the factors of judgment that impact RCT are little considered. The same problem occurs in assessing “cure.”
These difficulties show that it is impossible to reach the standards that would render the results of a RCT meaningful: “Not only must the patient sample be internally homogeneous, it must also accurately represent all the patients suffering from the “disease” in question. Otherwise the results obtained with the sample will not be applicable to any larger group.” Yet the results obtained from these studies ARE applied to a larger group, often at great cost.
Reinforcing the conceptual problems of RCT, there is a basic paradox between homeogeneity and generalizability – – called the “therapeutic paradox”: The greater the internal homogeneity of the sample, the more precisely it will distinguish treatment from no treatment, but the less representative it will be of any larger patient population. The treatment tested, therefore, will be distinguishable from the placebo but will have minimal practical application. And, vice-versa, the less homogeneous the sample, the less capable it will be of distinguishing treatment from no treatment, but the better it will represent the real-world population of patients with the given condition. Therefore, while the benefit of any new therapy will be difficult to demonstrate, if it can be demonstrated, it will have very broad application. In order to be applicable, a RCT must have BOTH of these qualities (already shown to be impossible in themselves), but if one is attained, the other is sacrificed.
The demand for radomization, which is desirable to avoid biasing the outcome and neutralize possible unknown contaminating variables which have not already been accounted for through stratification is something that is easier to achieve in principle than in practice. The freedom of choice given to the physician in selecting patients makes the unreachable goal of randomization difficult to even approach.
These conceptual problems alone expose the tenuous basis on which the efficacy of pharmaceuticals is assured. Factor in political, economic, and social interests and the situation becomes even more problematic. The conceptual difficulties with RCT leaves drug tests open to corruption, enabling a health care system where the rich get richer and the sick get sicker.