You've probably heard the saying "Extraordinary claims require extraordinary evidence." It's the starting point for perhaps the most common argument by atheists: "The existence of God is an extraordinary claim that lacks extraordinary evidence." Seems logical, right? The only problem is, how can we determine whether the evidence (or the claim) really is extraordinary?
A much more scientific way to formulate "Extraordinary claims require extraordinary evidence" is via Bayes' theorem. In Bayesian terms, an extraordinary claim is a hypothesis with a very low prior probability (e.g., “a coin flipped 5 times will land on tails every time”, which has a prior probability of around 3%). It follows that very strong evidence is required to move the probability high enough to believe the claim. Thus, it can be shown mathematically that extraordinary claims (defined this way) do in fact require extraordinary evidence. In the above example, that evidence could be a measurement that the coin's weight is very unbalanced or an observation that it has tails on both sides.
Applying that framework to the God claim, the strength of evidence required depends on a priori assumptions about the prior probability that God exists. Theists who start with a relatively high prior probability require less evidence. Atheists who start with a low prior require more evidence. Arguments about the sufficiency of the evidence for God become circular on both sides. Thus, it's imperative that we have a good, objective way to determine the prior probability.
Because we don't have specific, definite probabilistic information about the God question, we must use an uninformative prior. The simplest and probably most common of these is the principle of indifference, which says the prior probabilities of all hypotheses are equal. In the binary case of “Does God exist?”, the prior is 50%. Starting with a 50% probability may seem crazy if the claim seems ridiculous, but it makes good sense mathematically. The evidence (or lack thereof) is probably what makes such claims seem ridiculous in the first place, and the other terms in Bayes' rule account for that. Also, if the claim seems ridiculous to most people, that fact alone is evidence that would reduce the probability.
Using the principle of indifference, presuppositions about the probability of God's existence are eliminated as determining factors. The estimate of the probability that God exists now depends entirely on the evidence. In this case, “Extraordinary claims require extraordinary evidence” is a meaningless argument. It doesn't matter how extraordinary the claim is because the evidence will tell us whether to believe it. We'll still argue about the evidence and how to assign probabilities to it, but that's a lot more useful than debating a theist's circular argument vs. an atheist's circular argument.
There are other ways to determine uninformative priors, including some that let us use the “extraordinary claims” standard. But when applied to the God claim, they generally require arbitrary assumptions that lead to self-fulfilling conclusions. That might be good enough for testing the claim that I'm wearing three socks right now, but whether or not to believe in God is a much more important question – one that I don't think should be decided (either way) by arbitrary assumptions made before examining the evidence.