# Confidence intervals when using Bayes' theorem

I'm computing some conditional probabilities, and associated 95% confidence intervals. For many of my cases, I have straightforward counts of `x` successes out of `n` trials (from a contingency table), so I can use a Binomial confidence interval, such as is provided by `binom.confint(x, n, method='exact')` in `R`.

In other cases though, I don't have such data, so I use Bayes' theorem to compute from information I do have. For example, given events $a$ and $b$:

$$P(a|b) = \frac{P(b|a) \cdot P(a)}{P(b)}$$

I can compute a 95% confidence interval around $P(b|a)$ using $\textrm{binom.confint}(\#\left(b\cap{}a),\#(a)\right)$, and I compute the ratio $P(a)/P(b)$ as their frequency ratio $\#(a)/\#(b)$. Is it possible to derive a confidence interval around $P(a|b)$ using this information?

Thanks.

-
$a$ and $b$ are events. In my case, $a$ is a system failure (which is quite rare, so relatively hard to find "in the wild"), and $b$ is a pre-failure alarm, so I'm measuring the probability of failure given an alarm. – Ken Williams Aug 21 '12 at 18:56
The above comment was in response to someone who asked for more background on what $a$ and $b$ were, but seems to have deleted that comment. – Ken Williams Aug 21 '12 at 19:35
Well you can't just take the confidence interval for p(b|a) and scale it by p(a)/p(b) because of the uncertainty in the estimate of that ratio. If you can construct a 100(1-α)% confidence interval for p(a)/p(b) call it [A, B] then take the lower bound for a 100(1-α)% confidence interval for p(b|a) and multiply it by A and take the upper bound for p(b|a) and multiple it by B. That should give at an interval that has at least a 100(1-α)$^2$% confidence level for p(a|b). – Michael Chernick Aug 21 '12 at 19:37
Could work... getting a confidence interval for $P(a)/P(b)$ isn't obvious to me though - do you feel like moving this into the "Answer" area? I promise at least one upvote. =) – Ken Williams Aug 21 '12 at 21:46

2
Don't you want a Bayesian credible interval instead? That is directly computable from the posterior distribution of $a$. – whuber♦ Aug 21 '12 at 22:44
show 1 more comment

## 1 Answer

Well, you can't just take the confidence interval for $p(b|a)$ and scale it by $p(a)/p(b)$ because of the uncertainty in the estimate of that ratio. If you can construct a $100(1-\alpha)\%$ confidence interval $[A, B]$ for $p(a)/p(b)$, then take the lower bound for a $100(1-\alpha)\%$ confidence interval for $p(b|a)$ and multiply it by $A$ and take the upper bound for $p(b|a)$ and multiply it by $B$. That should give at an interval that has at least a $100(1-\alpha)^2\%$ confidence level for $p(a|b)$.

-

That seems workable at least as a first stab.  But I'm not aware of a method for deriving confidence intervals for the ratio of two Bernoulli probabilities $P(a)/P(b)$ given observed counts of $a$ and $b$ in a sample population. – Ken Williams Aug 21 '12 at 22:33
– Ken Williams Aug 22 '12 at 15:59
If I haven't erred, here's a function to do that ratio-of-Bernoullis interval estimation: ```binrat.confint <- function(x, y, n=Inf, m=n, p=0.95) {
  s2 <- 1/x - 1/n + 1/y - 1/m;
  x/y * exp(c(-1:1)*pnorm((1+p)/2)*sqrt(s2))
}
``` – Ken Williams Aug 24 '12 at 19:05
default
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 31, "mathjax_display_tex": 1, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9216858148574829, "perplexity_flag": "middle"}