The Bayesian Thinking

11 Jun 2018

Prologue To The Bayesian Thinking

Bayesian thinking is an approach to systematizing reasoning under uncertainty by means of the Bayes theorem.

The Intuition Behind The Bayes Theorem

[1] ReCap the Bayes theorem

The detailed explanation of 4 factors are in my prior post, The Bayes Theorem Significance.

[2] The intuition behind the theorem

The intuition behind encourages you to make further inference.
➀the hypothesis, mapped to the prior, which are all probabilities.
➁the likelihood function related to prior is expressed as the probability of the event occurrence of the observation given the event occurrence of hypothesis.
➂the total probability of the observation is the well regularized evidence.
➃the posterior is the probability of the hypothesis, given the observation.

[3] The Bayesian inference

➀at the first glance, we make an observation in the real world.
➁we’d like to identify it by making certain hypothesis of some classification.
➂the likelihood function estimates the possible probability of the observation given the hypothesis.
➃finally, the posterior is the probability of the hypothesis given the observation.
Such process is called the Bayesian inference, full compliant with the classification of an observed object, which the hypothesis is made all about.

By the way, observation, hypothesis, likelihood function are all based on the qualitative belief, the total probability of the observation and the posterior are the quantitative outcomes.

The Bayesian Inference Illustration

My illustration in this article was inspired from Introduction to Bayesian Thinking: from Bayes theorem to Bayes networks, Felipe Sanchez, it is using an example from The Anatomy Of Bayes Theorem, The Cthaeh. But, I have some different opinion.

[1] Begin by a question

➀suppose everyone could casually find some mass in your body, like skin. It might be a rare disease, according to the medical library, only 1 from 1000 people having a mass would be the cancer, given in below table.

	Probability
Cancer	0.001
Mass	0.999

This table reveals the already known prior, now turns into be the hypothesis of the probability of having a cancer.
➁suppose the accuracy of the medical detection is given in below table, where malignant stands for cancer of result, and benign stands for being detected as a normal mass.

	Cancer	Mass
Malignant (Cancer)	$P(Malignant\vert Cancer)$=0.99	$P(Malignant\vert Mass)$=0.01
Benign (Not a cancer)	$P(Benign\vert Cancer)=0.01$	$P(Benign\vert Mass)$=0.99

This table directly reflects the possible likelihood for all conditional combinations of 2 observations, malignant and benign.
➂unfortunately, you are detected as having a cancer, then, what's the probability that you are really having a cancer given that you are medically detected as a victim of cancer?
This given question is asking for $P(Cancer\vert Malignant)$, which is the posterior.

[2] Test of run #1

By the given hypothesis, likelihood, the Bayes theorem could be used for the posterior:
➀$P(Cancer\vert Malignant)$
=$\frac {P(Malignant\vert Cancer)\cdot P(Cancer)}{P(Malignant)}$
➁the total probability of malignant evidence:
$P(Malignant)$
=$P(Malignant\vert Cancer)\cdot P(Cancer)$+$P(Malignant\vert Mass)\cdot P(Mass)$
➂therefore, the posterior is
$P(Cancer\vert Malignant)$
=$\frac {0.99\cdot 0.001}{0.99\cdot 0.001+0.01\cdot 0.999}$=$0.090163$
; where $P(Mass\vert Malignant)$=$0.909837$, take it as $0.91$ after rounding.

[3] Test of run #2

Even if the accuracy of the medical detection is up to $0.99$, the probability for your mass is really a cancer given the malignant diagnostic result is only $0.09$. That’s why we decide to make the 2nd test.
➀first, we update the prior table with regard to the given run #1 result:

	Probability
Cancer	0.09
Mass	0.91

It is under the assumption that the run #1 is rather a plausible, not a vague result!!
➁recalculate with the Bayes theorem:
$P(Cancer\vert Malignant)$
=$\frac {0.99\cdot 0.09}{0.99\cdot 0.09+0.01\cdot 0.91}$
=$0.90733$
$\approx 0.91$
; where $P(Mass\vert Malignant)$=$0.09266\approx 0.09$, after rounding. Wow, it seems there is a great improvement in a malignant report and you do really have a cancer.

[4] Test of run #3

Let’s do it the 3rd run.
➀first, we update the prior table with regard to the given run #2 result:

	Probability
Cancer	0.91
Mass	0.09

It is under the assumption that the run #2 is rather a plausible, not a vague result!!
➁recalculate with the Bayes theorem:
$P(Cancer\vert Malignant)$
=$\frac {0.99\cdot 0.91}{0.99\cdot 0.91+0.01\cdot 0.09}$
=$0.999$
$\approx 1$
; where $P(Mass\vert Malignant)$=$0.0001$, after rounding.
It is now almost $100\%$ correct that the malignant report says that you have a cancer!!!

[5] Summary

This illustration begins with the given prior of having cancer, executes from run #1 to run #3, constantly updates the next prior probability with the current estimated posterior, finally get the expected result. It is called the Bayesian inference.

The Review::mjtsai1974

Above illustration of Bayesian inference might strike you on your head that by constantly updating the given prior(so that you can make finer hypothesis) would you gradually adjust the posterior(the experiment result) toward the direction you want.

[Question]

Below I comment out with 2 doubtable points:
➀why we update the prior, $P(Cancer)$ with $P(Cancel\vert Malignant)$ after each test?
➁is this the artificial bias leads to the contribution of $100\%$ identification of having a cancer given the malignant result?

[Answer]

➀I think it is indeed an artificial bias, since the term $P(Cancer\vert Malignant)$ is not equivalent to the very first given $P(Cancer)$ for all possible diseases one can have as a sample or population.
➁be remembered that it is the common practices in Bayesian inference.

Addendum

➀Introduction to Bayesian Thinking: from Bayes theorem to Bayes networks, Felipe Sanchez
➁The Bayesian trap, Veritasium channel
➂The Anatomy Of Bayes Theorem, The Cthaeh