Bayesian probability resources
You may have heard a lot about 'Bayesian' probability lately, for instance, in Paul Graham's and CRM114 anti-spam measures, and wonder what is the difference between regular probability theory and bayesian probability theory. The short answer is one of philosophy: probability you are taught in school (called frequentist by bayesians) takes the silly view that you are always thinking of the long-term; that, if you did the experiment a thousand times, what would be distribution of results, whether it makes sense to run the experiment a thousand times or not. In general, this approach is chock-full of ad-hockery, with no fundamental justification for the methods used. Bayesian Probability Theory is reasoning under uncertainty, or determining the plausibility of statements when we lack sufficient information to determine their truth or falsehood. (I'm not talking in general terms here, treat each word in the last sentence literally.) Freqeuncy-based probability theory is a special case of reasoning under uncertainty. BPT is a generalization of Boolean logic.
Long answer? Read these :) I have listed here only online resources for poor students like me:
- An Intuitive Explanation of Bayesian Reasoning by Eliezer Yudkowsky (famous for singularity stuff) this is ok, but you will want more
- Principles of Data Analysis by Prasenjit Saha, a nice short book that touches on several topics.
- From Laplace To SN 1987A: Bayesian Inference In Astrophysics by Tom Laredo, wonderful introduction to bayesian thinking, (you don't need astrophysics to read it.)
- Probability Theory: The Logic of Science by ET Jaynes. This (sadly incomplete) book explains the motivation for bayesian thinking from first principles, showing that probability theory is unique and consistent; it is the only correct method of plausible reasoning. The dead tree version will have the missing bits filled in (i believe) by G. Larry Bretthorst.
- Information Theory, Pattern Recognition and Neural Networks by David MacKay, a very good book that explains the deep connection between information theory and probability. The exercises are great, and vividly illustrate the points of the chapters. Read the others for understanding, and this one for development and application. The focus of the book, though is not on probability exclusively, so, while I can't recommend it enough, i also can't recommend it without at least some of the other resources.
well, i'm a student myself, so i may have made mistakes above. However, all the authors listed are well-respected and you can trust them to guide you.
main