My thoughts and opinions for Bayesian Machine Learning.

I want to document my journey in Machine Learning where I stamp interesting frameworks and why I find them interesting. A reminder to self: Great ideas do not come from nowhere. They are derived from fundamental concepts, collaboration, and lots of trial and error.

10/11/2025: Probabilistic Circuits (why Tractable Inference?)

Project Image 3 A generative model encodes a joint distribution $p(X)$ over many variables $X$. Even answering a simple equestion like “What is the probability that it is raining today and Anabel is going to eat hotpot soup?” requires computing a marginal probability: $p(E=e) = \sum_H p(e,H)$ where $E$ is an observed event and $H$ represents all other hidden variables. For general graphical models, this summation is #P-hard, meaning it is computationally intractable in the worst case. This motivates the design of tractable model classes - models for which inference is guaranteed to be efficient.

Anji Liu’s Notebook Tutorials: Am personally still navigating this PyJuice framework myself. PyJuice Tutorials
My notes on PCs: Fundamentals in Probability for PCs, Anabel’s Toy Example for PCs

31/10/2025: Bayesian Optimal Experimental Design (why Expected Information Gain?)

I wanted to investigate the origin story for InfoBAX, have a short introduction on BOED. How and why the ideas of info-theoretic BO came around. Useful Resources:

Bayesian Optimal Experimental Design notes: Desi Ivanova does a great simple introduction and visualization to BOED. Here: Desi’s 7 min Read
InfoBAX: Willie Neiswanger does a great talk on InfoBAX I really loved. Here: Willie Neiswanger’s Presentation
My thoughts on InfoBAX: InfoBAX_insights
Relevant Concepts: Lindley’s EIG, Hennig’s Entropy Search, Krause’s Mutual Information Maximization, A Geneticist’s Approximate Bayesian Computation demonstration.