My thoughts and opinions for Bayesian Machine Learning.

I want to document my journey in Machine Learning where I stamp interesting frameworks and why I find them interesting. A reminder to self: Great ideas do not come from nowhere. They are derived from fundamental concepts, collaboration, and lots of trial and error.

10/11/2025: Probabilistic Circuits (why Tractable Inference?)

A generative model encodes a joint distribution $p(X)$ over many variables $X$. Even answering a simple equestion like “What is the probability that it is raining today and Anabel is going to eat hotpot soup?” requires computing a marginal probability: $p(E=e) = \sum_H p(e,H)$ where $E$ is an observed event and $H$ represents all other hidden variables. For general graphical models, this summation is #P-hard, meaning it is computationally intractable in the worst case.

This motivates the design of tractable model classes - models for which inference is guaranteed to be efficient. I attempt to summarise key ideas in my notes below on Probabilistic Circuits, a line of work driven by Guy van der Broeck and collaborators that advances tractable, rather than approximante, probabilistic inference. Here are the resources:

• UCLA StarAI Lab’s 3 hour Lecture: The first two hours of this provides a great introduction to PCs (if you have a nice intuition on probability on marginals and conditionals). YOUTUBE Video
• Anji Liu’s Notebook Tutorials: Am personally still navigating this PyJuice framework myself. PyJuice Tutorials
• My notes on PCs: Anabel’s Introduction to PCs
• Relevant Concepts: Logic Gates (useful to know AND/OR/XOR nodes), Sum-Product Networks (SPNs).

31/10/2025: Bayesian Optimal Experimental Design (why Expected Information Gain?)

I wanted to investigate the origin story for InfoBAX, have a short introduction on BOED. How and why the ideas of info-theoretic BO came around. Useful Resources:

• Bayesian Optimal Experimental Design notes: Desi Ivanova does a great simple introduction and visualization to BOED. Here: Desi’s 7 min Read
• InfoBAX: Willie Neiswanger does a great talk on InfoBAX I really loved. Here: Willie Neiswanger’s Presentation
• My thoughts on InfoBAX: InfoBAX_insights
• Relevant Concepts: Lindley’s EIG, Hennig’s Entropy Search, Krause’s Mutual Information Maximization, A Geneticist’s Approximate Bayesian Computation demonstration.