Search⌘ K
Home
Courses
Fixing Random: Techniques in C#

Problems with Discrete Probability Distributions

Understand the challenges of implementing and optimizing discrete probability distributions in C#. Learn about the limitations of integer weights, inefficiencies in sampling workflows, and the constraints of small support distributions. This lesson prepares you to handle the complexity of stochastic programming beyond simple discrete cases and explores the trade-offs between inference accuracy and computational cost.

We'll cover the following...

Some Issues with Discrete Probability Distributions

So… we have good news and bad news.

The good news is:

  • We’ve described an interface for discrete probability distributions and implemented several distributions.
  • We’ve shown how projecting a distribution is logically equivalent to the LINQ Select operator.
  • We’ve shown how conditional probabilities of the form P(B∣A)P(B|A) "probability of BB given AA" are likelihood functions.
  • We’ve shown that combining a prior with a likelihood to form a joint distribution is the application of the monadic bind operator on the probability monad, and implemented this as SelectMany.
  • We’ve shown that computing the posterior distribution from the joint distribution can be done by applying a Where clause.
  • We’ve sketched out a statement-based workflow DSL that can be compiled down to our LINQ operators, and that lowering and then executing programs written in this DSL can compute a “sampling-loop-free” distribution automatically.
  • And that distribution object can then be efficiently sampled.

This is all great. But the bad news is that there are some problems; let’s list them in order roughly from smallest to largest:

First Problem

It’s often inconvenient to express weights as integers, particularly when you end up with largish co-prime weights; we’re risking integer overflow when we multiply or add them, and we lack facilities for common operations like “make me a ...