Attacking the Final Problem
Explore advanced techniques for addressing challenging problems in continuous distributions through improved importance sampling and helper distribution adjustments. Understand how to stretch and shift distributions to enhance expected value estimates in C# programming, enabling better handling of uncertain or complex probability regions.
We'll cover the following...
In the previous lesson, we finally wrote a tiny handful of lines of code to implement importance sampling correctly; if we have a distribution p that we’re sampling from, and a function f that we’re running those samples through, we can compute the expected value of f even if there are “black swan” regions in p.
Helper Distribution q
All we need is a helper distribution q that has the same support as p, but no black swans.
Great. How are we going to find that?
A variation of this problem has also occurred before: what should the initial and proposal distributions be when using Metropolis? If we’re using Metropolis to compute a posterior from a prior, then we can use the prior as the initial distribution. But it’s not at all clear in general how to choose a high-quality proposal distribution; there’s some art there.
There is also some art in choosing appropriate helper distributions when doing importance sampling. Let’s once again take a look at our “black swan” situation:
As we’ve discussed, we contrived the “black swan” situation by ensuring that there was a region of the graph where the orange line bounded a large area, but the blue line bounded a very tiny area there.
First off: in our initial description of the problem, we assumed that we only cared about the function on the range of ...