Shadows, Stencils, Shaders howto

Jun 30, 2009 at 2:36 AM

Hi Bryan,

I was wondering if you give a few pointers to let we mere mortals have a crack at the shadows on the XBOX.

I have searched now for the umpteenth time for those pointers you gave on where to attack the code, just can't find them.


Any chance of a reiteration for the forgetful.

Kind regards,


Jun 30, 2009 at 9:51 PM

It's in this thread right here maybe 2/3 of the way through -

Good luck!

Jul 1, 2009 at 12:16 AM

For the Interested, this is a path I may follow:

DirectionalLight along with DirectionalShadow are the types that implements the shadowing behavior. It would be easier I think to simply modify the code in-place rather than replacing the type. Read through both classes and you'll immediately see why shadows don't appear on the Xbox :) You can get them to show up by using a different surface format in DirectionalShadow, but they won't draw correctly.

The problem is that I implemented VSM (Variatic Shadow Mapping). This, AFAIK, requires the use of an Rg32 surface format. This works fine on windows, but as you'll notice, will not work on the Xbox. This is because the Xbox does not have the Rg32 surface format available. It has been proven, though, that ESM (Exponential Shadow Mapping) works really well since it uses (I think) Single surface format and has far less artifacts than VSM. So, ya, implement ESM :) Or, maybe you can find a way to make VSM to work on a format that works on both platforms.

Implementing ESM will require you to modify the baseShadowReceiver.fx, baseLightReceiver.fx, and SkinnedModelEffect.fx in the XNAnimation source code. You'll also have to rebuild XNAnimation after pushing the new shader byte code to SkinnedModelBasicEffectCode.cs.)

There will probably be other places you'll discover the need to touch as well. 

Tips and pointers anyone?

Many thanks.

And, yes, I will share of course. ;)


Jul 1, 2009 at 12:17 AM
Edited Jul 1, 2009 at 1:27 AM

There is no need to invoke Markov’s inequality, higher order moments or convolutions. In fact all we have to do is to write the basic percentage closer filtering formula for n equally weighted occluders o_i and a receiverr


The role of the step function H(x)  is to perform a depth test on all occluders, depth test results are then averaged together to obtain a filtered occlusion term. The are many ways to write H(x)  and a limit of exponential functions guarantees a fast convergence:

\displaystyle H(o_i-r) = \lim_{k \to +\infty} \frac{e^{ko_i}}{e^{ko_i}+e^{kr}}

We can rewrite the original PCF equation as:

\begin{array}{ccc} \displaystyle\frac{1}{n}\sum_{i=1}^{n}H(o_i-r)&=&\displaystyle\frac{1}{n}\sum_{i=1}^{n}\lim_{k \to +\infty} \frac{e^{ko_i}}{e^{ko_i}+e^{kr}} \\ &=&\displaystyle\lim_{k \to +\infty}\frac{1}{ne^{kr}}\sum_{i=1}^{n}\frac{e^{ko_i}}{e^{k(o_i - r)}+1} \end{array}

If we make the hypothesis that our shadow receiver is planar within the filtering window we are also implicitly assuming that the receiver is the most distant occluder (otherwise it might occlude itself, which can’t happen given our initial hypothesis), thus we have r > o_i.
Armed with this new assumption we observe that the term e^{k(o_i - r)}  quickly converges to zero for all occluders:

\begin{array}{ccc}  \displaystyle\lim_{k \to +\infty}\frac{1}{ne^{kr}}\sum_{i=1}^{n}\frac{e^{ko_i}}{e^{k(o_i - r)}+1} &\approx&\displaystyle\lim_{k \to +\infty}\frac{1}{ne^{kr}}\sum_{i=1}^{n}e^{ko_i} \\ &\equiv&\displaystyle\lim_{k \to +\infty}\frac{E[e^{ko}]}{e^{kr}} \\ \end{array}

As we already know k controls the sharpness of our step function approximation and can be used to fake soft shadows. Ultimately we can drop the limit and we obtain the ESM occlusion term formula:

\displaystyle \frac{E[e^{ko}]}{e^{kr}}

Exponential shadow maps can be seen as a very good approximation of a PCF filter when all the occluders are located in front of our receiver (no receiver self shadowing within the filtering window). There’s not much else to add, if not that this new derivation clearly shows the limits of this technique and that any future improvements will necessarily be based on a relaxed version of the planar receiver hypothesis.

I have got some work around the subject. But would appreciate any assistance that may be offered.


Jul 1, 2009 at 4:17 AM

Ok I got 3 different ESM implementations working in FX composer.

They use SHADOW_MAP_FORMAT "r32f" so should be ok according to MSDN

Now to just reverse engineer all your work...



Here it's ESM working in FX composer.

I sure hope the community is feeling helpful...



Aug 10, 2009 at 2:25 PM

Been gone from these forums for awhile.  Back on the shadows attack now.  Did you ever get this to work simon?  Standing by to help get this working.

Aug 10, 2009 at 6:49 PM

Hi as well Simon!

The API file is pretty out of date now. I'm wondering if you can pull down the source head and regenerate it. I would do so myself, but my computer just doesn't have the requisite RAM :(


Aug 10, 2009 at 10:21 PM
Yep. will do sir.

From: [email removed]
To: [email removed]
Date: Mon, 10 Aug 2009 10:49:20 -0700
Subject: Re: Shadows, Stencils, Shaders howto [OxGameEngine:61014]

From: bryanedds
Hi as well Simon!
The API file is pretty out of date now. I'm wondering if you can pull down the source head and regenerate it. I would do so myself, but my computer just doesn't have the requisite RAM :(