This time around we are trying something new and we are cross-posting it on #AltDevBlogADay. It seems to me that most of the subscribers to The Witness blog are non-programmers, so this way it may reach a wider audience. Hopefully this will also encourage me to post more regularly!

I’ve been working on The Witness for a bit more than a year. So, I thought it might be a good time to write a bit about my experience so far, in the hope of attracting some talent to join us.

I have to admit that I had many doubts about coming to work at Thekla. I had a comfortable and stable job at NVIDIA, where I was held in high regard. The prospect of joining a small indie studio certainly seemed a lot more uncertain. However, Jon’s confidence in the game he was designing, and in his ability to support a small team during its development was very reassuring.

Work conditions at Thekla match or exceed those at my previous employments. Compensation is competitive and the work environment is extremely flexible. Jon has very high respect for our time and personal needs, I’m sure he would go an extra mile to make sure you are satisfied as well. There are also unusual perks that you would not get anywhere else in the industry.

Not only the benefits are great, but I in addition to that, I get to work on the stuff that I really enjoy. That’s not just the global illumination implementation that I’ve been writing about, but also many other interesting problems in my areas of interest (mesh cleanup, parameterization, optimization, simplification, exporters, image filtering, compression, etc). I get to design our whole production pipeline; how the game, the editor and the modeling tools integrate with each other. I architect and implement most of the engine; its rendering pipeline, the level of detail and streaming systems, the platform abstractions, etc.

I think that we currently have a good foundation. Our main focus has been on quick prototyping. As the codebase became more complex I’ve tried to get things better organized, while still retaining our quick iteration times. Our builds are blazing fast, and the loading times are still fairly quick. The code is also designed to make it easy to prototype new gameplay elements, and while the rendering code has become more rigid, it’s still pretty easy to experiment and add new visual effects.

There’s still plenty of work left to be done, which is why we need an extra hand. The game needs lots of polishing and some optimizations. There are still a few gameplay elements that need to be prototyped, and as we ramp up into production we will probably face unforeseen problems that will need to be addressed. We also plan to port the game to other platforms, some of which might require significant reengineering.

However, for me, this job is not only attractive just because I get to work on cool and challenging problems. The Witness is a game unlike any other. It’s a unique creative opportunity. It’s an opportunity to use my skills to make a difference. To make something unique and valuable. To influence people in positive and meaningful ways.

As you can tell, I’m very excited to be part of it. If you would too, or you just want to learn more, don’t hesitate to get in touch with us!

Most importantly, stop harassing the guy whose name is on the patent (Lars): he’s just a programmer, not the management or lawyers responsible for filing the patent. This is stupid and unnecessary. I’ve removed the original post because of this; it can be found here for those who want to read it.

I don’t know much about this particular case, I don’t know whether Lars came up on his own with the algorithm that is being patented or not, and I’m not really qualified to discuss that.

But that’s not what I want to write about. What really struck me from this post is the idea that programmers are not to blame for filing software patents. I think that’s just wrong.

Lawyers alone cannot create patents, you also need inventors, and being just a programmer does not absolve you of your acts. Let me rephrase that: There would be no patents without inventors willing to file them.

Obviously, corporations provide incentives for employees to file patents, but in most cases it’s not the actual incentives what motivates people to patent their inventions. It’s the benefit of being a good corporate employee, not being considered a trouble maker, not loosing opportunities for promotion, not bringing negative attention to yourself, not going against the tide.


I firmly believe that patents discourage progress and impede the growth of public domain of knowledge. Today patents do not protect the inventor’s interests, but instead promote anti-competitive practices by corporations. Moreover, most software patents are vague, bogus, or trivial, they do not serve any social purpose other than expanding the patent portfolio of your corporate masters.

You may not agree with all that, but if you do, then don’t excuse yourself blaming the system.


During the 5 years that I worked at NVIDIA I constantly came up with algorithms and software ideas that could be patented. I implemented many of them, others I simply outlined.

So, I started a wiki page in which I documented these ideas. The goal was to prevent others from patenting them. I called them anti-patents. I usually came up with a new one every month, sometimes several.

This might seem exaggerated, but when you are designing new hardware features that no one has explored, it’s very easy to come up with new things to do with it that nobody has done before. I think this is true for almost any field when you are working on the bleeding edge.

Eventually I stopped maintaing the wiki, it was too much work to describe them in detail, and in many cases I considered them trivial. In spite of that I believe that most of them would have been pursued by NVIDIA if I had chosen to allow it.

Over time I ascended in the corporate hierarchy until I became part of a selected group in charge of the design of future GPUs.

I was working with people much more experienced and smarter than me. I didn’t want to get noticed for causing trouble, but for doing a good work. So, predictably my name ended up in several patent applications.

Today, I deeply regret that.

In any case, he managed to work around the problem and posted a video showing the results:

I’m glad that despite the bugs somebody is actually making use of that code and finding it useful! Cool work Matt!

• The Witness – Hemicube Rendering and Integration, September 29, 2010
• The Witness – Texture Parameterization, March 30, 2010

precise affects all results that contribute to the variable’s outcome by preventing the compiler from doing unsafe optimizations. For instance, to improve performance the compiler ingores the possibility of NaN and INF results for floating point variables from constant and stream inputs in order to do several optimizations. By using the precise keyword, these optimizations will be prohibited for all calculations affecting the result of the variable.

Besides the typos (ingores?), it’s not very clear what “unsafe optimizations” actually means, since only a single example is provided. On the other side, the OpenGL specification is a lot more “precise”:

The qualifier “precise” will ensure that operations contributing to a
variable’s value are performed in the order and with the precision
specified in the source code. Order of evaluation is determined by
operator precedence and parentheses, as described in Section 5.
Expressions must be evaluated with a precision consistent with the
operation; for example, multiplying two “float” values must produce a
single value with “float” precision. This effectively prohibits the
arbitrary use of fused multiply-add operations if the intermediate
multiply result is kept at a higher precision.

OpenGL makes it clear that precise causes operations to be performed in the exact order prescribed by the language rules. It also highlights that the precision of the operations must also be exactly as prescribed and not greater, which rules out the use of fused multiply-add if implemented according to the IEEE 754-2008 specification, to which most modern GPUs conform.

Today we take watertight rasterization for granted. This is something that most of us don’t even have to think about, since the fixed function hardware takes care of it automatically.

On the other side, when using tessellation, triangle vertices are generated by programable shaders that run independently for adjacent patches, which usually have arbitrary parametric orientations. Obtaining watertightness of tessellated patches requires a lot of care to guarantee that vertices shared between adjacent patches are evaluated consistently. As opposed to rasterization, in this case the responsibility to obtain watertightness lies in the hands of the programmer.

In my first experiments it became clear that this was not only hard to achieve, it was simply impossible. Fortunately, at NVIDIA I had access to the whole software stack, so by using a custom Cg compiler, running with developer drivers, and turning some knobs, I could prevent unwanted optimizations from happening and confirm that it was indeed possible to obtain watertight results. On the other hand, this solution was not very efficient, what we needed was to have fine grained control over the way the compiler turned our expressions into code; preventing optimizations only in the situations where it mattered.

Among all the solutions proposed, the addition of a precise qualifier and an fma (fused multiply-add) intrinsic is the solution that caught on.

To understand how the solution works, let’s look at some simple code that evaluates a cubic curve:

float3 cubic(float x, float3 p0, float3 p1, float3 p2, float3 p3) {
    float B0 = x * x * x;
    float B1 = (1-x) * x * x;
    float B2 = (1-x) * (1-x) * x;
    float B3 = (1-x) * (1-x) * (1-x);
    return p0 * B0 + p1 * B1 + p2 * B2 + p3 * B3;
}

In most situations that would work fine. However, when doing tessellation you need to be more careful. If you have two cubic surfaces adjacent to each other, but their parametric orientations do not agree, and we use that function to evaluate the surface on the boundary, then the resulting positions will also not agree.

That is because the following condition is not true:

cubic(x, a, b, c, d) == cubic(1-x, d, c, b, a);

That is, the evaluation of the curve is not symmetric. The result of evaluating the curve at x is not the same if you change the order of the control points and evaluate it at 1-x.

It might seem counter-intuitive, but the way the compiler translates those expressions into a sequence of instructions does not guarantee symmetry. The computation itself must be expressed independent of parametric orientation.

Simply adding the precise qualifier to the result does not really solve our problems:

precise float3 cubic(float x, float3 p0, float3 p1, float3 p2, float3 p3) {
    float B0 = x     * x     * x;
    float B1 = (1-x) * x     * x;
    float B2 = (1-x) * (1-x) * x;
    float B3 = (1-x) * (1-x) * (1-x);
    return p0 * B0 + p1 * B1 + p2 * B2 + p3 * B3;
}

Note the order in which the operations are performed:

(((p0 * B0) + p1 * B1) + p2 * B2) + p3 * B3

Writing the expression with parenthesis to emphasize the evaluation order makes it clear that the expression does not have a symmetric structure, so it still doesn’t meet our requisite. What we should do then is to rewrite the expression as follows:

((p0 * B0) + p1 * B1) + ((p3 * B3) + p2 * B2)

It looks like now it should not matter in what direction the curve is evaluated, the result should be the same… but that’s not yet the case. If you look at the basis functions you will notice another problem:

    float B0 = x     * x     * x;
    float B1 = (1-x) * x     * x;
    float B2 = (1-x) * (1-x) * x;
    float B3 = (1-x) * (1-x) * (1-x);

B1 and B2 are not symmetric either. In order to guarantee that B1(x) == B2(1-x), they should be written as follows:

    float B0 = x     * x     * x;
    float B1 = x     * x     * (1-x);
    float B2 = (1-x) * (1-x) * x;
    float B3 = (1-x) * (1-x) * (1-x);

That is getting closer to being correct, but there are still some problems. The precise qualifier only affects the expressions on the right hand side of the precise variable. That is, the above code is equivalent to:

float3 cubic(float x, float3 p0, float3 p1, float3 p2, float3 p3) {
    float B0 = x     * x     * x;
    float B1 = x     * x     * (1-x);
    float B2 = (1-x) * (1-x) * x;
    float B3 = (1-x) * (1-x) * (1-x);
    precise float3 p = ((p0 * B0) + p1 * B1) + ((p3 * B3) + p2 * B2);
    return p;
}

So, the compiler is free to mess with our carefully arranged expressions. Instead we should declare all of them as precise:

precise float3 cubic(float x, float3 p0, float3 p1, float3 p2, float3 p3) {
    precise float B0 = x     * x     * x;
    precise float B1 = x     * x     * (1-x);
    precise float B2 = (1-x) * (1-x) * x;
    precise float B3 = (1-x) * (1-x) * (1-x);
    return ((p0 * B0) + p1 * B1) + ((p3 * B3) + p2 * B2);
}

OK, this code is finally watertight! The only problem now is that the resulting code is much more expensive than our original implementation.

In both cases the evaluation of the Bezier basis requires about 7 scalar instructions. However, the evaluation of the curve used 12 instructions, while now it requires 21 instructions and one extra register. This is where we can use the fma intrinsic. We can use fma and still obtain symmetric results, but we have to tell the compiler explicitly when is it possible to do so. In this case, the resulting expression is as follows:

fma(p1, vec3(B1), (p0 * B0)) + fma(p2, vec3(B2), (p3 * B3))

This preserves the symmetry of the computation and only requires 15 instructions, much better!

In order to think more easily about the cost of the different implementations I use the following notation:

* --- > --- > --- >  = 4*3 = 12

* -+- * -+- * -+- *  = 7*3 = 21

* --- > -+- < --- *  = 5*3 = 15

Here the stars (*) represent mul instructions, the crosses (+) represent add instructions, and the arrows (><) represent fma instructions. The dashes (-) simply describe the dependencies between instructions and the order of evaluation in a loose way.

Take note, because in the next article I’ll be using this notation more extensively.

I really didn’t bother to restore the ones in which I only whine about my back pain; I’d rather not remember that part of my life. Some articles still have missing images, and I may update them progressively.

In the meantime I’ve written a couple of articles for The Witness blog and in the future I’ll continue posting most of my technical articles there as well:

• The Witness – Computing Alpha Mipmaps, September 9, 2010
• The Witness – Texture Parameterization, March 30, 2010

Last spring was fantastic. I went hiking almost every single weekend and felt in better shape than ever. I’ve started writing some notes about my hikes in an attempt to plan them better; I don’t think this stuff is very useful to anyone but me, but here it is anyway:

Nacho’s Hiking Notes

This summer I went to Spain and stayed there 5 weeks, much longer than usual. My grandmother died unexpectedly and it was good to be able to spend some more time with my family. When I got back I realized how lonely I am here in the US, how superficial are all my relationships.

I tried to stay in shape while abroad, but there’s nothing like the DAM workouts. I was flirting with lane 5 when I left and I’m now back to lane 3 again. The first two weeks have been excruciating, but I’m getting back into my routine and hopefully I’ll make my way up again.

Mariana is 14 weeks pregnant, so we are expecting to have another kid by the beginning of November. She has just finished her studies and has found herself a job at EMQ/FamiliesFirst. She works counseling problematic children and loves it so far, but she has a terrible schedule, her shift is on weekend nights, so we don’t get to see each other much over the weekends.

So, now Nachito and I have plenty of time to spend together. Spring is here and in Davis the weather is great during this time of the year, so we are spending most of our time outdoors. We have been working in our garden again, Nachito helps me spread compost and plant the starts, he loves playing with worms.

We have also been exploring the surrounding parks: Berryessa, Cold Canyon, Cache Creek, Capay Valley and various sections of the South Yuba river. The rivers in the foothills are still a bit too cold for swimming, but the creeks in the North Coast Range are just perfect on a hot day. Nachito loves climbing on rocks and he’s getting increasingly good at it.

I keep swimming, not for rehabilitation anymore, but for recreation and fitness. I’m now in better shape than I’ve ever been and I feel pretty good about it. However, my back is not entirely healed yet; I cannot sit for long periods of time. I still feel a mild pain after driving long hours, biking on a rugged road or going to the movies. So, I keep working on stand-up desk. At this point it feels very natural, I don’t get tired as I used to anymore. Sitting on a chair feels too constraining, they remove too many degrees of freedom, it just feels wrong.

The Witness is progressing well, although on my side things are going a bit slower than I hoped. I need to get my shit together and use my time more effectively. Part of the problem is that I have too many distractions at home, so I’ve finally rented an office and I’ll be moving in tomorrow. For the first time in my life I won’t have a computer at home, I’m going to stay like that for a while, but I’ve been thinking I should probably get a laptop.