The tension between Chaos and Order

I’ve been spending the last week in Japan, meeting with makers and crafts people. as always, it’s a humbling experience: the degree of refinement and care that exists here is really incredible. Every maker has gone very far into their craft, and found their own expression.

There are two things about this practice that I find inspiring and illustrative. One is the depth that artists go to. You will see someone get completely obsessed with something, and think “why?” I’ve done this myself in my (much more modest) making. I’ll dive down into some rabbit holes that doesn’t make sense to anyone but me. Being here and seeing the great art that comes from that is really inspiring. And a good reminder that following your instincts and obsession is critical for any inventor or disruptor. It won’t make sense to anyone, until it does. You have to give yourself permission to explore the things that make sense to you.

The second thing I saw here seems to connect to something I’ve been observing about LLMs and programming. There is an inherent tension here in the art, between the “order” of knowing how to do something well, and the “chaos” of the unknown, or even of breaking the rules. The best artists navigate this, often in different ways. For example, we saw a potter making very large vessels that sometimes break in the kiln. The broken ones are almost the more interesting, and they are keeping them as works.

In LLMs, I see a tension between the determinism of code, and the chaos/stochasticity of the model. Neither is valuable by itself - the models are unreliable, and the reliable code is brittle. But if you are careful, you can build mixtures of the two that are very effective - using each side of the spectrum to “shore up” the weakness of the other.

Go down the rabbit holes! Learn the rules, but break them carefully. Use code to make the models more effective, but not so much that they can’t do anything useful.

I like how it’s all connected.