The easy choice is: . This is monotonically increasing. and .

I needed to scale three such quantities and mush them together. I thought it’d be spiffy then to have three different functions that satisfy my criteria. The next logical choice was . It is everywhere positive and increasing. And, it has .

Now, I needed third function that was always positive, always increasing, had and . One choice was: . But, that seemed like overkill. It also meant that I really had to keep my tiny if I didn’t want to scale things into the stratosphere.

Playing with hyperbolas

So, I thought… why don’t I make a hyperbola, rotate it, and shift it so that the apex of one side of the hyperbola is at . And, I can adjust the parameters of the hyperbola so that . After a variety of false starts where I tried to keep the hyperbola general until the very end (, rotated by degrees, and shifted by ), I quickly got bogged down in six or seven incredibly ugly equations in eight or nine variables.

So, it was time to start trying to make it easy from the beginning. I noticed that if was going to rotate it by an angle in the clockwise direction, then I needed to be such that if my slope was going to work out right in the end. So, I’m looking at the basic triangle on the right then to determine all of my sines and cosines and such.

Based on that triangle, it was also obvious that the asymptote for my starting hyperbola had to have . I played around a bit then with making and . In the end, I found things simplified sooner if I started with and .

I also needed to be such that the point rotate up so that its -coordinate was . This meant that or .

So, my starting hyperbola then was: .

From there, I had to rotate the and by in the clockwise direction. This gave me:

A little multiplying out leads to:

From there, using , , and , we come to:

The only step remaining was to shift it all over so that when , we end up with . Plugging in , we see that . That equation balances when . We want it to balance when , so we’re going to substitute in place of the in the above equation to get:

We can easily verify that the point is on the curve. And, we can implicitly differentiate the above to get:

This is pretty good as it goes. The only step is to complete the square to find a nicer expression for in terms of . We start by adding to both sides to get:

This is easy enough to solve for by taking the square root of both sides and shuffling some things about:

Here are all three curves with . The exponential is in black, the hyperbolic tangent is in red, and the hyperbola is in blue:

The first image on the page here was made with Zach’s Vecto library with some post-processing in the GIMP. (Here is the source file: hyperbola.lisp.) The second image was made entirely within the GIMP. And, the last image was made using Foo Plot and the GIMP.

Each cosine wave has the following form

The image at the right here, is from the 443rd generation of a population of 200 where each gene represents 100 cosine waves. For the video below, I used a population of 100 genes, each representing the sum of 50 cosine waves.

Image Approximation with Genetically Selected Cosines from Patrick Stein on Vimeo.

Nitty Gritty Details

The genes are represented internally as an array of fixed-point numbers. Each number is five-bits integer, ten-bits fractional, and a sign bit. I scale the amplitudes down by a factor of 50 and scale the frequency and offset up by 50% when I convert them back to floating point from the fixed point representation just to get them into a more useful ballpark. During each generation, the fittest two quartiles of the population are cross-bred (in randomly selected pairs with two-point crossover) to replace the third quartile of the population, and the fourth quartile is replaced by mutations of randomly selected members of the top quartile. I also run through and remove any duplicate genes and replace them with randomly generated ones so that I don’t breed myself into a total corner.

I toyed around with two different fitness functions: RMS and entropy. So, I would calculate the image represented by the gene and subtract that from the original image. Then, I would calculate the RMS () or the sum of the entropies of the possible values (, where is the proportion of the pixels in the resulting image that had value ).

I bothered with entropy because I was hoping that if I could encode a gene in just a hundred or two hundred bytes that I might be able to compress the remainder (losslessly) and save bytes over just compressing the original image. The big problem with the entropy calculation though is that I get the same answer when every pixel value is off by 100 as I get when every pixel value is right on target. So, for a time, I was multiplying the RMS and the entropy for the fitness function. For the video and images above, I just used the RMS since it reached very close to the same solution as the multiplying method, and I saved myself fifty histograms per generation.

Source Code

Here is the Common Lisp source code for the genetic algorithm. It depends on the CL-PNG library for image input and output. As it appears here, it will create an output image from the fittest member of each generation, an image of the fittest member of the current generation, an image of the original with the fittest member of the current generation subtracted, a Lisp file containing the fittest gene of the current generation, and the current state of the random number generator when the run ends.

For the video above, I quickly put together some code to demonstrate the different parameters that the genetic algorithm can tweak and to show incrementally adding in more of the cosine waves. Here is some general utility code that is display-independent. And, here is the OpenGL display code which depends on the CL-OpenGL libraries. I included the source code here because I had trouble finding an example on the net of using Lisp-created textures with CL-OpenGL. I render the cosine waves into a buffer, copy that buffer into an OpenGL texture, and draw a quad with that texture. Note: it looks a little sluggish when run, but that is because I put a big (SLEEP …) in there to keep things from flying by too quickly.

The leader-bee is thick white string with green beads. The follower-bees are the thin black strings with red beads. Going clockwise from the top, the beads on each string represent amplitude, x-frequency, y-frequency, x-phase, and y-phase. The outside of the HUD represents positive infinity. The inner circle of the HUD represents negative infinity. The light yellow circle between the infinities represents zero. Note: the scaling is extreme. Two is almost 90% of the way from zero to infinity on this scale.

Fourier Swarm with Bee Display from Patrick Stein on Vimeo.

Technical details of the HUD scaling

Say you have a number that you want to map into the interval . Draw the upper half of the unit circle centered at the origin. Start moving from left to right along the circle until you find the spot with a tangent of slope . Use the -coordinate of this point. To map that interval onto my HUD, subtract your calculated -coordinate from two and use the result as the radius from the center of the HUD.

Here is the Lisp source code. It depends on Zach’s Vecto library.

Edit: Oops… there is a bug in the HUD code. The leader and first follower are not shown. The second follower is the thick white line. Ooops. Okay… I have replaced the video and the source code now.

Here is one of the nifty designs that I generated with his program. I encourage you to download it and play around a little bit. It is really great to see in real-time how activating certain frequencies affects the output.

For reference, here are the frequencies that I had activated for the above picture.

There are gorgeous specular features and shading in his output images. He could have just it left it flat-colored, but he went the extra distance to add a lighting model. The results are stunning. You should click on the above image to see it full-size. And, you should check out the the example on Neil’s page.

  

From there, I took the Fourier Transform:

  

Then, I did a horizontal Sobel operator on it to get this:

  

From there, I did the Inverse Fourier Transform to obtain this:

  

Enjoy!

  

This is what it looks like if I do the inverse FFT of that data. It’s not an incredibly interesting result, but it does show the power.

  

Here is another example of a starting point made entirely with the gradient paint tool:

  

And, its resulting image when I do the inverse transform:

  

On the other hand, if you just want to quickly plot something that you can save as an image, you either have to re-learn gnuplot (every time I use it, I have to sift through the help to figure out how to set up my tics and labels and output size and output filename) or you have to do screen grabs from one of the Java applets.

With the plotter that I wrote yesterday, you click on the Snapshot button, then right-click to save the image or copy it to your clipboard.

Enjoy.

Uniqueness

For some of these games, we mean a unique solution amongst all solutions without backtracking. For example, in solving a maze, we would not consider solutions that involve running down a corridor only to come back along the same corridor. For Lights Out games, the concept is simple but describing it takes work. Imagine that every square on the board, in addition to being lit or not, also has a letter ‘A’ on it at the start of the game. Whenever you press a button labelled ‘A’, it gets relabeled ‘B’. Every time you press a button labelled ‘B’, it gets relabeled ‘A’. Two solutions are identical if the board has the same configuration of ‘A’s and ‘B’s after both moves. Thus, hitting any button more than once was just a waste of time. However, the order that you hit the buttons makes no difference at all.

There is a notable crossword puzzle with two solutions. There are some notorious Sudoku puzzles with two solutions. I don’t know of any cryptograms with two solutions, but I am going to go asearchin’ some time.

Technically, a Sudoku puzzle with two solutions cannot be solved. You had to make a guess somewhere and it turned out you had twice the odds of being right that you should have. It is easy to accidentally take too many numbers out of the Sudoku grid and end up with an ambiguous puzzle. For a crossword puzzle, I doubt most people bother to check to see if the solution is unique. It probably is unique by sheer combinatorics. The same may be true cryptograms, as well. You’d have to specifically be trying to make them ambiguous to achieve it.

Trapdoors

Where are the trapdoors? Well, the puzzle creator has a much harder job than the puzzle solver for most games. It is easy to construct mazes and Lights Out games with unique solutions (in fact, it’s easy to show that if a Lights Out game is solvable, the solution is unique). It is not so easy to construct other puzzles that have provably unique solutions.

This brings us to the Discrete Logarithm Problem. Imagine that I told you that and asked you what is modulo 28. There is no really easy way to solve this problem, yet it is easy to formulate such problems. I checked that . Then, I picked and plugged it in to see that . I started from the answer and was quickly able to generate a problem that I can guarantee has a unique solution.

This is called a trapdoor function. It is easy to calculate, but hard to invert the calculation. It’s also not a very fun problem to do. You will never open up the comics page of your newspaper and find five of these problems below the bridge column and above the horoscopes.

I can easily imagine a whole variety of Lights Out type games where lights have more than one state. It would be easy to prove, for a wide class of such games, that when a solution exists, it is unique. These would all be solvable by inverting a matrix whose coefficients come from a finite field.

What other puzzles could take advantage of trapdoor functions to make the game easier on the puzzle maker than on the puzzle solver?

Me, I’ve learned to never trust my naïve intuition when it comes to probability questions. Well, it turns out that the naïve answer is perfectly correct in this case. Here is the epic journey I took to get there though.

Here’s what I wanted. I wanted to know the probability needed so the expected number of trials until the first success is some number given that each trial is independent and is successful proportion of the time. Suppose for a moment that I decided I am going to roll a six-sided die until I get a one. What is the expected number of times that I need to roll that die to get my first one? What I want here is the inverse problem. How many sides should my die be if I want it to take tries (on average) to get my first one?

In the picture at the right, each circle represents a trial. Each square represents a success. What I am looking for is what value should I pick for so that if you go left proportion of the time and go right the other proportion of the time, you will expect to hit circles along your descent. You could, if very unlucky (or lucky in this case since I’m going to nag you) go forever to the right and never get to a rectangle. So, already, we’re expecting an infinite sum.

Indeed, the expected number of circles you cross in a random descent of this tree is . Massaging this a little, we can get it into a simpler form: .

That infinite series is pretty nasty looking though. So, we whip out a handy table of infinite series and see that:

Whew! Big sigh. We’re through the ugly parts. Now, in our case, , and the equation we were interested in was . Plugging in for the in our infinite series solution, we find that

Two pages of ugly, ugly algebra bears out the simple guess. Wheee! If we want , then we have to pick . Hooray!

What if I wanted to be exact? Suppose I wanted to switch from points uniformly distributed in a box to points from a normal distribution while maintaining the expected distance from the selected point to the origin.

What exactly is the expected distance from a selected point to the origin if one picks each coordinate for the point from a uniform distribution on the range [-1,+1]? Let’s start with the one-dimensional case. The probability density is everywhere. So, the answer is as we would expect.

How about two dimensions? This is immediately more awkward. The one-dimensional disk looks exactly like the one-dimensional square. In two dimensions, the disk and square are quite different. We either have to integrate over the square and calculate the radius of each point, or we have to integrate over increasing radii and be careful once we get to the in-radius of the square. (image created in Lisp using Zach's Vecto library)

I originally chose to try to integrate over the square. The probability density is everywhere. This means the expected radius is . This gets to be no fun very fast since the inner integral comes out with a logarithm in it. I mean, do you want to finish this integral?

So, how about integrating over the radii? Again, the probability density is everywhere. This makes the expected radius:

And, this is only the two-dimensional case! In my original problem, the ambient space was 32-dimensional. My, oh my.