Alternatives (v0.2.20141117) Released November 17th, 2014
Patrick Stein

I have uploaded a new version of my Alternatives library. In addition to the ALTERNATIVES macro, there is an ALTERNATIVES* macro which allows one to specify a name for the set of choices. Then, one can check the DOCUMENTATION to see which alternative was last macroexpanded.

(defun random-letter ()
  (alt:alternatives* random-letter-algorithm
     "Always pick the letter A."

      "Choose any letter with equal probability"
      (random:random-elt "ABCDEFGHIJKLMNOPQRSTUVWXYZ"))))

(documentation 'random-letter-algorithm 'alt:alternatives)
     Always pick the letter A."

Alternatives (v0.1.20141115) Released November 16th, 2014
Patrick Stein

I have now released the code that I mentioned in my previous post Code That Tells You Why which lets one keep multiple implementations around in code and switch between them manually without much trouble.

A link to the source code is here:

Code That Tells You Why November 13th, 2014
Patrick Stein

A 2006 article by Jeff Atwood titled Code Tells You How, Comments Tell You Why showed up on reddit/r/programming today.

It makes a good point. However, it got me thinking that for cases like the binary-search example in the article, it might be nice to see all of the alternatives in the code and easily be able to switch between them.

One way to accomplish this in Lisp is to abuse the #+ and #- reader macros:

(defun sum-i^2 (n)
  (loop :for i :to n :summing (* i i))

  (do ((i 0 (1+ i))
       (sum 0 (+ sum (* i i))))
      ((> i n) sum))

  "Some people find a do-loop to hard to read
    (and 'too' too hard to spell, apparently)."

  (/ (* n (1+ n) (1+ (+ n n)) 6))

This is less than ideal for a number of reasons, including: one needs to make sure to pick “feature” names that won’t actually ever get turned on, the sense of + and - seem backwards here, and switching to a different alternative requires editing two places.

Another Lisp alternative is to abuse the case form:

(defun sum-i^2 (n)
  (case :now-i-know-better-and-can-do-this
     (loop :for i :to n :summing (* i i)))

     (do ((i 0 (1+ i))
          (sum 0 (+ sum (* i i))))
         ((> i n) sum)))

     "Some people find a do-loop to hard to read
    (and 'too' too hard to spell, apparently)."

     (/ (* n (1+ n) (1+ (+ n n)) 6)))))

This is better. No one can doubt which alternative is in use. It is only one edit to switch which alternative is used. It still feels pretty hackish to me though.

One can clean it up a bit with some macrology.

(defmacro alternatives (&body clauses)
  (flet ((symbol-is-***-p (sym)
           (and (symbolp sym)
                (string= (symbol-name sym) "***")))
         (final-clause-p (clause)
           (when (listp clause)
             (destructuring-bind (tag &body body) clause
               (when (and (symbolp tag)
                          (member (symbol-name tag)
                                  '("***" "FINAL" "BLESSED")
                                  :test #'string=))
      ((member-if #'symbol-is-***-p clauses)
       (let ((clause (first (rest anaphora:it))))
            ,@(rest clause))))

      ((find-if #'final-clause-p clauses)
          ,@(rest anaphora:it)))

      ((last clauses)
          ,@(rest (first anaphora:it)))))))

With this macro, one can now rewrite the sum-i^2 function quite readably:

(defun sum-i^2 (n)
     (loop :for i :to n :summing (* i i)))

     (do ((i 0 (1+ i))
          (sum 0 (+ sum (* i i))))
         ((> i n) sum)))

     "Some people find a do-loop to hard to read
    (and 'too' too hard to spell, apparently)."

     (/ (* n (1+ n) (1+ (+ n n)) 6)))))

If I wanted to try the my-first-attempt-was-something-like-this clause, I could stick a *** before that clause or change its name to *** or final or blessed, or I could move that clause into the last spot.

There is still an onus on the developer to chose useful alternative names. In most production code, one wants to clean out all of the dead code. On the other hand, during development or for more interactive code bodies, one might prefer to be able to see the exact “How” that goes with the “Why” and easily be able to swap between them.

(Above macro coming in well-documented library form, hopefully this weekend.)

Keeping Server and Client Separate September 8th, 2011
Patrick Stein

The problem

Whenever I write client-server applications, I run into the same problem trying to separate the code. To send a message from the server to the client, the server has to serialize that message and the client has to unserialize that message. The server doesn’t need to unserialize that message. The client doesn’t need to serialize that message.

It seems wrong to include both the serialization code and the unserialization code in both client and server when each side will only be using 1/2 of that code. On the other hand, it seems bad to keep the serialization and unserialization code in separate places. You don’t want one side serializing A+B+C and the other side trying to unserialize A+C+D+B.

One approach: data classes

Some projects deal with this situation by making every message have its own data class. You take all of the information that you want to be in the message and plop it into a data class. You then serialize the data class and send the resulting bytes. The other side unserializes the bytes into a data class and plucks the data out of the data class.

The advantage here is that you can have some metaprogram read the data class definition and generate a serializer or unserializer as needed. You’re only out-of-sync if one side hasn’t regenerated since the data class definition changed.

The disadvantage here is that I loathe data classes. If my top-level interface is going to be (send-login username password), then why can’t I just serialize straight from there without having to create a dippy data structure to hold my opcode and two strings?

Another approach: suck it up

Who cares if the client contains both the serialization and unserialization code? Heck, if you’re really all that concerned, then fmakunbound half the universe before you save-lisp-and-die.

Of course, unless you’re using data classes, you’re either going to have code in your client that references a bunch of functions and variables that only exist in your server or your client and server will be identical except for:

(defun main ()
  #+server (server-main)
  #-server (client-main))

Now, of course, your server is going to accidentally depend on OpenGL and OpenAL and SDL and a whole bunch of other -L’s it never actually calls. Meanwhile, your client is going to accidentally depend on Postmodern and Portable-Threads and a whole bunch of other Po-‘s it never actually calls.

Another approach: tangle and weave, baby

Another way that I’ve got around this is to use literate programming tools to let me write the serialiization and unserialization right next to each other in my document. Then, anyone going to change the serialize code would be immediately confronted with the unserialize code that goes with it.

The advantage here is that you can tangle the client code through an entirely separate path than the server code keeping only what you need in each.

The disadvantage here is that now both your client code and your server code have to be in the same document or both include the same sizable chunk of document. And, while there aren’t precisely name-capturing problems, trying to include the “serialize-and-send” chunk in your function in the client code still requires that you use the same variable names that were in that chunk.

How can Lisp make this better?

In Lisp, we can get the benefits of a data-definition language and data classes without needing the data classes. Here’s a snippet of the data definition for a simple client-server protocol.

;;;; protocol.lisp
(userial:make-enum-serializer :opcode (:ping :ping-ack))
(defmessage :ping     :uint32 ping-payload)
(defmessage :ping-ack :uint32 ping-payload)

I’ve declared there are two different types of messages, each with their own opcode. Now, I have macros for define-sender and define-handler that allow me to create functions which have no control over the actual serialization and unserialization. My functions can only manipulate the named message parameters (the value of ping-payload in this case) before serialization or after unserialization but cannot change the serialization or unserialization itself.

With this protocol, the client side has to handle ping messages by sending ping-ack messages. The define-sender macro takes the opcode of the message (used to identify the message fields), the name of the function to create, the argument list for the function (which may include declarations for some or all of the fields in the message), the form to use for the address to send the resulting message to, and any body needed to set fields in the packet based on the function arguments before the serialization. The define-handler macro takes the opcode of the message (again, used to identify the message fields), the name of the function to create, the argument list for the function, the form to use for the buffer to unserialize, and any body needed to act on the unserialized message fields.

;;;; client.lisp
(define-sender  :ping-ack send-ping-ack (ping-payload) *server-address*)
(define-handler :ping     handle-ping   (buffer) buffer
  (send-ping-ack ping-payload))

The server side has a bit more work to do because it’s going to generate the sequence numbers and track the round-trip ping times.

;;;; server.lisp

(defvar *last-ping-payload* 0)
(defvar *last-ping-time*    0)

(define-sender :ping send-ping (who) (get-address-of who)
  (incf *last-ping-payload*)
  (setf *last-ping-time*    (get-internal-real-time)
        ping-payload        *last-ping-payload*))

(define-handler :ping-ack handle-ping-ack (who buffer) buffer
  (when (= ping-payload *last-ping-payload*)
    (update-ping-time who (- (get-internal-real-time) *last-ping-time*))))

Problems with the above

It feels strange to leave compile-time artifacts like the names and types of the message fields in the code after I’ve generated the functions that I’m actually going to use. But, I guess that’s just part of Lisp development. You can’t (easily) unload a package. I can makunbound a bunch of stuff after I’m loaded if I don’t want it to be convenient to modify senders or handlers at run-time.

There is intentional name-capture going on. The names of the message fields become names in the handlers. The biggest problem with this is that the defmessage calls really have to be in the same namespace as the define-sender and define-handler calls.

I still have some work to do on my macros to support &key and &optional and &aux and &rest arguments properly. I will post those macros once I’ve worked out those kinks.

Anyone care to share how they’ve tackled client-server separation before?

Literate WordPress May 6th, 2010
Patrick Stein

I use WordPress along with the CodeColor plugin. This works fairly well for me. I find myself, however, breaking code up into sometimes awkward chunks to describe the code here or leaving things in very long segments.

Further, I often leave them ordered in my post in an order where they would compile if you just copied them into a file in the order presented. That’s not always the most useful order to present the code.

Literate programming seeks to tackle exactly these problems. So, I got to thinking: Why don’t I write some filters for noweb to output into WordPress+CodeColorer.

My Solution

My solution is in two parts, one filter of general use to all noweb users which simply inserts @language tags into the noweb stream and a noweave backend that outputs something you can just paste into the WordPress (HTML) input box.

Here is an example of how I used this to process the post.nw file that I used to compose this post.

noweave -n -filter langify-nw -backend wordpress-nw post.nw

  • langify-nw script
  • wordpress-nw script
  • tee-nw script useful for debugging noweb streams
  • post.nw literate source for this post (technically, I tweaked it a bit after proofreading without going back to the source combining the first two paragraphs and changing <h2>’s to <h3>’s and adding this parenthetical)

The langify-nw script guesses the language for each chunk based on the filename extension on the top-level chunk that incorporates that chunk. That’s why I used filename extensions on the top-level chunks in this post.

An Example

Several months back, I posted some code for rendering anti-aliased text in OpenGL. At the bottom of that post, I had an unwieldy 33-line function that you have to scroll to see with the way my blog is styled.

This would have been an excellent candidate for literate programming. Here is that same function broken up with noweb chunks with some descriptive text sprinkled between chunks.

At the heart of the algorithm, I need to take some point <px, py, pz>, run it backwards through all of the projection matrices so that I have it in pixel coordinates, and calculate the distance between that and the screen coordinates of the origin. So, I have this little function here to do that calculation given the object-local coordinates <px, py, pz> and the origin’s screen coordinates <ox, oy, oz>.

;;; dist-to-point-from-origin labels decl
(dist-to-point-from-origin (px py pz ox oy oz)
   (multiple-value-bind (nx ny nz)
       (glu:un-project px py pz
                       :modelview modelview
                       :projection projection
                       :viewport viewport)
     (dist nx ny nz ox oy oz)))

That, of course, also needs to determine the distance between two three-dimensional points. For my purposes, I don’t need the straight-line distance. I just need the maximum delta along any coordinate axis.

;;; dist labels decl
(dist (x1 y1 z1 x2 y2 z2)
   (max (abs (- x1 x2))
        (abs (- y1 y2))
        (abs (- z1 z2))))

Also, to do those reverse-projections, I need to retrieve the appropriate transformation matrices.

;;; collect projection matrices
(modelview (gl:get-double :modelview-matrix))
(projection (gl:get-double :projection-matrix))
(viewport (gl:get-integer :viewport))

Once I have those pieces, I am just going to collect those matrices, unproject the origin, then find half of the minimum unprojected distance of the points <1, 0, 0> and <0, 1, 0> from the origin.

;;; calculate-cutoff.lisp
(defun calculate-cutoff (font-loader size)
    #<:use "scale based on font-size">
    (let (#<:use "collect projection matrices">)
      (labels (#<:use "dist labels decl">
               #<:use "dist-to-point-from-origin labels decl">)
        (multiple-value-bind (ox oy oz)
            (glu:un-project 0.0 0.0 0.0
                            :modelview modelview
                            :projection projection
                            :viewport viewport)
          (/ (min (dist-to-point-from-origin 1 0 0 ox oy oz)
                  (dist-to-point-from-origin 0 1 0 ox oy oz))

Oh, and being the tricksy devil that I am, I scaled things based on my font size before doing any of that.

;;; scale based on font-size
(let ((ss (/ size (zpb-ttf:units/em font-loader))))
  (gl:scale ss ss ss)

Updates In Email