CL-MW: The development of a Master/Worker framework in Common Lisp

CL-MW is a Common Lisp Master/Worker framework for rapid prototyping of pleasantly parallel applications with fine-grained tasks. It focuses on low cognitive overhead for application development, easy integration with well-known batch scheduling systems, and scalability in the ~10K worker and the billions of tasks range. Hear about the challenges and solutions in the development of this framework.

If you have started, post a comment here or there to let us know how it’s going!

Your program’s goal is to find the people. You will get points for emitting the pixel coordinates of a pixel inside a person. You’ll get bonus points if it’s a head pixel. Be careful, though. You’ll lose points for emitting more than one pixel per person.

• Deadline: September 19th.
• Target Platform: SBCL or Clozure under Mac OS X (10.6.4).
• Prizes for: Tournament winner, Prettiest code
• Full Details: http://tclispers.org/contest-find-the-people

Teaching Introductory Programming Using Scheme

Daniel Feldman gave this presentation at the TC Lispers meeting in May 2010.

• Presentation Slides
• Course Description page

Teaching Introductory Programming Using Scheme on Vimeo.

ASDF2

Robert Goldman gave this lightning talk to the TC Lispers meeting in May 2010.

• Presentation Slides
• ASDF home page

ASDF2 on Vimeo.

Programming Contest

Patrick Stein gave this lightning talk to the TC Lispers meeting in May 2010.

• Presentation Slides

Programming Contest on Vimeo.

Lightning talk by Patrick Stein to the Twin Cities Lisp Users Group about hosting a Programming Contest. This presentation was recorded on 2010-05-25.

Weblocks Presentation

Patrick Stein gave this presentation at the TC Lispers meeting in April 2010.

• Presentation Slides

Weblocks on Vimeo.

Allegro Serve and Web Actions Presentation

Robert Goldman gave this presentation at the TC Lispers meeting in April 2010.

• Presentation Slides

Apology: Unfortunately, ScreenFlow bombed out on me when I went to stop recording. It subsequently saw that it had a partial project there but was unable to recover it. As such, there is no video available for this presentation. Feh. — Patrick

Hunchentoot Presentation

Paul Krueger gave this presentation at the TC Lispers meeting in April 2010.

• Presentation Slides

Hunchentoot on Vimeo.

Cocoa Lisp Controller Presentation

Paul Krueger gave this presentation at the TC Lispers meeting in April 2010.

• LispController Reference Manual
• Tutorial Document for Building Cocoa Interfaces with Lisp
• Paul Krueger’s Cocoa Lisp contributions to CCL

Cocoa Lisp Controller on Vimeo.

CL-Growl Presentation

Patrick Stein gave this presentation at the TC Lispers meeting in April 2010.

• Presentation Slides
• CL-Growl home page

CL-Growl on Vimeo.

ps. if you’re in range, come to the meeting… Common Roots Cafe in Minneapolis.

Here are the slides with presenter notes and the accompanying sample code.

Lisp GUI Discussion on Vimeo.

Robert Goldman, Paul Krueger, and Patrick Stein gave short presentations on different approaches to Graphical User Interfaces with Common Lisp.

• Lisp GUIs or “Bloodied but Unbowed” by Robert Goldman
• CCL/Cocoa Interface by Paul Krueger
• GUIs with CL-OpenGL by Patrick Stein

I run Mac OS X 10.6.2 as my primary development machine. I knew everything I had so far worked there under SBCL 1.0.30 and CMU-CL 20a. It took a little tweaking to get Sheeple built under Allegro CL 8.1 (Trial Edition), but after that I was five-by-five under Allegro, too. Unfortunately, I cannot get CL-OpenGL to run under Clozure 1.3-r11936 on Mac OS X. Also, ECL 9.10.2 doesn’t support weak key hashtables which Sheeple needs. And, my install of clisp broke somewhere along the lines, so I haven’t tried it.

I have an Ubuntu Linux box. There, I am using SBCL 1.0.11.debian (eeps). Everything ran perfectly over there (even displaying through X Windows back to my Mac).

I also run Windows Vista under VMWare Fusion on my Mac. I hadn’t done any development on it for months and months. Fortunately, in Google-ing to find out how to fix it, I stumbled upon what I had written about how I got things set up originally. Over the last two hours, I got SBCL upgraded to 1.0.29 under Vista. I got ASDF-Install set up with a bunch of help from this article. And, from there, I got CFFI and cl-opengl and ZPB-TTF and Sheeple installed.

ZPB-TTF and Sheeple both used some tar format options that archive_0.7.0 didn’t like. For those, I had to jump through some hoops to untar and retar them to get them to install.

Here was my final /Users/Patrick/.sbclrc file:

(require :asdf)
 
;; from Zach Beane
(defmethod asdf:perform :around ((o asdf:load-op)
                                (c asdf:cl-source-file))
  (handler-case (call-next-method o c)
    (#+sbcl sb-ext:invalid-fasl
     #+allegro excl::file-incompatible-fasl-error
     #+lispworks conditions:fasl-error
     #+cmu ext:invalid-fasl
     #-(or sbcl allegor lispworks cmu) error ()
     (asdf:perform (make-instance 'asdf:compile-op) c)
     (call-next-method))))
 
(dolist (pkg  '("alexandria/"
                "archive_0.7.0/"
                "asdf-install/asdf-install/"
                "babel_0.3.0/"
                "cffi_0.10.5/"
                "cl-opengl/"
                "flexi-streams-1.0.7/"
                "gzip-stream_0.2.8/"
                "salza2-2.0.7/"
                "sykopomp-sheeple-ceab213/"
                "trivial-features_0.6/"
                "trivial-gray-streams-2008-11-02/"
                "woolly/"
                "zpb-ttf-1.0/") )
  (pushnew (merge-pathnames pkg
                            (merge-pathnames "ASDF-Systems/"
                                             (user-homedir-pathname)))
           asdf:*central-registry*))
 
(asdf:oos 'asdf:load-op 'asdf-install)
 
;; for my sanity
(setf asdf-install:*locations*
      (list (list (merge-pathnames "ASDF-Systems/" (user-homedir-pathname))
                  (merge-pathnames "ASDF-Systems/" (user-homedir-pathname))
                  "My install spot")))
 
;; via http://sean-ross.blogspot.com/2007/05/asdf-install-windows.html
#+win32
(asdf:oos 'asdf:load-op 'gzip-stream)
 
#+win32
(asdf:oos 'asdf:load-op 'archive)
 
#+win32
(defun asdf-install-extractor (to-dir tarball)
  (let ((name nil))
    (gzip-stream:with-open-gzip-file (ins tarball)
      (archive:with-open-archive (archive ins)
	(let ((*default-pathname-defaults* (pathname to-dir)))
	  (archive:do-archive-entries (entry archive name)
	    (archive:extract-entry archive entry)
	    (unless name (setf name (archive:name entry)))))))
    (string name)))
 
#+win32
(push 'asdf-install-extractor asdf-install:*tar-extractors*)

The list of packages in the middle were about half installed manually to get the archive extraction code working and half installed through ASDF-Install. If I recall correctly, I had to manually install: archive_0.7.0, flexi-streams-1.0.7, gzip-stream_0.2.8, salza2-2.0.7, trivial-gray-streams-2008-11-02, and a fresh copy of asdf-install. I also had to download a compiled version of freeglut.dll and tuck it into my Windows\system32 directory.

Getting SBCL to use the fresh copy of asdf-install was annoying. I ended up starting up a Command Prompt as Administrator (right click on the “command.exe” icon or menu-item and select “Run as Administrator”). Then, I went to the SBCL directory (“C:\Program Files\Steel Bank Common Lisp\1.0.29\”) and did the following:

% rename asdf-install asdf-install.old
% mklink /d asdf-install "C:\Users\Patrick\ASDF-Systems\asdf-install\asdf-install\"

I had extracted the tar-ball from the ASDF-Install distribution into my ASDF-Systems directory.

Then, I went back and made my GUI code use double-buffered OpenGL windows because running Lisp to OpenGL to Windows Vista to VMWare to Quartz had some wicked flickering going on.

Two hours is a long time for something that’s supposed to be easily portable. But, I would have spent at least 95% of that time even if I were using an all-Lisp solution. And, this is far less time than I ever spent porting anything else to Windows.

I have dubbed this project Woolly (because it’s made from Sheeple and because Woolly sounds sorta like GUI).

I spent about half of my coding time so far getting the basic framework in place (proven through clickable buttons with labels). I am trying to keep it cleanly separated between generic GUI stuff and the CL-OpenGL specifics in the event that someone would like to port it to some other I/O spec.

The other half of my coding time was spent getting the font-rendering to be anti-aliased. I promise to write more about how I accomplished the font-rendering in a future post so that if you’re ever stuck rendering fonts in OpenGL, you won’t be stuck with pixelated blockiness or resorting to rendering to a texture-map and letting the mipmapper figure it out.

For this post, however, I’ll just show you the code that sets up the interface depicted here.

(require :asdf)
(asdf:operate 'asdf:load-op 'woolly-gl :verbose nil)
 
;; make it easier to change renderer/controller later
(rename-package 'woolly-gl 'toolkit)
 
(defun test ()
  (let ((font (sheeple:object :parents toolkit:=font=
                              :em-size 48
                              :pathname "okolaks/okolaksRegular.ttf")))
    (let ((app (sheeple:object :parents toolkit:=app=))
          (win (sheeple:object :parents toolkit:=window=
                               :title "Woolly Window 1"
                               :width 640
                               :height 480))
          (but (sheeple:object :parents toolkit:=button=
                               :offset-x 40
                               :offset-y 40
                               :width 300
                               :height 100
                               :font font
                               :label "Button")))
      (woolly:display-window win)
      (woolly:add win but)
      (woolly:main-loop app)
      (woolly:destroy-window win))))

Next up on my agenda is to make the background and button prettier. It should be easy enough to do with GL_LIGHTING and some vertex-coloring for gradations. After that, it’s on to more controls like labels, panels, checkboxes, drop-downs, borders, and (my dread) text input boxes. Then, it’s on to a layout manager.

Edit: Here’s the same GUI a day later. I’m using a simple lighting scheme and rendering the button in 3D. I haven’t yet hooked in the bit to render it depressed when the button is pressed. I’ve tested the code that draws it the other way, but I haven’t hooked it into the mouse handlers yet.

Edit #2: Actually, it only took a few minutes for me to hook in the rendering it pressed vs. unpressed. When I did it though, it looked like the label was sliding around because the effect of the contrast between the light and dark edges of the button was so great that you perceive the whole button sliding when it’s pressed. So, I added a little bit in there to actually slide the text by an amount close to what is perceived. So, now… it looks pretty spiffy.