Juho Snellman's Weblog

Pretty SBCL backtraces

jsnell@iki.fi — Thu, 20 Dec 2007 00:00:00 GMT

Every now and then I see complaints about the stacktraces in SBCL. They contain too little info, or too much info, or are formatted the wrong way, etc. But the backtrace printing isn't really any dark magic, it's just basic Lisp code. If you don't like the default format, just write a new backtrace function that prints something prettier/less cluttered/more informative/etc.

For inspiration, below is one implementation, based on a really quick hack I wrote in answer to a c.l.l post a few weeks ago. In addition to cosmetic changes, it adds a a couple of extra features: printing filenames and line numbers for the frames when possible, and printing the values of local variables when possible. Just call backtrace-with-extra-info in any condition handler where you'd normally call sb-debug:backtrace, or call it from the debugger REPL instead of using the backtrace debugger command.

The code assumes that you've got Swank loaded. For best results, compile your code with (debug 2) or higher.

(defun backtrace-with-extra-info (&key (start 1) (end 20))
  (swank-backend::call-with-debugging-environment
   (lambda ()
     (loop for i from start to (length (swank-backend::compute-backtrace
                                        start end))
           do (ignore-errors (print-frame i))))))
(defun print-frame (i)
  (destructuring-bind (&key file position &allow-other-keys)
      (apply #'append
             (remove-if #'atom
                        (swank-backend:frame-source-location-for-emacs i)))
    (let* ((frame (swank-backend::nth-frame i))
           (line-number (find-line-position file position frame)))
      (format t "~2@a: ~s~%~
                   ~:[~*~;~:[~2:*    At ~a (unknown line)~*~%~;~
                             ~2:*    At ~a:~a~%~]~]~
                   ~:[~*~;    Local variables:~%~{      ~a = ~s~%~}~]"
              i
              (sb-debug::frame-call (swank-backend::nth-frame i))
              file line-number
              (swank-backend::frame-locals i)
              (mapcan (lambda (x)
                        ;; Filter out local variables whose variables we
                        ;; don't know
                        (unless (eql (getf x :value) :<not-available>)
                          (list (getf x :name) (getf x :value))))
                      (swank-backend::frame-locals i))))))
(defun find-line-position (file char-offset frame)
  ;; It would be nice if SBCL stored line number information in
  ;; addition to form path information by default Since it doesn't
  ;; we need to use Swank to map the source path to a character
  ;; offset, and then map the character offset to a line number
  (ignore-errors
   (let* ((location (sb-di::frame-code-location frame))
          (debug-source (sb-di::code-location-debug-source location))
          (line (with-open-file (stream file)
                  (1+ (loop repeat char-offset
                            count (eql (read-char stream) #\Newline))))))
     (format nil "~:[~a (file modified)~;~a~]"
             (= (file-write-date file)
                (sb-di::debug-source-created debug-source))
             line))))

For example on the following code:

(declaim (optimize debug))
(defun foo (x)
  (let ((y (+ x 3)))
    (backtrace)
    (backtrace-with-extra-info)
    (+ x y)))
(defmethod bar ((n fixnum) (y (eql 1)))
  (foo (+ y n)))

The old backtrace would look like:


1: (FOO 4)
2: ((SB-PCL::FAST-METHOD BAR (FIXNUM (EQL 1)))
    #<unused argument>
    #<unused argument>
    3
    1)
3: (SB-INT:SIMPLE-EVAL-IN-LEXENV (BAR 3 1) #<NULL-LEXENV>)

And the new backtrace like:

1: FOO
   At /tmp/testlisp:5
   Local variables:
     X = 4
     Y = 7
2: (SB-PCL::FAST-METHOD BAR (FIXNUM (EQL 1)))
   At /tmp/testlisp:8
   Local variables:
     N = 3
     Y = 1
3: SB-INT:SIMPLE-EVAL-IN-LEXENV
   At /scratch/src/sbcl/src/code/evallisp:93 (file modified)
   Local variables:
     ARG-0 = (BAR 3 1)
     ARG-1 = #<NULL-LEXENV>

An improvement? That's probably in the eye of the beholder, and depends on the codebase and the use cases. For example I can imagine that for large functions showing the values of local variables in the trace would make it way too spammy. But that's besides the point: if the default stacktrace format is making debugging difficult for you, it's not hard to customize it.

Faster SBCL hash-tables

jsnell@iki.fi — Mon, 01 Oct 2007 05:15:00 GMT

Long time, no blog. I have an excuse though, since I moved to Switzerland for a new job a month ago, and haven't had a lot of time for things like blogging or hacking Lisp (the latter is usually a prerequisite for the former for me).

Anyway, I finally finished and committed the third rewrite of my patch for speeding up the embarrassingly slow hash-tables in SBCL. It turned out to be a really frustrating game of whack-a-mole, with every change uncovering either some new deficiency or another interaction between the GC and the hash-tables that the old implementation had handled by always inhibiting GC during a hash-table operation.

The main user-visible change is that SBCL no longer does its own locking for hash-tables (the fact that it locked the tables was always just an implementation detail, not a part of the public interface). This follows the usual SBCL policy of requiring applications to do take care of locking when sharing data structures between threads.

The exact details are pretty boring, so I won't repeat them here (read the commit message if you really want to know). Instead I'm just going to post a pretty benchmark graph, since it's been way too long since I last did one of these:

Sadly those improvements don't mean that SBCL now has the fastest hash-tables in the West, it just means they don't completely suck. For some reason the issue of SBCL hash-table speed has come up more often during the last couple of months than during the previous three years combined, so it was probably time to get this sorted out.

ICFP 2007

jsnell@iki.fi — Wed, 25 Jul 2007 07:30:00 GMT

For the last five years or so it's always been my firm intent to take part in the programming contest associated with the International Conference on Functional Programming (ICFP). And each year something has prevented it. But this year there was no emergency at work, no computer hardware broke, no sisters were getting married, etc. So instead of playing poker on the net, which had been consuming all of my free time for the last couple of weeks, I read the 22 page spec and fired up emacs. (Just kidding, emacs was already running).

The surface task was to write an interpreter for a weird string-rewriting language. The organizers supplied a large blob of data, which when run through the interpreter would produce as output some image drawing operations (for which you basically had to write some kind of a visualizer if you wanted to achieve anything). The goal was to come up with some prefix to the program which would make it instead produce output that would be as close as possible to a certain target image.

The intended way to achieve that goal was to notice that the drawing operations generated by the blob would first write a clue message, which would then be hidden in the final image by other image operations. This seems like a really bad decision. I luckily noticed the message since my first version of the image conversion tool didn't support the flood fill operation. But apparently a lot of teams never saw the message, and were left to stumble in the dark for the whole weekend. The image that could be drawn by using the clue would then lead to another obscure puzzle. Again, I was lucky to figure out the solution after a while, but judging by IRC and mailing list traffic a huge amount of teams never got it, and were basically stuck.

That clue could then finally be used to produce some concrete details on how the big blob of data was using the string-rewriting language to produce the image. There was even a catalog of the functions that the blob contained. But the really useful data seemed to be hidden behind yet more puzzles. So at this point I just did a minimal hack to make a token improvement to the produced image: the source image had a grove of apple trees, the target had pear trees. And according to the catalog the function apple_tree was exactly as large as pear_tree. So I wrote a prefix that overwrote the former with the latter. And then I submitted that prefix, and switched to doing something more interesting. (I think that token improvement was still in the top 20 something like 8 hours before the contest ended, which probably says something about how much progress people were making).

I did rather enjoy writing the interpreter and the visualization tool, and the specifications for both were mostly very good. Unfortunately the spec contained only a couple of trivial test cases with the expected results, so if your interpreter had a problem, figuring out what exactly was going wrong just from looking at execution traces was really hard. The organizers originally replied on the mailing list that such debugging "is exactly part of the task", but later released an example trace from a few iterations at the start. There was a documented prefix that would run some tests on the implementation, and generate an image from those results, but the coverage of those tests didn't seem to be very good. (I had several bugs that only showed up with the full image, not with the test one).

The part of the interpreter that many teams seemed to have big trouble with was that you couldn't really use a basic string or array to represent the program. If you did, performance would be orders of magnitude too slow (people were reporting getting 1 iteration / second, when drawing the basic image would require 2 million iterations) due to excessive copying of strings. Now, this was even pointed out in the specification! Paraphrasing: "these two operations needs to run faster than in linear time". And still people tried to use strings, bawled when their stupid implementation wasn't fast enough, and decided that the only solution would be to rewrite their program in C++ instead of their favourite Haskell/Ocaml/CL. Good grief...

For what it's worth, I used just about the stupidest imaginable implementation strategy beyond just a naive string: represent the program as a linked list of variable length chunks, which will share backing storage when possible. My first CL implementation of this ran at about 5.5k iterations / second. This was good enough at the stage in the competition that I got to, and would've been easy to optimize further if I'd decided to continue (afterwards I made a 15 line change that gave a 8x speedup, so the basic image now only takes 41 seconds to render on an Athlon x2 3800+). And this was with a stupid data structure and couple of minor performance hacks. It seems obvious that practically any language could have been used to write a sufficiently fast interpreter. It never ceases to amaze me how programmers would rather blame their tools than think about the problem for a couple of minutes.

Anyway, the organizers obviously put in a huge effort for this contest, so thanks to them for that. It's just that the format really wasn't what I was looking for in a programming contest. But at least it was interesting enough to temporarily shake me out of playing poker into doing some hacking again :-) (Faster SBCL hash tables coming soon, I hope).

I've made the source code for the interpreter available since several people have asked for it. I'm not sure why they've asked for it, since it's not very good code, and probably contains no worthy insights. But if you want it, it's there.

Addenda: After writing the above, I read a few messages on the mailing list which claimed that there really wasn't much of a puzzle aspect, but that success was mainly determined by how good tools (compilers, debuggers, disassemblers, etc) you were able to write. While it's possible that after the initial two humps that I described above the puzzles were irrelevant, that wasn't my impression. At the point where I stopped, it didn't feel to me as if sufficient knowledge was available for writing the tools, but rather was hidden behind encrypted pages, steganography, etc. None of which I really wanted to deal with.

There was definitely enough information available to make a start at reverse-engineering, but I don't think there was enough time to reverse-engineer enough of the system to figure out how to write the tools, write them, and then use the tools to actually solve the real problem. I'm sure things were different for larger teams, but that doesn't really comfort me as a one person team :-) My impression is that in the earlier ICFP contests the tasks were such that it was possible for a single programmer to achieve a decent result, even if it's unlikely that it's good enough to win. In this case you don't get any points for the reverse-engineering or for the tools, but just for the end result.

(Having written the above, I'm now sure that the eventual winner will turn out to be a single programmer who only started working on the task 8 hours before the deadline).

Code coverage tool for SBCL

jsnell@iki.fi — Thu, 03 May 2007 10:00:00 GMT

SBCL 1.0.5.28 includes an experimental code coverage tool (sb-cover) as a new contrib module. Basically you just need to compile your code with a special optimize proclamation, load it, run some tests, and then run a reporting utility. The reporting utility will produce some html files. One will contain an aggregate coverage report of your whole system, the others will show your source code transformed into angry fruit salad:

For a more substantial example, here's the coverage output for the cl-ppcre test suite.

There are still some places where the coverage output won't be what most people would intuitively expect. Some, like the handling of inlined functions, would be simple to solve. It's just not yet clear to me what the right solution would be. For example in the case of inlined functions the right solution might be suppressing inlining when compiling with coverage instrumentation, or it might be to say "don't do that, then" to the users. Others are fundamentally unsolvable, due to the impossibility of reliably mapping the forms that the compiler sees back to the exact character position in the source file. Hopefully this'll still turn out to be useful in its current state.

If you have any suggestions for improvements, I'd love to hear them.

ILC 2007 Summary

jsnell@iki.fi — Wed, 11 Apr 2007 22:00:00 GMT

I wrote several almost finished blog posts during ILC, but didn't get around to posting them "live" due to the issues with wireless access and a generic lack of time, due to being off having a jolly good time. Then I did some more traveling after the ILC, and didn't manage to get them posted right afterward either. And now that I'm finally back home, most of what I wrote then no longer seems worth posting, since it's lost the immediacy.

So here's a few things that come to mind now.

The good

The organization was stellar in almost all respects. A huge thanks to Nick Levine and anyone else who was involved. Cambridge was just incredibly pretty, and the weather ranged from great to "not bad". There were some very good talks, though disappointingly most of the best ones were from Schemers :-) The last day of talks was particularily good. I had incredible fun meeting old friends, most of whom I hadn't seen for a year, putting faces to names I knew from the net, and talking to completely new people. Special honorable mentions in the latter category go to Jeremy Jones and Richard Brooksby, with whom I had several very interesting and fruitful discussions.

I also got lots of very valuable SBCL feedback and new ideas, for all kinds of things from the GC to the user interface for my code coverage tool for SBCL (work in progress). It looks as if we need to beef up the SBCL marketing department, though. I had several discussions of the form "Q: What would it take to make SBCL do FOO? A: It's already done that for the latest X releases.". In the worst case with the same person asking for three different features in succession, all of which had been implemented :-) For example no-one seems to be aware that SBCL/Slime have stepper support. Not horribly good stepper support, but support nonetheless. Also got to talk shop with SBCL developers and Clozure/ITA people, which is always good. And maybe even managed to offload some ideas that I'd proof-of-concepted, but have no intention of ever properly implementing myself.

Got a surprisingly large number of congratulations on graduating. And the guys had even got me a present (a copy of the Lisp 1.5 manual that Nikodemus had found from a bookshop in Cambridge, MA). Thanks! Conveniently the title of the programming contest for the next ILC was pre-announced as "Lisp 1.5", so the manual might even be useful, not just cool :-)

I think the Ravenbrook guys are going to try integrating MPS with SBCL, since CMUCL didn't work out for them. While it's unlikely to replace the current SBCL GC for licensing reasons (it's currently under a GPLish license), it would be very interesting for two reasons: as a benchmark for the current GC and as a first step towards pluggable GCs. The first one would be good since we know that the SBCL memory management is suboptimal in many ways. It'd be valuable to find out what the real cost of fixing many of those suboptimalities is. As for pluggable GCs, Frode wrote a nice message to sbcl-devel about that. If MPS is better for someone's use case than SBCL's gencgc and they can live with the license, it'd certainly be nice for them to be able to just switch GCs. And of course at some point implement other alternative GCs.

Compared to the ECLMs, surprisingly many people that I talked to weren't yet using Lisp seriously, but were just interested about it. Some might think that this is bad, but I think it's really great that there are people still in that stage who are interested enough to travel to and attend a multi-day Lisp conference. And of course there were a lot more serious Lisp users than newbies.

Overall my ILC experience was very positive. I'll talk next about some bad stuff, but that's just because I believe that you can't just sweep that stuff under the rug.

The bad

I think that program-wise there was maybe a day of talks that could've been discarded with little loss. Or if not a whole day, than at least enough to make the rest of the schedule less tight. For example the History of Lisp presentation was total crap (not just somewhat bad, but "I'd rather listen to an hour of silence"-bad), and the information theory one had no business being presented in a Lisp conference. Given what little I heard of the review process in other cases, I don't understand how the latter ever got accepted.

I understand that people don't really go to a conference for the talks, but that doesn't mean that anything goes. My plea to the next ILC program committee is threefold:

Please invite only speakers with something to say that's relevant to Lisp now or in the future, not in the last millennium.
More specifically, I'm sure there's a temptation to "honor" the 50th birthday of Lisp by historical navel-gazing. Please don't give in to it.
If you don't get enough good submissions, don't accept the irrelevant ones as padding.

My attempts at industrial espionage were mostly a failure. Both Duane and Jans ran out of time before getting around to stuff that would've been both worth stealing. For example Duane didn't have time to demo their profiler, which I'd heard described as the gold standard of Lisp profilers, and of course I can't really try it out myself due to the license. I was surprised that the Allegro equivalent to SBCL's optimization notes didn't have any kind of UI for mapping the notes back to the original source, making it look mostly useless. Or at least Duane, who is probably an expert at reading them, did get confused by the results a couple of times despite it being a scripted demo :-)

[ Which isn't to say that Franz's presentations were bad. I just didn't get much out of them SBCL-wise. ]

The controversial

Some stuff has received a lot of airtime after the conference.

Before the conference I expressed some puzzlement about there being an invited talk about CL-HTTP, which I regarded as a choice that was completely out of touch with the current state of the Lisp world. Seeing the talk didn't change my opinion (oh, wow, still using the White House information system from the Clinton administration as the example?). E.g. when Mallery asked about who had ever used CL-HTTP, and practically no hands went up, unlike with every other similar question that was asked during the conference. But amazingly enough, in the last day two presentaters appeared to be seriously using CL-HTTP. (IIRC they were the RacerPro and XMLisp ones).

Most of the Allegro features that Duane and Jans had time to show were things that SBCL already does in some form. It's just that they're exporting their internals, and in some cases the interfaces don't seem very polished. I guess READ-LINE-INTO (?) wouldn't be a bad addition, but e.g. MEMCPY-UP and MEMCPY-DOWN were just completely wrong.

So I wasn't horribly impressed with what they talked about. But unlike Luke, who was stirring up the debate both at ILC and after, I think that it is a very worthwhile goal to give Lisp users access to low level facilities, and that we really should be suppling non-consing / resource-reusing versions of functions where possible. STRING-TO-OCTETS and OCTETS-TO-STRING are an obvious example where SBCL could be improved.

Yes, it'd be really great to just cons indiscriminately, but no matter what the GC scheme is, there will be programs where consing will be deadly. And yes, it'll mean that code written for performance might be a bit ugly, but it's still better than dropping to C from Python for performance, etc. Of course SBCL users can use many of those low level facilities right now, but most of them are undocumented and unexported, which sets the bar for using them pretty high.

The end

Anyway, it was lots of fun! I hope to see all of you again next year.

ILC 2007 MPS Tutorial

jsnell@iki.fi — Sun, 01 Apr 2007 14:05:00 GMT

Oh, man. My excitement about the CMUCL/MPS integration seems to have been premature :-)

Paraphrases from the early part of the MPS tutorial:

"We didn't actually get too far with the actual implementation of MPS and CMUCL, since we were unable to boostrap CMUCL if we made any (even tiny modifications)." (But apparently they have all the design issues solved).

"Unfortunately Dave Jones who's been doing the work on this is ill and thus not at the conference."

"Used CMUCL rather than SBCL since Carl Shapiro had earlier expressed interest in integrating MPS and CMUCL. No particular reason besides that." (In answer to my question about why they didn't try SBCL if bootstrapping was a problem).

ILC 2007 pre-conference stuff

jsnell@iki.fi — Sun, 01 Apr 2007 13:45:00 GMT

(Stuff from Saturday, before the actual conference starts. Sorry for any typos, I wrote it late last night after half a bottle of wine, and didn't have time to proofread it this morning. And am now in the middle of a tutorial. I'll fix it up later.).

Woke up at 0500. Almost missed the plane lifting off at 0745 despite that, since Taxis were nowhere to be found. Met Martti, fellow Helsinki Lisper, at Heathrow, and was entertained by his tales of British engineering for most of the trip from Heathrow to King's Cross.

The conference accommodation is nice, especially for the price. Except for the British plumbing, but complaining about that is about as original as complaining about left side traffic. I got a room in the top floor, which seems to be an attic that was later converted to dorms. It looks pretty dramatic, in a good way (with the room being horseshoe-shaped and varying in height between 4.5 meters to 0.5 meters). Unfortunately I don't have a camera.

Cambridge looks really pretty. I haven't yet random walked around the city properly, and probably won't have time to do so on this trip. I did go to the conference tour, though. Thanks to Martin Simmons for doing the hard work of punting on the punt that I was on. I didn't get horribly much out of the guided walking tour part, but at least it meant visiting various places that I would never have gone to on my own.

The sexp-formatted conference badges that Christophe designed look sweet, though they're not a big surprise since I'd seen them in the earlier stages.

We had a very nice dinner at a Turkish place that Christophe recommended, and which surprisingly enough was able to give a table for 12 with no warning at 1930 on a Saturday. IIRC the name of the restaurant was Anatolia, and based on some after-the-fact backtracking the location is off the conference-provided map, but probably on the Bridge Street that Sidney Street transforms into in the intersection to St. Johns Street. I really liked the food. Didn't mind the wine either, though I won't pretend that I can make any kind of judgment on its quality.

All of tomorrow's 4 tutorials look interesting, but since they're in parallel I can only do two. The MPS tutorial is a must-see for me. Choosing between industrial espionage (performance tuning in Allegro) and cool Lisp hacks (ContextL) will be tough.

It's now 00:30 (2:30 Finnish time, so I've been up for 19+ hours). Time to get some sleep, and hope that I can get this entry posted tomorrow. No wireless reception in the room, and I couldn't get a wireless connection up in the Library Common Room. Some people reportedly had more luck with it.

Master's thesis accepted

jsnell@iki.fi — Mon, 05 Mar 2007 21:00:00 GMT

My Master's thesis on type inference of dynamic object-oriented languages was accepted last week. With some luck I should graduate this month, though that's still in the hands of the university bureaucracy. Unfortunately the thesis is written in Finnish, so most of you won't be able to read it even in the unlikely chance that you wanted to.

I plan to celebrate this by going to ILC. The schedule and abstracts for the talks were just published. Lots of stuff that looks interesting, and only a couple of total wtfs. I'm intrigued by the mention in Richard Brooksby's tutorial abstract about a CMUCL port that uses MPS for the GC. Does anyone know more about that?

Compilation speed in SBCL over the years

jsnell@iki.fi — Wed, 07 Feb 2007 07:00:00 GMT

SBCL has a reputation of being slow at compiling programs. This has historically been a deserved reputation, since the cleanups done after the fork from CMUCL made the compiler a lot slower. While speed doesn't really matter when compiling individual functions, it's very important for large programs. Even if most Lisp development is done interactively, compiling a function or a file at a time, at some point one needs to ensure that the system still builds from scratch. With a large code base, this can take ages. It doesn't help that doing correct incremental builds for large Lisp systems is challenging. (See Andreas' asdf-dependency-grovel for one way to make it less painful).

In fact, most of my early contributions to SBCL were for making the compiler faster, since on the computer I had at the time an SBCL build took 1.5 hours. And even several workstation upgrades later, I still sigh every time I kick off a full SBCL build.

I just committed another batch of compiler speedup patches, and got to wondering what the cumulative effect of all these improvements over the years has been. So I ran my standard compilation speed benchmark suite on current CVS head, and on SBCL versions from one, two and three years ago. All SBCLs were built for x86, with the default build options. CMUCL 19d was also included in the test for comparison.

Here are the results (click on thumbnail for a larger image):

I'm pretty happy about the consistent trend for faster compilation times, despite new features and optimizations getting added, which usually act to slow down the compiler.

So the next time you're cursing about how long your build of McCLIM - or something similar - is taking, remember that things could be worse. You could be using SBCL 0.8.8 :-)

Cross-referencing facility for SBCL

jsnell@iki.fi — Wed, 06 Dec 2006 01:30:00 GMT

(Darn, I got scooped by Xach.)

As of 1.0.0.18 SBCL has a proper xref implementation, used for answering questions like "where is this function getting called from". I hope it will be more usable than the (very clever) heap-groveling hack that M-x slime-list-callers uses. The intended benefits of the xref over that approach are:

More information: also supports who-macroexpands, who-binds, etc in addition to who-calls.
More usable: Slime will take you directly to the correct form, not just the right toplevel form.
More accurate: The heap-groveling would for example completely miss inlined functions (but see below for some examples of less accuracy).
More reliable: The heap-groveling would also lead to assertions being triggered in the SBCL internals that it was abusing.
Faster: lookups should be an order of magnitude faster.

There are some cases where xref information isn't currently stored. As a rule of thumb, if there's a defun or defmethod involved on some level, xreffing will work. Otherwise it probably won't. The latter case covers code like:

A macro expanded at the toplevel
```
(defmacro def-foo () )
(def-foo )
```
Code inside a lambda that's not inside a named function (slime-list-callers will show this call to BAR, slime-who-calls won't)
```
(defvar *a*
  (lambda () (bar))
```
Code in a defclass initform (likewise)
```
(defclass ()
   ((a :initform (bar))))
```

Code inside a macrolet definition body

(defun foo ()
  (macrolet ((bar ()
               (baz)))
    (bar)))

I have some ideas on how to get xref information recorded also for these, and other similar cases, but haven't yet worked out the details. If you run into any situations (other than the ones listed above) where the xref isn't working as you expect it to, I'd love to hear from you.

To use new xref, you'll also need to upgrade to CVS Slime, and the use the normal Slime cross-referencing commands. Thanks to ITA Software for funding the work on the xref (and on some other SBCL improvements, which I never got around to blogging about).