Kevin Reid's blog

Improving my web presence, out of necessity. Also, Apache configuration and LJ OpenID

kpreid@mac.com — Mon, 21 Dec 2009 01:58:18 GMT

A machine I used to use to host some web services, bots, and repositories became no longer accessible from the Internet, as a result of which I've had to move what I was serving from it; some to switchb.org, some to personal machines.

I took the opportunity to clean things up a bit, as a result of which I now have better backups, more polished services, and know a little bit more about configuring Apache — though not as much as I perhaps should.

My Subversion repositories are now served over HTTP, and therefore browsable; and they are now backed up daily (using svnsync triggered by a cron job) to my laptop, and thence to all its backups.

(I wasted several minutes on remembering that cron will ignore the last line of a crontab file if it doesn't end with a newline; after listening to me grumbling about this, someone made a suggestion to end the file with a comment, so that the last line is harmless whether ignored or not, and also reminds one of the issue.)

If you have a working copy of one of my repositories (E-on-CL, E-on-JavaScript, MudWalker, Den, etc.), here's a guide to the changed URLs.
My other Tahoe-LAFS volunteer grid storage node is now residing on a machine on my home LAN.
Finally, some simple data-querying web services I wrote for Waterpoint's word games have now been moved to switchb.org; I also took the time to prettify their URLs (no cgi-bin or .cgi) and write documentation.

I haven't yet gotten to working on the bots, darcs repositories, or miscellaneous other stuff I had there.

(Pondering moving my blog over to switchb.org as well so as to not have ads, especially now that I found I can still have LJ-friends by way of OpenID. (Hm, but reading friends-locked posts over RSS might not work since there's no username+password for LJ to accept. Anyone have experience with that situation?))

Apache configuration questions:

If I have multiple variants of a document (e.g. foo.html foo.pdf foo.txt) handled by MultiViews, so the canonical URL of the document is extensionless (“foo”), how do I properly control the default variant to serve in the event that the client does not express a preference via the Accept header? (Without doing so, I found that it would serve the .txt version, whereas I would prefer the HTML.) All that I found that worked was to create a type map file named “foo” with quality values, and force it to be interpreted as a type map using <FilesMatch>. This seems kludgy to me.
What is the right way to serve CGIs, not in a designated cgi-bin directory, and without any .cgi extension in the URL? I initially tried to apply mod_rewrite, but I couldn't get it to work such that /foo internally contacted foo.cgi whereas /foo.cgi redirected to /foo. I resorted to another <FilesMatch> explicitly listing the name of each CGI and doing SetHandler cgi-script.
What is the right way to handle “testing” vs. “deployment” configurations, where the relevant Directory, Location, etc may be different depending on which server or subdirectory the site is set up on? I see that one may use environment variables — should I just set up variables containing the path prefixes for the particular host before including the generic configuration file?

And now for something utterly internet-pop-culturish

kpreid@mac.com — Tue, 15 Dec 2009 02:56:15 GMT

Photo taken by me in a Staples store. Text added with Zach Beane's roflbot.

Unifying status messages?

kpreid@mac.com — Mon, 14 Dec 2009 16:12:03 GMT

Has anyone invented a common protocol for, in a desktop environment, updating status (available/away/do-not-disturb and “what I'm doing” and so on) in social applications? I have 5 different such applications (iChat, Colloquy, Skype, MudWalker, Steam), and I usually don't bother to update all of them since it's too much trouble to do regularly. Is there some protocol that, say, the Linux-on-the-desktop folks have invented for having just one widget to inform everything?

[edit] Clarification: I mention Linux-on-the-desktop just because I don't know much aboutit and there seems to be some innovation from that direction in this kind of area. The desktop in question is Mac OS X; I'm expecting to have to write the software myself; I just want to not reinvent the wheel with regard to protocol.

ASDF PSA: defpackage in .asd is not obligatory

kpreid@mac.com — Wed, 25 Nov 2009 22:06:21 GMT

There seems to be a recurring misconception about writing .asd files. It is not necessary to start your asd with (defpackage :foo.system ...) (in-package :foo.system) to “avoid polluting the package asdf loads your system in”.

Every time an .asd file is loaded, ASDF creates a fresh package to load it in. The relevant code from ASDF is:

(defun make-temporary-package ()
  (flet ((try (counter)
           (ignore-errors
                   (make-package (format nil "ASDF~D" counter)
                                 :use '(:cl :asdf)))))
    (do* ((counter 0 (+ counter 1))
          (package (try counter) (try counter)))
         (package package))))

(defun find-system (...)
  ...
      (let ((package (make-temporary-package)))
        (unwind-protect
             (let ((*package* package))
               ...
               (load on-disk))
          (delete-package package)))
  ...)

So, whenever you load an asd file, the package is fresh and :uses CL and ASDF.

There are reasons to define a package for your asdf system definition.

If you are creating any classes, such as custom ASDF component types, then, since classes are not garbage collected (MOP specifies that the list of a class's subclasses can be retrieved, and no implementation I know of uses weak references for the purpose), naming them with symbols in a consistent package (rather than the asdf temporary package) ensures that reloading the system definition does not create useless additional copies of the classes.
If you are defining functions or other named entities in your system definition (which should of course only be done if they are also necessary to load the system) and want to be able to refer to them, such as for debugging, with nice interned symbol names.
If you want to be able to (load "foo.asd") rather than going through asdf:find-system.

The disadvantages I can think of for defining a package for your system definitions are that the package list becomes more cluttered (only noticeably if for some strange reason you're loading many system definitions and not loading the defined systems), you're using more of the global package namespace (probably not a significant concern if you use a naming scheme like foo and foo.system), and your code is marginally more complex.

So there are good reasons for (and maybe some against) defpackage in your asd, but package hygiene is not one of them.

[Edited 2009-11-26 to be less worded against defpackage after consideration and feedback.]

91% off sale at SparkFun Electronics

kpreid@mac.com — Tue, 24 Nov 2009 23:55:01 GMT

You probably already know about this if you're interested, but SparkFun Electronics will be having, on January 7, a “Free Day” on which you can order up to $100 and pay only shipping (plus any amount over $100). Shipping, to my location by the cheap options, was shown as ~$10 (thus “91% off”, not considering tax).

(I have no affiliation with, and have not ordered anything from, SparkFun Electronics; this is just some info that showed up in my feed reader. As far as I’ve heard they’re good folks.)

SparkFun’s product line is somewhat oriented towards the hip electronics hobbyist, the sort of stuff that Make talks about: Arduino, text and graphic LCDs, blinky light kits, wireless stuff (serial links, GPS, Bluetooth), robots, sensors, breakout boards/modules for surface mount components. Many of these items are high priced, but they have notable cheap stuff too; when I heard about this sale yesterday my first-cut list of 11 interesting items only totaled $62.

Avoiding repeating myself (on the command line)

kpreid@mac.com — Wed, 28 Oct 2009 01:53:09 GMT

I was working on some automated document generation, building the process, revising the input document, and checking the results, and got tired enough of typing "make output/foo.txt && open output/foo.txt" and then going back and editing that line when I wanted to check the PDF version etc. that I wrote a tool which I hope will have more general application as well. I called it “maken”:

#!/bin/sh
# Make files and view the results.
make "$@" && open "$@"

Or it could be generalized into being a combinator, running any two commands, rather than just make and open, specified as the first two arguments:

#!/bin/sh
ca="$1"
shift
cb="$1"
shift
"$ca" "$@" && "$cb" "$@"

(Which, for the Haskell folks, should be called &&&, except that that would be annoying to enter in the shell.)

Google Wave: Tell me what it's good for

kpreid@mac.com — Sat, 24 Oct 2009 01:58:52 GMT

I'm on Google Wave (by way of my participation in GSoC 2009). Tell me what's worth my time to do with it.

My ID is apparently kpreid.switchb.org@googlewave.com. (kpreid was taken, hmph.)

Irony, exhibit 1

kpreid@mac.com — Sat, 17 Oct 2009 14:22:22 GMT

I blame a certain English teacher last year for introducing “FAIL” into my vocabulary.

kpreid@mac.com — Tue, 22 Sep 2009 03:20:08 GMT

Every time you write “ | ” in shell, that's concurrent programming.

This is not a LaTeX quine

kpreid@mac.com — Tue, 22 Sep 2009 01:45:09 GMT

\documentclass{article}
\usepackage{verbatim}
\begin{document}
  \verbatiminput{\jobname.tex}
\end{document}

What do you call a method, er, header in Java?

kpreid@mac.com — Fri, 18 Sep 2009 03:54:36 GMT

What do practicing Java programmers call the part of a method definition that isn't the { body }?

__________/ this bit here \___________
public static void main(String[] args) {
  // not this part
}

A quick search turned up the Java Language Specification, Second Edition calling it a MethodHeader in the grammar, but it doesn't use that term in the text and I want to know what term(s) are used in practice, not solely by language lawyers.

(“Signature” is not correct, as that includes only the method name and parameter types, not modifiers and result type.)

It's that time of day again...

kpreid@mac.com — Fri, 18 Sep 2009 03:29:24 GMT

Both of the non-internet-based computer-related classes I'm taking this semester are (only offered as) once-per-week night classes, nominally ending at 9:30. So I'm talking with someone after class — — gee, it's bedtime already.

GSoC conclusion.

kpreid@mac.com — Fri, 11 Sep 2009 01:09:27 GMT

Late update on Caja-CapTP:

Google Summer of Code is over. I passed based on revised goals, but I'm not happy with the state of the code and I did not complete any significant part of the original plan.

Regarding the items mentioned in my last update:

Write more documentation/comments.
Commit as much of the work-in-progress as I can.
...including the incomplete CapTPConnection, even though its tests don't pass yet, so that the partial work can be counted.

I committed CapTPConnection, and found and fixed enough infrastructure (whenResolved, CommTable, SwissTable, deSubgraphKit, etc.) bugs that it starts up and can do a basic operation or two. It's not useful for anything, but it's a lot closer to running than it was at the time of my last update.

Also, I removed dependencies on 'window' so in principle it will operate on a non-browser (server) JavaScript implementation. This has not been exercised because there is no browserless driver for the test scripts yet.

I stated that I would continue working on Caja-CapTP past the GSoC work period; however, with the time occupied by schoolwork, I have not had time or brain space to do so yet. I am not going to abandon the project.

an approximation, the usefulness of which depends on the application

kpreid@mac.com — Fri, 11 Sep 2009 00:54:48 GMT

Java programming class (yes, I know Java already) today, introducing loops. Task: write a factorial program. I wrote mine to use int arithmetic if possible, floating-point if not. Test the edge cases, of course. It says:

500! ≈ Infinity

I'm glad I used the “approximately equals” sign.

Tell me what college to transfer to. (Or, Decloaking, part 2)

kpreid@mac.com — Tue, 08 Sep 2009 14:32:22 GMT

As I wrote before, I am currently attending MVCC, a two-year college, and need to choose a college to transfer to (for a bachelor’s in computer science).

So, tell me what college(s) I ought to consider.

(I have of course also heard the advice that it doesn’t matter that much, but I've got to choose from some short-list...)

I am willing to consider any location in the contiguous US, but have been looking mostly at New York choices as a way to make the research list manageably short.

I am not looking for vocational training; I can learn this year’s or last year’s hot technologies just fine by myself, thank you. I’m looking for general education (“well-rounding”, shall we say), useful theory, and practice in thinking. I wish to avoid a high-pressure or competitive academic environment. Also, at MVCC, I have greatly appreciated the instructors’ approachability, availability, and even willingness to acknowledge mistakes.

Outside of education, I am particularly interested in there being social opportunities for the socially awkward; chances to talk to like-minded people (i.e. total geeks), and opportunities to talk to unlike-minded people (for the practice!).

Please give me your recommendations. Just a name, your personal experience, comments on others’ recommendations, whatever info you’d like to share.

(On the other hand entirely, I’d also consider going directly from MVCC to a full-time job given the right opportunity. This will be the topic of an upcoming post.)

Input wanted: posting short stuff

kpreid@mac.com — Tue, 01 Sep 2009 19:53:55 GMT

I've been having a lot of thoughts lately I'd like to publish, but seem a little bit too short for A Blog Post. Some options

Considered signing up for Twitter. Pro: short stuff is expected there; participating in the hot new thing; people interested in short-form will be on twitter and using the follow feature. Con: Reliability problems; YA thing to manage credentials and backup for; no hyperlinks; my favorite username is taken.
I could post stuff here. Particularly, LJ supports titleless posts.
I could use some other site, or build my own publishing system. [er, why?]

On reflection, I'm thinking to just post the stuff here and not have yet-another-distinct-place/site.

Readers, what would you prefer?

(This post is tagged “lisp” solely so that Planet Lisp will pick it up.)

GSoC update: Stuff done, more to do.

kpreid@mac.com — Wed, 12 Aug 2009 15:58:02 GMT

Stuff done:

Implemented generic literals.
Implemented cyclic references in Data-E.
Various cleanup, debugging support, and committing of stuff previously written.
Wrote a wiki page documenting Caja-CapTP's major components and files.

Stuff to do:

Write more documentation/comments.
Commit as much of the work-in-progress as I can.
- ...including the incomplete CapTPConnection, even though its tests don't pass yet, so that the partial work can be counted.

Brain-dump for the beginning JavaScript programmer

kpreid@mac.com — Wed, 12 Aug 2009 15:48:29 GMT

A while ago I got the idea to write a “my experiences with JavaScript” post; unfortunately, I’ve forgotten what I was going to put in it. However, yesterday someone said approximately “I need to learn JavaScript in 3 hours, what should I know?”, and I thought of quite a few not-immediately-obvious things to point out.

General

Beware the bad tutorials.
As far as I know, Mozilla’s Core JavaScript 1.5 Reference and Core JavaScript 1.5 Guide are not bad. (I start with the former for library/standard objects info, and the latter for language syntax.)
Anything Crockford says goes.
It’s probably a good idea to use JSLint but I haven’t myself.
If you use Firefox with Firebug, or Safari, write your code to use the console object, if present; it is a very nice tool for your inevitable print-debugging, and Firebug’s version even lets you click on objects shown to inspect them.

Miscellaneous practice

Don’t ever write a busy-wait/polling/delay loop. It won’t work and it'll waste your users' time. JavaScript in the browser is an event-driven system; use setTimeout (callback once) and/or setInterval (callback repeated with specified period) as appropriate. You can write a background loop with setTimeout: function loop() { if (termination condition) { return; } else { ...do something...; setTimeout(loop, 0); } }; loop();
Don’t throw "foo"; but throw new Error("foo"); because that gives Firefox, at least, an opportunity to insert call stack information.
JavaScript has two "nothing" values: undefined, of type "undefined", and null, of type "object". undefined is returned by failed property lookups and the default return value of functions. null doesn’t show up much unless you use it, and is useful in your code for distinguishing deliberate absence from an accidental missing value (bug).

Scope

Local variables are declared with "var", but they are scoped to the nearest enclosing function, not the nearest { block }!
Functions can be closures. Use this when it’s useful.
Don’t write setTimeout('foo()', delay) (because string-eval is a bad habit), write setTimeout(foo, delay) or setTimeout(function () { foo() /* for more complicated things */ }, delay).

`this`

Don’t call a user-defined constructor (say, Foo), as Foo(), only new Foo(), or it will overwrite global variables instead of creating an object. (new binds this to a new object inside the function; a regular call leaves it as the global object.)
foo.bar() is the same as (foo.bar)(), and is not the same as var x = foo.bar; x(). The former binds this inside the function to the foo object; the latter does not.

Properties and variables

for (x in y) ... does not do what you might first expect; it iterates over all properties of the object, even inherited ones.
If you see $('foo'): That is not some magic new operator. $ is an identifier character. That expression is a function call. From what I hear, both Prototype and jQuery define functions named $.
foo.bar is exactly equivalent to foo["bar"].
foo.bar returns undefined if bar is not a property of foo; bar throws an error if bar is not a property of whatever the environment is. So if you’re doing browser feature detection, you write code like if (window.console), not if (console), even though window is the global object and its properties are variables.

Numbers

All numbers are double-floats, not integers. They can exactly represent integers up to about 2⁵³, but you should still keep the nature of floating-point arithmetic in mind.
The == operator does a scary number of implicit conversions; use === unless you know == does what you want.
- Even === compares floats according to IEEE floating point numeric comparison; this means that NaN !== NaN.

GSoC update: Yes, you may change it.

kpreid@mac.com — Sat, 08 Aug 2009 19:56:04 GMT

Had a bit of a breakthrough in debugging facilities: realizing that there's no reason I shouldn't modify my local copy of the Cajita runtime to provide more debug information (and it doesn't matter if it subtly breaks security, even, as long as Caja-CapTP is still compatible with the proper version).

Only a week or so left. I've discussed the matter with MarkM, and he pointed out (again) that I should work toward having useful-to-other-people components even if I haven't finished the one I intended to do.

Toward this end, I am going to work on Data-E (which is already part of the Caja-CapTP project) as a standalone serialization utility (for network and disk) as I understand it will be useful for Shakhar.

Todo items:

Implement generic literals (description) to make it more JavaScript-natural.
Implement cyclic object graph handling using Ref promise implementation already done.
Commit all in-progress work as reasonable changesets (even the tests that don't yet pass).
Document what works, design decisions, etc.
Continue to work on CapTP itself as time permits.

I intend to continue working on the Caja-CapTP project beyond GSoC; both simply because it is a project of mine, and to mitigate my failure to complete the planned work on schedule. However, for the next few months it will have to compete with my college work rather than my procrastination.

Want to write some miscellanous Common Lisp code?

kpreid@mac.com — Sun, 02 Aug 2009 18:10:35 GMT

Rosetta Code is a wiki where a variety of programming problems and language features are demonstrated in many languages, allowing language learning and comparison of features and paradigms.

If you'd like to contribute Common Lisp code to the project, I've just completed a classification of tasks in CL (like my prior one for E), so that it's easy to find the kind of problem one wants to work on at the moment.

One thing I find interesting about working on Rosetta Code is that many tasks bring with them some particular perspective — someone's notion of how programming necessarily works — and it can be a challenge to figure out the analogous way, or best way, to do it in your language is, and then explain it.

GSoC update: Hurry up and procrastinate.

kpreid@mac.com — Mon, 13 Jul 2009 13:33:56 GMT

I'm having severe getting-around-to-actually-doing-the-work problems; I am far behind schedule. I think the problem is framing this as “work” rather than just another of the projects that I've found interesting and tinkered with. (I also blame Team Fortress 2 and Rosetta Code for providing attractive distractions...)

I've ported the “Ref” facility from E; this is necessary as CapTP is built on Ref semantics (promises, proxies, broken refs). I hope to very soon get the actual CapTP connection module up, then (as I wrote before) “define, document and implement a CapTP-over-HTTP protocol”.

(I previously mentioned implementing the Surgeon (serialization tool) but I then remembered that that's actually irrelevant to CapTP.)

Also: working without Internet access removes a whole lot of potential distractions — and one’s access to online documentation. Luckily I had some source code around which provided examples.

Common Lisp hackery: writing fasls from non-file input

kpreid@mac.com — Mon, 13 Jul 2009 13:09:44 GMT

Common Lisp provides compile-file whose purposes is to convert a CL source file into an (implementation-defined) format which usually has precompiled code and is designed for faster loading into a lisp system.

compile-file takes the pathname of a textual lisp source file. But what if you want to compile some Lisp code that's not in a file already, perhaps because you translated/compiled it from some not-amenable-to-the-Lisp-reader input syntax, or because it contains unREADable literals? You can use a "universal Lisp file", which I know two ways to create (use whichever you find cleaner):

(cl:in-package :mypackage)
#.mypackage::*program*

(cl:in-package :mypackage)
(macrolet ((it () *program*))
  (it))

Suppose this is in "universal.lisp". Then to use it:

(defvar *program*)

(defun compile-to-file (form output-file)
  (let ((*program* form))
    (compile-file #p"universal.lisp"
                  :output-file output-file)))

This is just a minimal example; you'll also want to appropriately handle the return value from compile-file, provide an appropriate pathname to the universal file, etc. For example, here's an excerpt of the relevant code from E-on-CL, where I have used this technique to compile non-CL sources (emakers) into fasls:

(defparameter +the-asdf-system+ (asdf:find-system :e-on-cl))
(defvar *efasl-program*)
(defvar *efasl-result*)
...
(defun compile-e-to-file (expr output-file fqn-prefix opt-scope)
  ...
  (let* (...
         (*efasl-program*
           `(setf *efasl-result*
                  (lambda (...) ...))))
    (multiple-value-bind (truename warnings-p failure-p)
        (compile-file (merge-pathnames
                        #p"lisp/universal.lisp"
                        (asdf:component-pathname +the-asdf-system+))
                      :output-file output-file
                      :verbose nil
                      :print nil)
      (declare (ignore truename warnings-p))
      (assert (not failure-p) () "Compilation for ~A failed." output-file))))

(defun load-compiled-e (file env)
  (let ((*efasl-result* nil)
        ...)
    (load file :verbose nil :print nil)
    (funcall *efasl-result* env)))

Note that the pathname is computed relative to the ASDF system containing the universal file; also note the use of a variable *efasl-result* to simulate a "return value" from the compiled file, and the use of a lambda to provide a nonempty lexical environment, both of which are features not directly provided by the CL compiled file facility.

Web site moved; feedback wanted

kpreid@mac.com — Sun, 14 Jun 2009 04:37:10 GMT

I have moved my web site to a new location: http://switchb.org/kpreid/.

I have done this because Apple has announced they are shutting down homepage.mac.com. While they offer a replacement, I do not want to have to change my URLs ever again, so I bought a domain name.

As well as moving the files I have updated my picture, changed some of the URL layout, updated some miscellaneous information, added an “About me” section for the front page, and fixed broken external links. Please let me know if I missed anything...

or even just critique it without respect to what's new.

GSoC update: Going both ways.

kpreid@mac.com — Tue, 02 Jun 2009 18:52:11 GMT

I'm not making progress as fast as I would like; mostly due to assorted distractions rather than Doing The Work.

That said, I've gotten both ends of serialization implemented, at least for non-circular cases: the deJSONTreeKit and deSubgraphKit have both builder and recognizer implementations. (For a terminology introduction, see the serialization paper.)

Next up is the surgeon (high-level serialization interface), and designing uncallers suitable for JavaScript. I definitely need to increase my rate of progress to get this done on schedule.

GSoC update: It runs!

kpreid@mac.com — Sat, 30 May 2009 03:58:06 GMT

I completed enough of the E-on-JavaScript improvements, and wrote the beginnings of one Data-E kit in Cajita, together with the Updoc tests for it, and a Makefile to cajole the JavaScript and "animate" the Updoc - convert it into a HTML page that actually runs the tests.

I also improved various readme files and pages, hopefully such that someone else can get it all installed on their own system starting from my project page without too much prior knowledge.

Near-term plan: Put aside the EoJS, and get the Data-E kits working; then the surgeon; then the CapTPConnection; then define, document and implement a CapTP-over-HTTP protocol.