Arc is a new dialect of Lisp by Paul Graham and others. Recently, Graham has released a working codebase for Arc, which started a small community of people who play with it and improve it. As part of it, there's a world-writable version control repository for Arc called "Anarki".

Seeing that a comprehensive test suite was absent, I decided to start writing one. I decided to use the Test Anything Protocol (or TAP for short), which is a simple-to-implement STDOUT and STDERR based mechanism for test scripts to emit results of their test assertions to the TAP harness. This was partly because I expected the Arc test suite to be written in more than one language, and partly because I have been involved with the development of Test::Run, which is a test harness for TAP, and other TAP efforts, and so wanted to eat my own dog food.

So I've written (and later on enhanced) a rudimentary implementation of a TAP emitter in Arc. This TAP emitting module can be found in arctap.arc in the repository, and is used by the Arc tests. Some tests were written in Perl, primarily to test and fix bugs in the Arc TAP implementation. The Perl tests use Test::More, which is a TAP-emitting module for Perl, and Test::Trap. The latter is used to trap the STDOUT and STDERR of Arc programs that are run by the Perl scripts. This is done to test the sanity of the Arc interpreter and the Arc-TAP implementation.

So far the work on the test suite uncovered several bugs in Arc and one bug in Test::Run, and also resulted in some enhancements to Arc. Plus, I was able to better familiarise myself with Arc, and naturally to write test suite code which helps in preventing future regressions.

One feature of Test::Run that helped here was its AlternateInterpreters plugin (which is an idea I originally derived from something Curtis "Ovid" Poe (also a TAP worker) had blogged about). Currently, my interpreters.conf.yml file reads:

---
- cmd: ./arc.sh
  pattern: \.arc(?:\.t)?\z
  type: regex

This uses ./arc.sh to run the tests by default. The exact configuration is a bit more complicated than that (see Test.sh), but still the prove utility that is part of Test-Harness-2.xx, which is the older TAP harness, would not do. This is because it is impossible to assign a she-bang at the start of the Arc files, because it seems it confuses the Arc interpreter.

Currently 66 assertions in 7 files were written so there's still a long way for a decent coverage of the Arc features. But so far it seems that using TAP for the Arc test suite has been a sound choice, and I'm happy with it, and it probably facilitated putting the test suite together.

(Thanks to infi, mauke and simcop for reviewing earlier drafts of this post.)