Subject: Freecell Solver's Test Suite + Conversion to CMake

From: Shlomi Fish <[email-masked]>
Date: Thu, 31 Jul 2008 13:27:04 +0300

Hi all!

(All the lists should have a reply-to, so please only reply-to one list).

I recently resumed work on Freecell Solver (FCS) somewhat more intensively. 
Having gotten tired of Autoconf/Automake/Libtool (a.k.a Autohell), I decided 
to convert to CMake ( http://www.cmake.org/ ) for its configuration stage. 
However, since FCS lacked automated tests, I was afraid that by the 
conversion I would break something. (It's kinda paranoid, but still).

As a result, I started working on a test suite. The first tests I wrote were C 
files generated by Perl and Template toolkit to test the card input/output. 
It was a start but not enough.

Then I started working on http://fc-solve.berlios.de/verify-code/ , which is a 
Perl CPAN module to verify solutions of Freecell-like games. At first, it 
could only do Freecell, but was eventually made more generic and can now 
solve many other variants. It is primarily intended for Freecell Solver, but 
is generic enough to be of use for other solvers.

Using it, I added tests that verify the entire solutions of about 10 deals and 
configurations of various games. Afterwards, I also made sure that the exact 
output of the tests remain the same (an output regression) by keeping track 
of the SHA-256 hash of the solution.

Then I came to what I wanted - CMake. I began converting FCS to CMake here:

http://svn.berlios.de/svnroot/repos/fc-solve/branches/conversion-to-cmake/

It's naturally a long process, because CMake is not backwards compatible with 
the GNU Autotools (and for a good reason, because they are very quirky), but 
I've been making progress. Hopefully, I can make use of my experience with 
converting FCS to CMake in working on http://thewml.org/ .

I should also note that someone I met on IRC expressed interest in 
contributing to FCS. However, he hasn't even played Freecell yet, so it may 
take some time, until he actually contributes.

Regards,

	Shlomi Fish

-----------------------------------------------------------------
Shlomi Fish       http://www.shlomifish.org/
Interview with Ben Collins-Sussman - http://xrl.us/bjn8s

I met a guy in the bar, talked to her and she gave me her phone number.

---

From: "Nadav Har'El" <[email-masked]>
Date: Mon, 4 Aug 2008 09:55:50 +0300

> to convert to CMake ( http://www.cmake.org/ ) for its configuration stage. 

Hi Shlomi, can you please tell us a bit about this "CMake", of which I have
to admit I never heard? What makes it better than autoconf (forget automake,
I consider it cosmetics) - at least for you? Does it have any downsides
compared to autoconf?

For reference, here is what I use autoconf for. In the (good?) old days,
not all machines I had access to were PCs with Intel CPUs running Linux.
I found myself compiling the same program I wrote on Linux, Solaris,
DEC Alpha, AIX, Silicon Graphics, and AT&T System 5, all in the same week.
And each of those systems had its own compiler peculiarity, different
installed software, library differences, and most notorious - compiler bugs
(demonstrating, by the way, the point that free software *does* have fewer
bugs). This is where autoconf came in extremely handy. Does CMake also
aim to help you in this scenario - of people who don't use Windows or Linux?

Nadav.

-- 
Nadav Har'El                        |           Monday, Aug  4 2008, 3 Av 5768
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |Those who beat their swords into
http://nadav.harel.org.il           |plowshares will plow for those who don't.

---

From: Shlomi Fish <[email-masked]>
Date: Mon, 04 Aug 2008 11:50:55 +0300

Hi Nadav!

On Monday 04 August 2008, Nadav Har'El wrote:
> On Thu, Jul 31, 2008, Shlomi Fish wrote about "[hackers-il] Freecell 
Solver's Test Suite + Conversion to CMake":
> > I recently resumed work on Freecell Solver (FCS) somewhat more
> > intensively. Having gotten tired of Autoconf/Automake/Libtool (a.k.a
> > Autohell), I decided to convert to CMake ( http://www.cmake.org/ ) for
> > its configuration stage.
>
> to admit I never heard? 

Well, I have mentioned it in passing in the Haifux mailing list in relevance 
to the Autotools presentation:

http://hamakor.org.il/pipermail/haifux/2008-July/000356.html

And Artium mentioned it on his blog:

http://art-blog.no-ip.info/newpress/blog/post/146

You can also read about it in the Wikipedia:

http://en.wikipedia.org/wiki/CMake

> What makes it better than autoconf (forget 
> automake, I consider it cosmetics) - at least for you? Does it have any
> downsides compared to autoconf?

OK, here are the advantages as I see them:

1. Its files are written in a consistent high-level language, whose 
interpreter is written in C++. This is instead of Autoconf's 
m4-generating-Bourne-shell setup, where both m4 and /bin/sh are quirky and 
get broken very easily.

2. Configuration and compilation are much faster than in Autoconf, yielding a 
faster development process.

3. It has a BSD License instead of the Autotools' GPL (which may not be very 
relevant, but it's still a nice bonus).

4. It causes much less unexpected problems and regressions than Autoconf. 
Autoconf requires constant maintenance, while CMake tends to keep very good 
backwards compatibility.

5. It has some user-interfaces for the configuration: ccmake (Curses-based), 
cmake-gui (Qt/Win32), etc. 

6. It merges in the functionality of Automake and libtool, both in a more 
consistent and less error-prone way.

7. CMake contains CPack, which allows to prepare source and binary 
distributions of the projects with ease.

8. It also supports CTest which is an interface to an automated testing 
framework.

9. Has native support for Win32.

10. Since CMake is installed separately on the system, the source pacakges are 
smaller. I was able to reduce the size of the FCS tar.gz from over 500,000 
bytes to slightly above 200,000 bytes.

-----------------

As for the downsides, Artium mentioned them in his blog:

http://art-blog.no-ip.info/newpress/blog/post/146

1. No support for "make uninstall".

http://www.cmake.org/Wiki/CMake_FAQ#Can_I_do_.22make_uninstall.22_with_CMake.3F

2. No support for building both static and shared libraries.

http://xrl.us/omn7i

3. Poor documentation (I personally feel that the Autotools documentation is 
much worse).

4. Inadequate Defaults (building a debug version without optimisation as 
default).

---------------

Personally, I feel that CMake is still better.

> aim to help you in this scenario - of people who don't use Windows or

Yes, it does. Reading from the wikipedia:

{{{{{{{{{{{{
Support for cross-platform builds, and known to work on Linux, other POSIX 
systems (including AIX, *BSD systems, HP-UX, IRIX/SGI, MinGW/MSYS and 
Solaris), Mac OS X and Windows 95/98/NT/2000/XP,
}}}}}}}}}}}}

CMake is a GNU Autotools replacement, on everything that is implied from that.

Regards,

	Shlomi Fish

-----------------------------------------------------------------
Shlomi Fish       http://www.shlomifish.org/
Rethinking CPAN - http://xrl.us/bjn7p

---

From: "Nadav Har'El" <[email-masked]>
Date: Mon, 4 Aug 2008 18:11:26 +0300

> m4-generating-Bourne-shell setup, where both m4 and /bin/sh are quirky and 

The whole idea of Autoconf's "m4-generating-Bourne-shell setup" is that
the result (the bourne shell) can run on any computer without needing to
install anything. Therefore, whatever input language your autoconf alternative
has, generating Bourne shell from it is a *good* idea, I think...

So, if CMake doesn't generate Bourne shell, does it mean your users will
need to install it before they can compile your code? Don't you consider
this a problem?

See more on this below.

> faster development process.

This surprises me. Compilation speed really has nothing to do with autoconf -
CMake can't make gcc (or whatever) any speedier. And while running "configure"
for the first time isn't very quick, normally running for the second time is
indeed quick (because autoconf caches its results).

> consistent and less error-prone way.

Sounds interesting. Like I said, I'm not a big automake fan, so indeed I'd
like to see a better approach .

> 9. Has native support for Win32.

What is Win32? Who uses that system anyway? :-)

> bytes to slightly above 200,000 bytes.

What does this matter?
As a user, given the choice between downloading an extra 300,000 bytes
(which takes a few seconds on today's internet) and having to install an
entire package I don't have (CMake), I'd definitely prefer the first option..

-- 
Nadav Har'El                        |           Monday, Aug  4 2008, 4 Av 5768
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |Are you still here? The message is over.
http://nadav.harel.org.il           |Shoo! Go away!

---

From: Shlomi Fish <[email-masked]>
Date: Mon, 04 Aug 2008 18:34:10 +0300

Hi all!

All hail my new sig. I replaced my old one that was mis-understood in several 
ways due to being taken out of its original context.

On Monday 04 August 2008, Nadav Har'El wrote:
> On Mon, Aug 04, 2008, Shlomi Fish wrote about "Re: [hackers-il] Freecell 
Solver's Test Suite + Conversion to CMake":
> > m4-generating-Bourne-shell setup, where both m4 and /bin/sh are quirky
> > and get broken very easily.
>
> install anything. Therefore, whatever input language your autoconf
> alternative has, generating Bourne shell from it is a *good* idea, I
> think...

Bourne shell (and especially the one generated from Autoconf) is very 
limiting, error-prone, and tends to break a lot. I recall many times when I 
needed to tweak ./configure scripts by hand due to a bug or limitation there.

So it's not a good idea to rely on it. If I can:

1. Make sure I have prepared more consistent, usable and high-level (e.g: 
CMake, Perl, Python, etc.) framework.

2. Make sure that it is portable enough.

3. Require the user to install it there.

4. This in turn will run the "./configure" stage and possibly other stages.

It will save a lot of future frustrations on my and my user's part.

> need to install it before they can compile your code? Don't you consider

Yes, the users will need to install it before they compile my code. But it's a 
simple matter, and then they can enjoy CMake for life with all the programs 
that require it (see http://www.cmake.org/Wiki/CMake_Projects ).

So I don't consider it a problem. Forcing one to rely on the least common 
denominator that is pre-installed on all UNIX-like systems is a bit absurd in 
this day and age.

>
> See more on this below.
>
> > 2. Configuration and compilation are much faster than in Autoconf,
> > yielding a faster development process.
>
> This surprises me. Compilation speed really has nothing to do with autoconf
> - CMake can't make gcc (or whatever) any speedier. And while running
> "configure" for the first time isn't very quick, normally running for the
> second time is indeed quick (because autoconf caches its results).

Maybe it just feels faster, or the makefiles are less bloated, and don't have 
the long GCC command line. Don't know. I'm doing some stuff that's not 
GCC-related.

> What is Win32? Who uses that system anyway? :-)

A few people, apparently. It's a priority for me, and for many other FOSS 
projects.

>
> > 10. Since CMake is installed separately on the system, the source
> > pacakges are smaller. I was able to reduce the size of the FCS tar.gz
> > from over 500,000 bytes to slightly above 200,000 bytes.
>
> What does this matter?

CMake is only one package, and it's probably avialable from your 
distribution's sources (or else they won't be able to build KDE 4). And you 
can save the extra 350,000 bytes (possibly more as more Autoconf 
functionality is there) times and again when downloading source packages and 
their new versions. A stitch in time saves nine.

Regards,

	Shlomi Fish

-----------------------------------------------------------------
Shlomi Fish       http://www.shlomifish.org/
What Make Software Apps High Quality -  http://xrl.us/bkeuk

Shlomi, so what are you working on? Working on a new wiki about unit testing 
fortunes in freecell? -- Ran Eilam