Subject: Cross UNIX Portability - Facts and Myths

From: Shlomi Fish <[email-masked]>
Date: Sat, 3 Jan 2004 21:57:40 +0200 (IST)

Note: I wrote this by inspiration from a comment I saw in a Joel on
Software article. I posted it to the Joel on Software forum as well:

http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=99574&ixReplies=1

Enjoy!

Regards,

	Shlomi Fish

Cross UNIX Portability - Facts and Myths
----------------------------------------

The purpose of this post is to specify the various issues that pertain
to writing cross-platform UNIX applications. It is not meant to tell
you exactly how to avoid these problems. That where books like
the excellent "Porting UNIX Software" by Greg Lehey
( http://www.oreilly.com/catalog/port/ ) or "Advanced Programming in the
UNIX Environment" by W. Richard Stevens (which I haven't read yet but have
heard only positive things about). However it is meant to be exhaustive.

1. Cross-Architecture Programmers need to be aware of several things

Some architectures are little-endian (like Intel x86's), others are
big endian (like Power PCs or SPARCs). Some are 32-bit, others are 64-bit.
Padding of struct fields vary between the architectures as well. Running
into these things can easily be avoided by a clueful developer, but he
needs to be aware of them.

One incident that I remember was a Windows code where the content of the
struct was written straight to the disk, thus designating the file format
we used and distributed. (i.e: "write(my_file_handle, &mystruct,
sizeof(mystruct));"). This is one thing that an experienced UNIX programmer
won't do. And I was told the code was written by someone who was better at
Object Oriented Programming than anyone else in Tel Aviv.

2. Writing software that compiles everywhere is hard ; Writing software
that runs perfectly everywhere is trickier.

Getting software to compile on a different UNIX platform is _relatively_ easy.
Tools like GNU Autoconf/Automake/Libtool can help you with it.

Getting a software to run equally well everywhere is tricky. I recall a time
that a simple sockets' program my partner and I wrote and executed on
perfectly on  Linux, failed from some reason on Solaris. We did not know
what was the problem for a long time, until we Googled it and found that we
had a problem in our program that occurs only on Solaris and not on Linux,
and we were easily able to write a workaround that worked equally well on both platforms.

Of course, had it been a larger program with much more system calls, then
making sure it is portable would have been much harder.

However, if your program is written well, actively tested by your QA
engineers or volunteers, it should run equally well on every UNIX platform.

3. The GNU Autotools - a Two-Edged Sword

Autoconf/Automake/Libtool contain some very good functionality which is a
must for writing portable code. However, they are very, very hard to use as
they assume the greatest common denominator of a working UNIX system. One can
work with them, but it also take a lot of time and frustration to do so.
Recently, we have seen an inflation of open source "start up" projects to
replace them with something better. Most of them did not see a significant
use yet. For a partial list, check:

http://vipe.technion.ac.il/~shlomif/software-tools/

4. UNIXes come in all shapes and sizes

UNIXes vary in their quality, compatibility and feature set. It is generally
assumed that Linux and Solaris (and as far as I know the BSDs too) are the most
complete and "just working" of them, and from there it is going steadily
downhill. I once heard an experienced hacker say that "HP-UX is not UNIX
and AIX is even less than that".

Again, the books I recommended may give some perspective on what to expect
in this regard.

5. Abstraction Libraries - a Partial Solution that is better than nothing.

Many abstraction libraries have been developed for UNIX (see
http://vipe.technion.ac.il/~shlomif/abstraction/) They abstract the
system functionality in a set of layers above them, that are usually
saner, and more well-documented and centrally documented. Several software
packages use them instead of implementing their own ad-hoc proprietary
abstraction logic, but it's hard to tell whether this trend is growing or
not. It is up to you to evaluate the suitability of them for your projects.

6. Portability with Win32 - Impossible but Doable

So far, a great deal of prominent UNIX software was ported to Win32 (either
open source or commercial). Usually it involved a great deal of difficulty
as Win32 does not natively support a large number of UNIX mechanisms, that
purely UNIX hackers take for granted. Nevertheless, it is doable. However,
not all projects found it a high priority to do so, for various reasons.

7. C++ Portability - g++ or bust

C++ compilers vary a great deal in implementation of the C++ standards
and in which features are supported. Mozilla's C++ Portability Guide define
what is the gcd of this:

http://www.mozilla.org/hacking/portable-cpp.html

As can be easily derived this is C++ that is not quite C++. An anecdote is
that a friend of mine (that I highly admire his skill as a programmer) got
burned once when he wrote code with "friend" methods that compiled fine on
Linux, and did not work properly in a certain proprietary compiler that
was needed to compile it for the Solaris target platform. He had to eliminate
the use of friends classes.

The only real solution is to restrict your use to the only cross-platform,
open source, compiler that implements a broad enough subset of the language
to have breathing air with - g++ of the GNU Compiler Collection. This compiler
is written in ANSI C and offers true compatibility across platforms, with
good code performance. So, my suggestion, is to restrict your work to g++
and possibly to relatively new versions of the other compilers as well.

8. Perl/Python/PHP/Ruby/Tcl - The Pot at the end of the Rainbow

Perl and friends are high-level P-code interpreted virtual machines which
transparently abstract all the system's functionality in neat convenient
packages. Each language like that, has its own advantages and disadvantages,
and discussing the pros and cons of them are out of scope of this document.

The important thing to remember is that it is much easier to write programs
that will work well on one modern platform as well as on any other. As such
, and because of their good support of other useful paradigms, they have seen
wide deployment by the "hackers" crowd, who used them for writing more and
more software. This trend seems to grow.

Note that sometimes, speed of execution prevents various distributable software
from being written in these languages, as they tend to perform quite poorly
compared to the most optimized C code. (or in many cases even a very sloppily
written one). Nevertheless, they do fill a growing niche quite competently.

Still, one has to stress that even with these languages, portability can

9. Java - nice try, but...

Java was over-hyped as a "write once ; run everywhere" solution. As someone
once noted it was more of a "write once ; debug everywhere" one, at least at
its beginning. Now it is much more stable, (and less hyped than it had been),
and so seems to be used appropriately.

Java tried to combine the best elements of C++ and those of high-level
languages like Smalltalk. It was a good attempt, but most Perl and Python
hackers felt it hardly went far enough for it to be usable. See for instance
Paul Graham analysis of it in his "Java's Cover" essay:

http://www.paulgraham.com/javacover.html

Java probably has its niche, but it seems that most of the uses Sun would
have wanted it to believe it would fit into, can better be fulfilled in
Perl, Python and friends, or alternatively in a carefully written C or C++
code. Add to that the licensing and availability issues than its vendor imposes
on them, and you'll understand why it isn't very popular.

10. Conclusion

Writing a software that will run on most modern UNIX systems out there is
possible, but requires clueful, knowledgeable developers. (And users or
QA engineers who will assist him on testing it everywhere it needs to run on.
) Restricting yourself primarily to a certain number of UNIX flavours
(probably just Linux) will make your life much easier.

On the other hand, writing code that will run on Windows, requires testing
on a great deal of Windows variants after compilation. And when developing
solely for Windows, your code can probably never be able to run anywhere
except there. (not including the various recent .NET/Mono focus).

Writing software for UNIX or a particular UNIX flavour offers a more
open-source environment than Windows, reduced cost of tools (albeit perhaps
not reduced TCO - no one seems to know for sure), components that as a
general rule, "just work", and if you're using the right tools and
methodologies, even portability to Win32. UNIX Portability is a fact and
not an ideal, as many open source packages are known to compile and
work on a large number of modern and legacy systems.

----------------------------------------------------------------------
Shlomi Fish        [email-masked]
Home Page:         http://t2.technion.ac.il/~shlomif/

Writing a BitKeeper replacement is probably easier at this point than getting
its license changed.

	Matt Mackall on OFTC.net #offtopic.