Subject: The Great Divide

From: Shlomi Fish <[email-masked]>
Date: Sun, 21 Dec 2003 20:30:00 +0200 (IST)

Recently it was rumoured that Windows Longhorn will contain a shell
called "Microsoft Shell", or MSH for short -

http://weblogs.asp.net/jnadal/posts/34413.aspx

Basically, it's a .NET/WinFS based shell, with Object-Oriented semantics
similar to Python or Perl or whatever. But it is not compatible with them,
or with the existing Windows shells of CMD.EXE and WSH (Windows Shell).
While it offers some interesting concepts, and is sort of an evolutionary
innovation, the question is why didn't Microsoft settle for something
more cross-platform, like Perl/Python/Ruby, and had to re-invent another
incompatible proprietary layer that they will later have to maintain.

This made me think that the computer world today has a sort of "a Great
Divide" between Windows and the UNIX world. Microsoft, and other Windows
vendors, do not seem to care about making sure the UNIX world could benefit
from the advancement it makes in Windows, and sometimes does not duplicate
similar functionality from UNIX in an exactly compatible way. On a similar
vain, portability to Windows is sometimes too big an effort to cause many
UNIX-based developers to not bother with porting their software their.

At this point, I'm not sure I can make an observation about the future, but
this is the status quo. There are two classes of systems out there, with
two different APIs, a large amount of incompatiblity and two different
philosophies. Will Windows ever become another UNIX-compatible operating
system? Will UNIX see a rise of high-quality and complete user-centric
programs that already exist in Windows? Will one of the worlds (probably
the Windows one) become obsolete? Only time can tell. In the meantime, I'd like
to try to define some of the differences between the two worlds.

1. Developer or User-centric

This was borrowed from the latest "Joel on Software" article (
http://www.joelonsoftware.com/articles/Biculturalism.html), and I should
note that I haven't settle my mind if Joel is correct in his analysis.

What Spolsky claims is that UNIX programmers want to make software that is
usable for other programmers, while Windows programmers want to make software
that is usable for non-programmers. Obviously, there are notable exceptions of
this in both worlds. Still he illustrates his point with this thought
experiment:

<<<
Suppose you take a Unix programmer and a Windows programmer and give them each
the task of creating the same end-user application. The Unix programmer will
create a command-line or text-driven core and occasionally, as an afterthought,
build a GUI which drives that core. This way the main operations of the
application will be available to other programmers who can invoke the program
on the command line and read the results as text. The Windows programmer will
tend to start with a GUI, and occasionally, as an afterthought, add a scripting
language which can automate the operation of the GUI interface. This is
appropriate for a culture in which 99.999% of the users are not programmers in
any way, shape, or form, and have no interest in being one.

I'll just let you decide if it's true.

2. In UNIX: Open Source Prevails; in Windows: not necessarily

From the history of UNIX one can judge that open source solutions have tended
to see much larger acceptance than proprietary ones. For instance, the NeWS
graphics system did not see great acceptance while X-Windows (which was much
inferior) did, because X was free while NeWS wasn't. Similarly, NFS prevailed
where the proprietary AT&T alternative did not, because NFS was open source.

Proprietary solutions seem to be isolated and most of the time ultimately being
replaced by open source equivalents.

In the Windows world however, there is plenty of proprietary software. In fact,
most people can probably use Windows for a long time, without encountering any
trace of an open source piece of software.

3. In UNIX: Development is Distributed and Vertical; In Windows:
Development of the core services and applications is centralized

In UNIX, development of programs is mostly done for all versions of UNIX, by
many different factors. X/XFree86 is developed for all UNIXes, as well as the
GNU software, Perl/Python/PHP/Ruby/Tcl, KDE/GNOME/other desktops and window
mangers, etc. Furthermore, most of these systems restrict themselves to a
certain common UNIX API, which is well defined and stable, and so do not need
to concern themselves with changes in the underlying operating systems.

These development groups work independently of one another, with a lot
of information hiding, and well-defined APIs, and a gradual advancement
towards a common goal. Some of them even port their software to Windows.

On Windows, on the other hand, the development of the core system, the most
common APIs and SDKs, and the most common applications is conducted by one
central software house: Microsoft. Furthermore, this software house can only
use its own developers for development of its products, without aid or
cooperation from the outside (by virtue of the guiding philosophy).

In the open source world, if something lacks a needed feature, you can only
blame yourself for not fixing it. In Windows, on the other hand, you have
every right to make serious ranting towards "Microsoft" for not fixing a bug,
or neglecting to implement a feature. After all, it does not let you fix it
yourself!

-------------

That's everything I can think of for now. Can you think of more differences?

Regards,

    Shlomi Fish

----------------------------------------------------------------------
Shlomi Fish        [email-masked]
Home Page:         http://t2.technion.ac.il/~shlomif/

An apple a day will keep a doctor away. Two apples a day will keep two
doctors away.

	Falk Fish

---

From: "Nadav Har'El" <[email-masked]>
Date: Sun, 21 Dec 2003 23:30:28 +0200

On Sun, Dec 21, 2003, Shlomi Fish wrote about "[hackers-il] The Great Divide":
>..

You call this a "question"? It's more of a rhetoric question - a question
whose answer is obvious. Compatibility with other vendor's products is the
last thing Microsoft wants. Compatibility with Linux - which apparently is
the only competitor which seems to be gaining any ground on Microsoft - is
counterproductive for them.

A better question is what you'll be able to do with this shell. The original
Unix shells were so successful because you could operate all the applications
in the system from them (this was 100% true until a few years ago, and became
less true but still mostly true after the advent of stuff like KDE and Gnome).

In languages like Perl operating other applications becomes less convenient
but still possible, and on the other hand you get the full programming power
of general-purpose programming languages, and the possibility of extensive
library collections. So Perl et al. are not shells in the original sense of
the word, but more of high-level languages.

But what will make this Microsoft Shell successful? Will you be able to
fully control all applications in the system from it, meaning that applications
will be *built* to be fully controlled and automated from this shell (like
Unix applications were designed to be run from Unix shells)? Will this happen
for all applications, or just Microsoft applications, further increasing
their monopoly?
Or will Microsoft Shell remain a dud that nobody uses because you can't
really do anything useful from it, like command.com, or whatever it is
called in recent Windows systems?

> philosophies.

I actually find this good, not bad. A recent article by Bruce Schneier and
friends suggests that this diversity is important for national security
(the resilience of computerized systems), just like monoculture is bad in
agriculture.

> Will Windows ever become another UNIX-compatible operating system?

Of course not, because if Windows and Unix/Linux/Freebsd became compatible,
why would anyone pay 2000 shekels for a copy of Windows? Most advanced users
I know cite incompatibility, not "incovenience", as the reason of not using
Linux.

> Will UNIX see a rise of high-quality and complete user-centric
> programs that already exist in Windows?

Yes. This is evolution in progress, and nothing can stop evolution.

> Will one of the worlds (probably

Yes. Mark my words - Microsoft Windows will become obsolete in 10 years.

15 years ago I said the same thing about MS-DOS and Turbo Pascal. I hope
I'll be right again :)

In fact, the whole concept of a large form-factor "personal computer" as
we know it is likely to be obsolete in 10 years.

> build a GUI which drives that core. This way the main operations of the

Return to my shells argument. The very fact that all applications can be
fully controlled from the command line is what makes Unix shells so
amazingly versatile.

> 2. In UNIX: Open Source Prevails; in Windows: not necessarily

This is true for historical reasons, and is likely to continue in the
next decade (I believe). It is exactly this open source nature that drives
the evolution of Gnu (and later, Linux), and evolution is a pretty mighty
force, perhaps even more than Microsoft.

-- 
Nadav Har'El                        |      Sunday, Dec 21 2003, 27 Kislev 5764
[email-masked]             |-----------------------------------------
Phone: +972-53-790466, ICQ 13349191 |A diplomat thinks twice before saying
http://nadav.harel.org.il           |nothing.

---

From: Shlomi Fish <[email-masked]>
Date: Mon, 22 Dec 2003 12:50:53 +0200 (IST)

> counterproductive for them.

Note that it is possible to install a UNIX compatibility layer (Interix)
as well as Perl and Python on Windows. They are just not installed by
default. And they work pretty well, and if you know what you're doing you
can have pretty good compatibility between the two systems.

The question is why would anyone want to use this shell when you know the
scripts you write for it will only run on Windows and even then only on
Windows starting from Longhorn. Unless simply as a convenient wrapper to
automate .NET processes, rather than a system administration tool.

> for all applications, or just Microsoft applications, further increasing

As far as I understood, MSH will operate on .NET objects. What
interactions it will have with real processes, files, etc. remain to be
seen, probably whatever .NET gives it.

> called in recent Windows systems?

It's called CMD.EXE. CMD.EXE is a superset of command.com which is much
more usable but still shivers in comparison to UNIX shells. Many of the
more clueful NT sys-admins make use of it extensively. Ask Chen - her
Mercury team uses it.

If you remember, I wrote a CMD.EXE Batch script to automate building
Freecell Solver on Windows.

> (the resilience of computerized systems), just like monoculture is bad in

If you say so.

> I know cite incompatibility, not "incovenience", as the reason of not using

Do they refer to incompatibility at the mundane user level (i.e:
inexistence of Word, Excel, etc.)? Or do they refer to API
incompatibility?

> 15 years ago I said the same thing about MS-DOS and Turbo Pascal. I hope

Your predictions remind me of a "Hitchhiker's Guide to the Galaxy" quote:

<<<
There is a theory which states that if ever anyone discovers what the
universe is for and why it is here, it will instantly disappear and be
replaced by something even more bizarre and inexplicable. There is another
which states that this has already happened.

So in a way MS Windows replaced DOS, and it is possible it will be
replaced by something else, _that is not Linux_.

> force, perhaps even more than Microsoft.

I sure hope so as well.

Regards,

	Shlomi Fish

	Falk Fish

---

From: "Nadav Har'El" <[email-masked]>
Date: Mon, 22 Dec 2003 13:09:19 +0200

> > (the resilience of computerized systems), just like monoculture is bad in

It's not I who said so :)
In case you still haven't read it, Bruce Schneier et al.'s "Cyber Insecurity - 
The Cost of Monopoly" can be found in
	http://www.ccianet.org/papers/cyberinsecurity.pdf

It was written about 3 months ago, so it is still very relevant.

> inexistence of Word, Excel, etc.)? Or do they refer to API

I'm talking about ordinary (though advanced) users, not programmers.
Hardware drivers that come together with hardware are incompatible with
Linux, Word/Excel/etc. files cannot be read or written "perfectly" on Linux.
hyped games don't run on Linux, executable files you get via email don't
run on Linux (this can be a good thing :)), hyped software often doesn't
run on Linux. Linux doesn't look the same as Windows. Windows experts
(e.g., the kid nextdoor) don't know how to help you on your Windows
machine. Linux is also incompatible with our native language (though this
is improving quickly).

> replaced by something else, _that is not Linux_.

Of course. Though using the familiar philosophy question "if you take a
sweater and replace threads, one by one, when does it become a different
sweater?", if Linux (or other Unix variants) continue to evolve gradually,
you'll still have Linux in 2010, but it will be a very different Linux.
I think someone said (I don't remember the exact quote) "I don't know what
I'll be running in 2010, but it will be called Linux." :)

I've been using Unix since 1985. Unix has changed a great deal since, but
I still consider myself to be a Unix user (even if lately we started to
use the word Linux instead of Unix).

-- 
Nadav Har'El                        |      Monday, Dec 22 2003, 27 Kislev 5764
[email-masked]             |-----------------------------------------
Phone: +972-53-790466, ICQ 13349191 |Politics, n: from Greek, "poly"=many,
http://nadav.harel.org.il           |"ticks"=blood sucking parasites.

---

From: Shlomi Fish <[email-masked]>
Date: Mon, 22 Dec 2003 19:29:32 +0200 (IST)

> It was written about 3 months ago, so it is still very relevant.

Yes, I've heard about it. But I have a problem with it: I'm not sure a
situation where there are 5 or more dominant systems with nearly equal
amounts of market share is even remotely possible. Eventually, one system
become much better than the rest, and so most hackers choose to work on it
and with it. I don't think it is possible to enforce a situation in which
all hackers are divided equally among 5 different operating systems with
radically difference codebases.

I think a better idea would be to:

1. Employ Secure Coding Practices for the development of the chosen
system. (using secure routines or more secure languages and APIs, code
auditing, etc.)

2. Educate Sys Admins on how to keep their systems up to date.

3. Employ a fast and automatic way to deploy software updates.

And other measures that can decrease security problems even in a
monoculture. A situation where 20% of the systems can become compromrised
is not much better than a situation in which 90% of them can.

The problem with the current Windows mono-culture is that Windows was
coded very irresponsibly as far as network security is concerned.
Microsoft was practically a home/office user software vendor that decided
to try and conquer the server/networked operating systems market. As such,
it did not (and perhaps still does not to a large extent) have the
suitable practices for releasing a software for becoming a part of
the Wild Internet.

> machine. Linux is also incompatible with our native language (though this

True. The problem is, of course, that many development shops don't care
enough about the Linux market (due to its size) to port their software to
it. This is sort of a chicken and egg problem.

> I'll be running in 2010, but it will be called Linux." :)

It was said of Fortran, originally, IIRC.

> use the word Linux instead of Unix).

Well, when I started using UNIX in 1995/6, I first used other UNIXes:
FreeBSD, IRIX, and SunOS 4.1.3. I later became familiar with Linux when I
installed it on my home computer and started experiencing with it. (after
I saw how great UNIX was). Nowadays, we can expect that more and more
people will be introduced to UNIX through Linux.

I still use the term "UNIX" when I refer to something that is not specific
to Linux but to UNIX in general. It may be confusing to beginners to
understand the boundary between what is common to UNIX and what is Linux
or GNU-specific. Gladly, Linux and other UNIX systems are more compatible
than not.

You are right that in the future we may see a completely different idea of
what Linux or UNIX is. But it is possible that we may never see a
revolution in which people decide to scratch UNIX and start from an
entirely different concept (and others actually dump UNIX in favour of
it).

Regards,

	Shlomi Fish

	Falk Fish