Subject: The Growing Complexity of UI Design

From: Shlomi Fish <[email-masked]>
Date: Mon, 12 Jul 2004 15:03:45 +0300

I am writing this article to express some thoughts that came to me after 
reading the excellent book "The Design of Everyday Things" by Donald A.
Norman. You can find my review of it here:

http://www.perl.org.il/kwiki/index.cgi?BookReviewTheDesignOfEverydayThingsByShlomif

What I've been thinking is that as time passes, the demands from a usable
user-interface (GUI, command-line or otherwise) become greater and greater. 
Constantly, new user-interface features and improvements are made that
set the standard higher and higher.

Examples for such improvements are:

1. Tabbed browsing. Did not exist in browsers until recently, but now 
available in Opera, Mozilla and Konqueror. Many people who get used to it
find living without it, very hard.

2. Right-mouse-button. Initially, it did not do anything specific, and
was left to the whims of the GUI designer. For several years now, it has
brought a context-sensitive menu. So, people who get used to pressing the
right mouse button, will be annoyed with older GUIs that don't. (and in
the traditional Macintosh one-button mouse, it's a certain key on the
keyboard + click)

3. The scroll-wheel. Easy to get used to, and later you find mice that
don't have it (and applications that don't support it) frustrating.

4. Prefix command-recall. We all know the history feature of bash. (up and
down buttons). When I used Matlab there was a similar feature there, only
you could position the cursor at the middle of the line and recall the 
previous
command starting with this prefix.

I looked for such a feature in bash and was told of the GNU readline key
operations that does that. (after some rant on why I don't read the man page)
Editing the inputrc, I was able to add them as Ctrl+P and Ctrl+N.

A few weeks ago, someone asked for the same feature in the Haifux mailing 
list,
and this time I instructed him how to do just that. He said this was the 
reason
he was still using tcsh as his login shell despite the fact that he otherwise
found bash more superior.

5. HTML Navigation Menu. 

----

And there are many other examples. And it seems that more and more 
user-interface paradigms like that are added. One side-effect is that the
application implementing all of them are becoming more complex. If someone
had to re-implement everything from scratch it will take him quite a lot
of time to get to a program that most users will find usable.

Now, it occurred to me several times that I tried to do something with a GUI
and found out it was not available. One example I can think of is the 
ML-Donkey Gtk+ GUI tabs for the searches' results. There is one tab for each
search result. However, pressing the right button on a tab, just brings in a
drop-down menu of the tabs. It does not have a "Close Tab" item, which is what
I came to expect of it.

It also happened to me several times with KDE apps which tend to be more 
polished. (can't readily recall anything). Another example is the folder
search tool-bar Mandrake added to Kmail in their KDE 3.2.3 RPMs. It is 
sometimes convenient, but there doesn't seem to be anyway to remove it.

It is possible that most of this complexity can be conquered by coding 
re-usable components which abstract the needed functionality in a generic
way. Say, a tab manager that already implements "Close Tab", "Detach Tab",
and other commands user expect. This may at least relieve the application
designer from handling most of the complexity himself. I am, however, 
skeptical that all the necessary functionality can be kept out of the 
application this way.

Another thing that is possible is that eventually usability improvements
will no longer be added to computers, because all the possible ones
have already been added. I'm not sure that would ever be the case - it's just 
speculation.

Nevertheless, I believe that the growing complexity of UI designs is a 
problem that application programmers will increasingly and increasingly
face in the near future.

Regards,

	Shlomi Fish

-- 

---------------------------------------------------------------------
Shlomi Fish      [email-masked]
Homepage:        http://shlomif.il.eu.org/

Knuth is not God! It took him two days to build the Roman Empire.

---

From: "Nadav Har'El" <[email-masked]>
Date: Mon, 12 Jul 2004 16:44:19 +0300

> Norman. You can find my review of it here:

Interesting. Putting it on my wishlist, right after "Hawaii's Spam Cookbook"
(really!).

> find living without it, very hard.

This is indeed an improvement, because it does a feature that was always
available (namely, opening several browsing windows), better.

> the traditional Macintosh one-button mouse, it's a certain key on the

While I like this right-mouse-button idea, I am not sure how much of an
"improvement" this is. It is certainly not a natural thing for new users (I've
seen this on more than one occasion). It also adds a lot of "invisibility"
to the GUI: you can't see how to do this action, but if you right-click
on exactly the right spot, you'll suddenly have that menu appear.

A few days ago I was creating a slideshow in OpenOffice and wanted to
duplicate a slide. But I couldn't find that action. I (thought that I)
tried every menu, hovered over every tool trying to guess what its shape
meant (or waiting for the tooltip), and finally I hovered every graphical
element on the window that I thought it may be relevant, and right-clicked
on it to see maybe the "Duplicate Slide" option will appear.
So you see how sometimes the added complexity may not be such an improvement,
especially for experienced users who know what they're looking for, but
forget where to find it.

> don't have it (and applications that don't support it) frustrating.

I actually prefered TCL/TKs age-old technique of letting you pan the window
with the middle mouse button, without needing extra hardware (a scroll
wheel). The scroll wheel is useful for reading text on the web, but I
can't find much more use for it; I prefer having my hands on the keyboard
(where I do more than just scrolling), rather than on the mouse.

> command starting with this prefix.

You can add to this list command-line completion, starting with simple
filename completion, and ending with ZSH's super-duper everything-can-
be-completed. The TAB key has become one of the most worn-out on my keyboard
(it's especially worn-out when compared to its squeaky-clean neighbor, the
CAPS LOCK).

> 5. HTML Navigation Menu. 

What is that? Do you mean those meta tags that allow you to navigate
within a large document, and that hardly anyone ever uses?

> application implementing all of them are becoming more complex. If someone

Actually, the idea with windowing toolkits is that all this is done
automatically for you. For example, when you create a menu entry, the toolkit
might ask you to specify a keyboard shortcut, and it automatically handles
this keyboard shortcut for you. When you create a button, it may ask you
for a "tooltip" text. When you create a scrollbar, it might bind the
scrollwheel automatically.

When new paradigms come along, sayu "the right button", toolkits should
adopt and make it easy to provide these new features by doing minimal
programming.

> skeptical that all the necessary functionality can be kept out of the 

Tcl/Tk (or actually, IncrTcl) had the concept of "mega-widgets", objects
which are written in a high-level language (Tcl) and compose a number of
low-level widgets. You can easily imagine a "tab" mega-widget, which
(like other geometry-managing widgets) will have children windows that
contain the individual tabbed windows.

> problem that application programmers will increasingly and increasingly

I don't think this is a new problem. I first noticed it about 7 years
ago when I wrote a Motif application using an old version of Motif that
did not support the then-new "fads" of toolbars and tooltips. But my
users (and I) expected these features, so I had to write them myself in
low-level Motif, which was sort of ugly.

In any case, I believe there's a much bigger problem of growing complexity
of computer software, of which the GUI is just one small issue. When I
started learning computers, I could read a 300-page book and say "I know
Unix, C, and Yacc!", and feel like you understand everything that is going
on on your computer. Today, you would have to read a 1000 page book and
say "I know C++, and a little bit about Linux. But I really have no idea
on what's going on on my computer behind my back. Heck, there might be 7
worms and trojans running, and I wouldn't even know". The complexity of
computer software is slowly, slowly (but surely) approaching that of the
human body, and when it does, no one will be able to understand it or
control it fully - just like nobody fully understand or control the human
body.

-- 
Nadav Har'El                        |      Monday, Jul 12 2004, 23 Tammuz 5764
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |Hardware, n.: The parts of a computer
http://nadav.harel.org.il           |system that can be kicked.

---

From: Shlomi Fish <[email-masked]>
Date: Mon, 12 Jul 2004 20:18:09 +0300

On Monday 12 July 2004 16:44, Nadav Har'El wrote:
> Interesting. Putting it on my wishlist, right after "Hawaii's Spam
> Cookbook" (really!).
>

Hmmm... never heard of Hawaii's Spam Cookbook. I'll try looking for it in 
Amazon when I have a few spare cycles.

> "improvement" this is. It is certainly not a natural thing for new users
> (I've seen this on more than one occasion). It also adds a lot of
> "invisibility" to the GUI: you can't see how to do this action, but if you
> right-click on exactly the right spot, you'll suddenly have that menu
> appear.
>

That's true. Still, once you tell someone: "on most applications pressing the 
right mouse button on most items will bring in a context menu with some 
useful stuff.", they'll get the hang of it and then like it. If you put 
someone in front of a spreadsheet program or the UNIX command prompt without 
any instructions on how to begin, they are also unlikely to figure out how to 
start by themselves.

To quote one of my favourite fortunes:

<<<
The only "intuitive" interface is the nipple. After that, it's all learned.
(Bruce Ediger, [email-masked], in comp.os.linux.misc, on X interfaces.)

> on it to see maybe the "Duplicate Slide" option will appear.

Hmmm... I can recall that similar things have happened to me as well a couple 
of times with several GUI programs. Usually, the thing to do at this time is 
to consult the help or people on the IRC or in mailing lists. However, this 
is already an indication that the GUI is not designed well.

> So you see how sometimes the added complexity may not be such an
> improvement, especially for experienced users who know what they're looking
> for, but forget where to find it.
>

Hmmm....

> wheel). 

Well, actually, you need the middle mouse button which many mice don't have as 
well.

> The scroll wheel is useful for reading text on the web, but I 

Right. I've seen my sister edit files in kedit and she actually selects text 
with the mouse, press the Edit->Copy or Edit->Cut menu items and then moves 
the cursor and presses Edit->Paste. Using Ctrl+X/Ctrl+C/Ctrl+V (or whatever 
your editor binds to that) is so much faster, IMO.

> > 4. Prefix command-recall. We all know the history feature of bash. (up
> > and down buttons). When I used Matlab there was a similar feature there,
> > only you could position the cursor at the middle of the line and recall
> > the previous
> be-completed. The TAB key has become one of the most worn-out on my
> keyboard (it's especially worn-out when compared to its squeaky-clean
> neighbor, the CAPS LOCK).
>

:-)

> within a large document, and that hardly anyone ever uses?

Not really. I mean the side-bars or top-bars you see as part of the page that 
help you navigate a page. Like on:

http://shlomif.il.eu.org/

Or:

http://freshmeat.net/ (as an example for a top one).

> automatically for you. For example, when you create a menu entry, the
> toolkit might ask you to specify a keyboard shortcut, and it automatically
> handles this keyboard shortcut for you. When you create a button, it may
> ask you for a "tooltip" text. When you create a scrollbar, it might bind
> the scrollwheel automatically.
>

Right. I said so myself in my original post. However, it's still an added 
overhead on the part of the programmer.

> > way. Say, a tab manager that already implements "Close Tab", "Detach
> > Tab", and other commands user expect. This may at least relieve the
> > application designer from handling most of the complexity himself. I am,
> > however, skeptical that all the necessary functionality can be kept out
> > of the application this way.
>

Well, most modern toolkits support such "mega-widgets". Bot Gtk+ and Qt allow 
you to define new widget classes which can be as complex as you like. A 
tab-manager mega-widget would have to be a container that allows managing 
several windows. This is not enough, because you also need to manage the 
documents behind them (the model of the application).

> I don't think this is a new problem. 

I never said it is.

> I first noticed it about 7 years 

Ouch!

> control it fully - just like nobody fully understand or control the human

Well, as far as computer architectures themselves they become more complex, 
but not by far. Understanding the machine and Assembly level itself is fairly 
easy. And the x86 family of architectures have some complexities that are 
otherwise eliminated in more modern architectures that were more 
well-designed.

Now, in regards to the software that runs above it. Well, this reminds me of 
what Joel Spolsky says in his "The Law of Leaky Abstractions" article:

http://www.joelonsoftware.com/articles/LeakyAbstractions.html

One particular paragraph that is relevant here:

<<<
Today, to work on CityDesk, I need to know Visual Basic, COM, ATL, C++, 
InnoSetup, Internet Explorer internals, regular expressions, DOM, HTML, CSS, 
and XML. All high level tools compared to the old K&R stuff, but I still have 
to know the K&R stuff or I'm toast.

I think software is becoming more complex because there are so many players 
around. Microsoft wants it this way. The W3C wants it the other. The KDE guys 
have their own direction, which differs from GNOME's and from Mozilla's. And 
naturally, there's also Apple with their Mac OS X technologies. So much 
choice and every end-user application builder can make a lot of different 
choices.

I think understanding UNIX basics is probably still doable, and most of what 
exists there still holds in the grand scheme of things. There are just much 
more layers above it, but they are userland abstractions, written in C or 
something similar that run as ordinary processes. If you wish to learn C++ go 
ahead, but it won't be more helpful than learning C in trying to understand a 
Linux system, and will take much more time and frustration than learning C 
and then learning OOP using a high-level language. (where OOP just works)

As for your comment that:

> The complexity of

Well, I disagree on several premises:

1. The reason nobody fully understands or controls the human body is not 
because it is complex. It is because there is still a lot that is unknown 
about it, and that it is actually a physical and bio-chemical medium that is 
otherwise hard to control. There are things that are of considerable 
complexity (like theoretical mathematics) that can be relatively easily 
conquered.

2. For that reason, a software of the same complexity of the human body, may 
not necessarily be equally as hard to manipulate or understand. Software is 
just 0's and 1's. We can always easily attach gdb to a running process, or 
read the software's source code, etc. - things that are not as easy to do in 
the human body. (due to physical and technological constraints). 

Furthermore, a software running on a system is split into layers and 
components. When I'm hacking on the GIMP for example, I do not need to 
concern myself with the kernel or the libc, or the window manager, or 
X-Windows, or whatever. These things just work!

Recently, for example, I discovered a bug in Gtk+ that made one of the 
plug-ins I hacked on misbehave. At first I did not understand why it 
happened, but after consulting with a few experts, I was able to find out it 
was indeed a bug, and that it was registered in bugzilla, with a proposed 
fix. I wrote and revised a patch to fix it, applied it on my version of gtk+ 
and hopefully it will be integrated into the next patch release of gtk+.

So, ok, the gtk+ layer does affect the GIMP, somewhat. I recall that before Qt 
became open-source the KDE developers also said they could not re-implement 
it themselves as open-source, because they were already very familiar with 
the code of the proprietary vesion. But nonetheless, I don't need to 
understand everything that is running on my system to hack on one particular 
thing.

You are right that software development in some contexts is becoming more 
complex, with more and more software technologies and sub-technologies that 
have to be used together to create something meaningful. But that complexity 
is still something that can be conquered.

Regards,

	Shlomi Fish
--

---

From: "Nadav Har'El" <[email-masked]>
Date: Tue, 13 Jul 2004 10:44:04 +0300

On Mon, Jul 12, 2004, Shlomi Fish wrote about "Re: [hackers-il] The Growing Complexity of UI Design":
> easy.

But I as I explained, this is becoming more and more irrelevant. When a
new machine runs as many as billion instructions per second, you cannot
seriously hope to single-step through them, or anything close to that, in
hopes of understanding what your machine is doing.

This is like saying "I can fully understand the human body and cure any
disease, because I understand the bases of the DNA, and the 21 amino
acids coded by them". This is obviously a falasy. While indeed everything
in the body was ultimately created by these DNA bases creating amino acids
creating proteins and enzymes creating ..., the end result is so complex that
we are a long way from figuring out its every details.

> what Joel Spolsky says in his "The Law of Leaky Abstractions" article:

So I guess you agree with me, then.

> around.

I don't agree. Even if Microsoft was the only player around (heck, isn't
this what's going on in the Windows platform?), software would get more
and more complex.

I see this as a consequence of the "evolution", or "survival of the fittest"
process through which software is created. It's exactly what happened in
the world of leaving things - though simple organizims didn't die out
(e.g., prokaryotes), more and more complex life- forms appeared. The complex
life forms weren't "needed" - the simple life forms were doing (and are
still doing) just fine - but once they appeared, and survived, you couldn't
really stop them from keeping on surviving and being the most visible (but
not numerous!) type of life-form on the planet. 

> complexity (like theoretical mathematics) that can be relatively easily 

I don't agree with you. The reason why the human body and Mathematics have
both been studied for centuries without us yet fully understanding them
*is* these topic's complexity. We understand the inner structure of the
atom (which is much harder to see inside than a human body) better than we
do the inner structure of the human body, because the latter is much more
complex.

> the human body. (due to physical and technological constraints). 

Do you remember by fork-bomb example, which I compared to cancer? A process
would split indefinitely, choking all other processes on the machine,
depriving them of resources. This is similar to a cancerous cell splitting
indefinitely and killing the normal cells around it. In the human body,
you indeed can't give a command "kill all cancerous cells". But as I
explained in that posting, it may not trivial in a computer setting as well -
the "cancerous" process might be a shell process, so you can't give a
command to "kill all shell processes" because this will kill important
processes as well. You are left with two options, both requiring you to
fully understand the complexities of the software involved:

  1. Write a program which recognizes the behavior of "cancerous" processes
     and killing them. (sort of like an anti-virus)

  2. Use a trick like the one I explained in the fork-bomb posting: killing
     the "fork-bomb cancer" by filling the computer with benign processes
     that, after suffocating the fork-bomb, will kill themselves politely.

I can see how computer software gets more and more complex, and doing
these things become harder and harder, and eradicating computer "diseases"
will not be a trivial thing. It won't be as hard as eradicating human
diseases, but it won't be something that and joe-sixpack-sysadmin could
solve on his own.

This is already getting evident with viruses, worms, spyware, and other
crap which is harder and harder to get away from your computer. Most
sysadmins I know already cannot clean a computer after a breaking
(with rootkit installed, etc.). Some people think that the "Trusted
Computing Initiative" will solve this problem. I doubt it.

> X-Windows, or whatever. These things just work!

This is exactly how the human body works. There are very low-level processes
like those going on in the mytochondria and ribosomes, higher level processes
like the blood circulation, and higher level process like individual organs,
hormone glands for "configuration", the brain as a "shell", the eyes and
ears as "input devices", and so on. This seperation makes the body easier
to understand, but doesn't make it fully understood.

-- 
Nadav Har'El                        |     Tuesday, Jul 13 2004, 24 Tammuz 5764
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |An error? Impossible! My modem is error
http://nadav.harel.org.il           |correcting.

---

From: "Nadav Har'El" <[email-masked]>
Date: Tue, 13 Jul 2004 13:51:31 +0300

> Do you remember by fork-bomb example, which I compared to cancer? A process

All my examples so far were of security violations, so this might lead the
reader to believe that complexity is only an issue when there is a malicious
adversary, or the computer equivalent of an "infection" or "cancer", involved.

But this is not so. I'll give you one example of many I saw in recent years.
A system administrator I know had the problem that every time he tried to
send out mail from a Unix system, the sending process hung for a minute or
so. That was a new problem - before it started, sending never hung for more
than a second.

This system administrator could not find the cause of the problem.
Sure, he had excellent diagnostic tools (ps, ls, cat, etc.) which are far
better than those available now for the human body. But just having these
tools was not enough. He didn't even know where to start. There could be
many concievable reasons that causes the mail sending to take a long time.
Lacking any clue on where to start his investigation, the system administrator
in question made a reasonable assumption - that something must have changed
in the system recently that caused this new problem. He knew that the only
thing that had changed was the installation of a few OS patches the day
before, and so blamed the new problem on these new patches. And since these
were the latest patches, there was nothing to do but wait until the next
round of patches, which fix the problems caused by the previous patches.

Or so he thought. Incorrectly. It turns out that the problem was completely
different. When he told me about this problem, I solved it for him in 5
minutes. I'll cut a long story short: the reason for the hangs was RBL
(anti-spam blacklists) lookups which were done also for outgoing mail.
The system administrator didn't even know that they were using RBL, let
alone for outgoing email (!), and couldn't know that the RBL's name servers
had stopped working recently. Only someone like me with a broad insight of
today's Unix, networking and Mail issues, could figure this problem out in
five minutes without spending a week on debugging, tracing and log-reading.

My insight in computers is similar to that of an experienced doctor on the
human body. And just like becoming a doctor is hard, it is becoming harder and
harder to be a "doctor" for computer problems. As computer software continues
to get more complicated, two things will happen: first, what I described as
"a week of debugging, tracing and log-reading" can turn into a month, and
then a year - until these foolproof techniques are no longer feasable and only 
"approximate" techniques that can't solve *every* problem could be used.
Second, the situation where one person can be an "expert doctor" on "everything
on the Unix System" will not be able to continue. You'll have "general
practitioners" that know about common, generic, problems that afflict Unix
systems, and "experts" that each focuses on one thing - mail servers, http
servers, perl programs, etc.  This is very much what happened in medicine.
Instead of generic solve-every-medical-problem quacks of centuries ago, we
now have general practitioners (rofe mishpacha) and specialized expert
doctors in various parts of the body, various types of disease and various
types of treatments and diagnostic techniques.

I think now it might be clearer why I am claiming that the complexity of
software is beginning to resemble that of the human body. But I emphasized
that it's just *beginning*. The human body is still many orders of magnitude
more complex than current software. But in a few decades, who knows? If
you remember a book review I wrote some months ago, Ray Kurzweil estimates
that this could happen as soon as the year 2020...

-- 
Nadav Har'El                        |     Tuesday, Jul 13 2004, 24 Tammuz 5764
[email-masked]             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |I don't live on the edge, but sometimes I
http://nadav.harel.org.il           |go there to visit.