What Makes Software High-Quality?

This work is licensed under the Creative Commons Attribution 2.5 License (or at your option a greater version of it).

Revision History
Revision 167502 April 2007shlomif
Forked the template from a previous work and working on it.
Revision 187104 May 2008shlomif
Finalised the first draft. About to receive feedback.
Revision 187204 May 2008shlomif
Fixed many spelling/grammar/etc. problems, trailing whitespace, etc.
Revision 188206 May 2008shlomif
Fixed many spelling, grammar and syntax problems (thanks in part to Omer Zak). Added some bolds, and id=“”s, added the about section, reorganised the text a bit, and added the note about responsiveness and startup time.
Revision 188406 May 2008shlomif
Changed “a software” to something more idiomatic, and fixed a bad phrasing towards the beginning.
Revision 521815 May 2008shlomif
Forked the document from the first revision, and converted to DocBook-4.5.
Revision 532924 May 2008shlomif
Added the suggestion of speaking about “Why software quality is important”, and the note about popularity to the introduction. Placed the Intro FCS story in the “Motivation” section. Added the “organisation” sub-section to the Intro. Added the note about the generic “weight function”. Added a link to “The Stanford Checker”. Corrected many “a software” instances. Added the note about how Linux and Windows looked to the Aesthetics. Added the note about low-quality code and organisational quality to the “good code” section. Added the footnote about a comprehensive comparison of Freecell solvers. Added the note about solution length and “other advantages” of fc-solve. Updated the “Thanks” section. Made many other corrections.
Revision 549209 June 2008shlomif
Fixed “manifest” to “is manifested”.
Revision 485905 June 2011shlomif
Convert to Unicode single-quotes and double-quotes.

Abstract

This document will discuss what makes an open source program (and by induction other programs) high-quality. It will cover the parameters that make software applications high-quality and ways to achieve them.


Table of Contents

Introduction
Motivation
Organisation of this Document
Parameters of Quality
The Program is Available for Downloading or Buying
The Version Number is Clearly Indicated
The Source is Available, Preferably under a Usable Licence
It “Just Works”
The Program has a Homepage
The Program is Easy to Compile, Deploy and Install
The Program Has Packages for Most Common Distributions
The Program Has Good, Extensive and Usable Documentation
Portability
Security
Backwards Compatibility
Good Ways to Provide Support
Speed and High Performance
Aesthetics
Conclusion
Ways to Achieve Quality
Having Modular, Well-Written Code
Automated Tests
Beta Testers
Frequency of Releases
Good Software Management
Good Social Engineering Skills
No Bad Politics
Good Communication Skills
Hype
A Good Name
Conclusion
Analysis of the Quality of the Various Freecell Solvers
Introduction
The Analysis Itself
Conclusion
About This Document
Copyrights
About the Author
Acknowledgements

Introduction

Why is high quality in software important? Low-quality software applications will require the users or end-developers to work around their bugs and limitations, write a lot of extra functionality themselves, and as a result, duplicate a lot of effort and cause a lot of frustration and unhappiness. This is assuming they don’t give up on it soon or right away, and end up looking for something else.

But what makes a popular software application “high-quality”? A high-quality program or library can be defined as one that is usable as is, that causes very few frustrations, and that requires little if any modifications or workarounds to get it up-to-speed.

I’m not talking about the application that’s the most hyped, or the fastest, most featureful, or best. Sometimes you can often hear endless rants and Fear, Uncertainty and Doubt (FUD) attacks about it. Often, it has competing programs that are superior in many respects.

In this essay we’ll look at what quality metrics apply to software.

Motivation

How did it occur to me to ponder this question? I am active in the fc-solve-discuss mailing list, which is dedicated to computerised techniques for solving the Freecell solitaire card game, and for discussing other automated solving and research of Solitaire games. This mailing list was started after I wrote Freecell Solver (with a lowercase “c” and an uppercase “S”), an open-source solver, which has proven to be quite popular. However, the mailing list was not restricted to discussing it exclusively, and was joined by many other Freecell solving experts, researchers and enthusiasts.

Recently, Gary Campbell wrote this message reading:

I think the solver discussion in the Wikipedia should mention that the FCPro solvers give quite lengthy and virtually unusable solutions. No human wants to follow the several hundred steps that usually result in order to get at what’s really required to solve a given layout. If someone wants a solution that can be understood, they want one that is under 100 steps. This is a major contribution and it should be stated. There are a lot of “toy” solvers and only a very few of “industrial strength.” One of the latter is the solver by Danny Jones. I don’t see that on your list. The results he has gotten from his solver pretty much put your solver to shame. The more you hype Freecell Solver, the more criticism you open yourself up to.

Unfortunately, it’s hard for me to determine what “industrial strength”, “enterprise grade” and other such buzzwords are. But I’ll try to define high quality here, and try to show when a program is high quality.

Organisation of this Document

I initially wanted to give some examples for open-source software that I considered to be of exceptional high-quality, but I decided against it.

That’s because their exceptional quality is a matter of taste, and it may provoke too much criticism against this essay as a whole. Thus, I’ll just give some examples, possibly accompanied by screenshots, of places where one program does something better than a different program. I won’t even going to imply that the latter is in fact lower-quality in all possible respects.

This article is written from my point-of-view as a developer of “Free and Open Source Software”, and will focus on open-source and pseudo-open-source software. However, a lot of what I say is more universal.