Sample Challenge Document
(Do not send this version. Use the downloadable plain text version instead). Read the introduction here.
Dear [insert editor or publication name here],
I'm writing to suggest that [insert publication name] produce an
investigative feature which I believe could prove very popular among
your tech-savvy readership.
As you know, the technology press has been buzzing about alternative
operating systems over the past year. While the vast majority of this
press coverage has focused on the challenge the open source movement
presents to the industry, very little of it has concentrated on the
actual usability of alternative operating systems or the software
built for them. When considering questions such as "Do alternative
operating systems stand a chance against Windows on the desktop?,"
this consideration is key.
The reason this question is rarely addressed is, I believe,
because a relatively small proportion of journalists writing about
alternative operating systems are actually using or testing them
personally. This is probably a function of tight deadlines and
limited testing resources, but the result is a tendency to repeat
catch phrases such as "Linux is making inroads in server space,
but faces a tougher challenge on the desktop," or "BeOS appears to
be a technically excellent alternative, but faces limited hardware
and application support." Furthermore, it appears to me that BeOS is
often glossed over or given no coverage at all, for whatever reason.
Here is my suggestion for a [insert name of publication or site]
feature story:
o Set aside a Windows machine or two with a couple of gigabytes
of free disk space. Be sure to investigate publically available
hardware compatibility lists for both operating systems before
beginning. One
of many Linux hardware compatibility lists can be found at
http://www.redhat.com/corp/support/hardware/ .
o Select one or two people from your organization who have experience
working with MacOS or Windows, but no experience working with
alternative operating systems. You may want to perform the same tests
with users at different levels of technical experience: Beginning,
intermediate, advanced, etc.
o Provide them with Linux and BeOS installation CDs, of the most recent
available versions. Any distribution of Linux is fair game. Provide
them with any amount of documentation, including books, manuals,
guides, FAQs, web sites, etc. Do *not* provide them a "guru" or
human assistant.
o Have the users install BeOS and Linux partitions on the computer
provided. Observe the users (or interview them later), noting any
part of the install process that proved difficult. Interview them
about the experience.
o Have the users configure the systems. Have them set their video
cards to work at a preferred resolution and bit depth. Have them
configure their network cards to attach to a TCP/IP network, such as
the company LAN. Have them configure dial-up networking services to
connect to an ISP. Have them perform any other basic configuration
tasks you can think of. Observe the users, noting any part of the
configuration process that proved either easy or difficult. Interview
them about the experience.
Note: Because Linux comes in so many different flavors and
distributions, the results of the tests will be heavily colored by
the particular Linux distribution chosen. Be sure and note which
distribution was used in the test.
o Once configured, have the users begin to actually use the
system. Have them find a simple text editor in which to work with
plain text documents. Have them locate (download if necessary) a
full-featured word processor and/or spreadsheet application. Have
them manipulate files in the file system: move, copy, rename,
create folders, etc. Have them download, install, and run any type
of software available "out there." Have them double-click an icon
and cause a document to appear in an appropriate application. Have
them change that filetype association (either for that document
alone, or on a system-wide basis). Have them switch among running
applications. Have them find, configure, and use an email client,
check and organize their mail, etcetera. Have them change their
wallpaper, folder backgrounds, etc. Have them change their screen
resolution. You might want to have them set up a web or FTP server,
or enable telnet access into their machines. Have them change their
network information: IP and DNS addresses, for example. Have them
access shared Windows volumes on a network. Have them exchange
data between partitions on the same machine; can they move BeOS /
Linux data over to Windows or vice versa? Have them play some music
(possibly from CD audio or from MP3 files). Have them record audio
or video data to a file. Have them create and work with a personal
contact database. Have them configure their screen saver. Ask them
to get information on their installed hardware and the resources it's
consuming. And so on and so on. Observe the users, noting any aspect
of usability process that proved either easy or difficult. Interview
them about the experience.
o Interview the users about the "sex appeal" of the operating systems
tested. Was the look and feel appealing to the users? Did the operating
systems feel "responsive?" Did they remain responsive under heavy
multitasking loads?
o For extra credit, you may want to provide the users with additional
or replacement hardware. For example, you might hand them a new
video card (or install it for them) and ask them to get their systems
working happily with the new hardware.
Your final analysis may be divided into sections such as:
- Installation: If unable to install, everything else is irrelevant.
- Configuration: The defaults may be good, but how easy are
they to change?
- Familiarity: How long does it take to get "up to speed?"
- Acclimatisation: Beyond the basics, what else is this system good for?
- Specialisation: Does the OS make good sense only for certain kinds
of tasks?
- Dual-boot issues: Do users need to boot other OSes for
specific tasks?
Of course this is just a set of suggestions, and you'll want to
modify them to suit your needs or target readership. If you intend
to perform the experiment in a "scientific" manner, you may want
to hire the services of an official usability laboratory, though
informal experiments certainly have their value as well. You may
also find it beneficial to be in communication with BeOS and Linux
experts/representatives prior to writing your final analysis.
I fully expect to see Linux win on some of these points, and BeOS
on others. I fully expect different kinds of users to have different
experiences and perceptions. The point of the excercise is to raise
the questions, "Which of the popular alternative operating systems
are best suited to meet the needs of new users looking for a way
out of Windows? Which of the popular alternative operating systems
present fewer hurdles when it comes to installation, configuration,
and general usability? Which of the popular alternative operating
systems presents a better base of easy-to-use productivity, media,
networking, and other types of software?" This challenge is not meant
to be an "Enterprise Shoot-Out" or "Server Comparison." Rather, it's
a simple test of which alternative operating systems are well-suited
for users looking to replace or augment Windows or MacOS on the
desktop. I'm sure that you and your readers will find that the test
and its results make for interesting reading.
Linux, as you may know, is a free operating system. You may install
it without licensing issues from any existing Linux distribution CD,
or download the entire system (not recommended). Start at www.linux.org
for more information.
Thanks for your time,
[your name here]