Appendix B - Heuristic evaluations

Evaluation 1

This assignment will be an evaluation of our group's address book application for the notebook. In order to do a heuristic evaluation of the low-fidelity prototype I've used the following scenarios:

The ten usability heuristics states that the addressbook prototype should have the following properties to be a good design:

Scenario 1: Searching for a person by name

When doing a person search in the addressbook the user fills in the first and second name and the program does a letter-by-letter match on the second name. If there is a match the person who has been search for is highlighted in the addressbook. Some comments about this scenario can be expressed. Because the addressbook does a second name match it would perhaps be more logical if the last name field was placed above the first name field. The dialog when filling in the names is quite natural since a new name can be filled in when pointing the stylus at a name field. One comment here though is that the user would be helped if there were some sort of visualization of how the written text was translated. Otherwise the user gets no feedback about the translation and why a letter appears as another letter. The translation visualization would provide error prevention as well as good feedback. There should also be an error message when the program doesn’t recognize a letter. The buttons in the stylus-input-window is good as the user got the options to clear, cancel or submit (press ok) the input. The only feedback the user gets when there is no match after a search is a line in the status field. There should be a more clear feedback message when this occur, perhaps a sound could be used that states that there was no match after the name fields have been filled. Another thing that can confuse the user is the search button that should be used in order to do an e-mail or first name search. There is a possibility that a user who isn’t familiar with the program thinks that he/she got to press search to do a first name search and haven’t noticed that the program does the search automatically. In that case the program will do a first name search and the full name search result will disappear. One way to solve this problem would be to make the search button "non-pressable" when not only the first name field is filled. One good thing about the letter-by-letter search is that the user may get noticed that a user isn’t in the addressbook before he/she has written the whole last name. This is at the cost of consistency though, as the other search functions need the search button to be pressed.

Scenario 2: Searching for a person by e-mail

To do an e-mail search the user fills in the e-mail address and presses the search button. The e-mail input works in the same way as the name input, which has been commented above. Otherwise the e-mail search is quite logical and straightforward. There could be an @ already filled in to prevent that error to occur.

Scenario 3: Modifying a person in the addressbook

Provided that a person in the addressbook is selected a user can press the modify button to change some of the marked persons info. Here we have stated that the change will be done directly in the fields in the addressbook. I think it would be easier and more natural if a new window pops up upon modify with the info to be changed. This way the user will get a better visualization that the info can be changed. It will then also be easier to add more than one address/telephone/e-mail. There should also be an error check function that checks for a @ in the e-mail address and that avoid the user from typing letters in the telephone number field. Those error check functions would easy prevent errors of trying to e-mail an address that is not valid at an early stage. An even easier way to change some info about a selected person is to use the shortcut by opening the pop-up menu on the info to be changed.

Scenario 4: Adding a person to the addressbook

The primary way to add a person is said to be by typing first and last name into the fields or typing in the e-mail address and then pressing the add button. When adding by filling in the name I would like a pop-up window in the same way as mentioned in the modify scenario. In the pop-up the name will then be filled and the rest of the info needs to be put there. When using this kind of pop-up window you actually don’t need to type in the name before pressing add, because this could be done in the new window. Of course there should be error-handling functions in the same way as explained above when adding information. When adding a user by filling in the e-mail field the user could either press add and then the procedure will be as explained. The user also got the option to download info from the university server. If for some reason the server IP isn’t filled in the user will encounter an error message. It would be nice if the user could be linked to the server properties from here. It can be questioned if all the users know what an IP address is and though if the program speaks the users language. There could be some sort of default value that comes with the program when the user gets his/hers notebook with the programs installed. Whenever the campus computer administrator changes this IP address there will be a clear description how to change it.

Calling a person in the addressbook

The way to call a person is quite straightforward. The user selects the person to call and then presses the make-call-button. A problem will occur though if the person that the user tries to call has more then one telephone number. In this case the user should get a pop-up window that states this fact and asks the user to select which of the numbers to call from a list. The user also has the possibility to call a group of persons to do a small teleconference. Selecting person by person and adding them to the contact list does this. In this case it can happen that a selected user that the user has added doesn’t have a phone number or there is more than one number filled in. If a person that the user added to the contact list doesn’t have a phone number in the addressbook the user will encounter an error message when trying to make a call. This message will state the fact that a person with no phone number can not be called and there will be a possibility for the user to fill in a phone number for that person or to remove that person from the contact list. If a person that is tried to be called has multiple phone numbers the program will ask what number to call.

Sending an e-mail to a person in the addressbook

Sending e-mail to a person or a couple of persons in the addressbook is done in the same way as making a call. Therefore there will be the same sort of error-handling when a person has none or multiple e-mail addresses.

Other issues

A consistency regarding the dialogs exists in the addressbook. In the pop-up windows there will always be one OK-button and one Cancel-button when necessary. These buttons will be placed in the same way throughout the program. The Cancel-button also represents a clear marked exit to the user. The error messages are good and easy to understand throughout the program and there is a lot of shortcuts to make the users work easier. Other from what have been mentioned earlier the dialogs are simple and natural and there aren’t any functions that heavily load the user memory. One thing that could be added is a help choice along with the file- and edit-choices. This way the user can get help through the program to solve simple questions.

Evaluation 2

This assignment is a heuristic evaluation of the low-fidelity prototype in our "Address book application for a Notebook". This evaluation may not be as good as if the evaluation had been performed by a group of individuals. Furthermore, I'm one of the developers in this application that may make it harder to find the applications week spots but on the other hand those problems found are real ones. The evaluation has been performed on by using the tree scenarios, search the address book for a person, add a person and send mail to the selected person.

The experience and errors when walking through these three scenarios are discussed in the Ten Usability Heuristics presented to us in the course. Problems found are graded on a scale of 1 (minor) to 4 (extremely serious)

Simple and natural dialogs

There are only two dialogs, the preferences and the stylus input dialogs. These are simple enough but if we were to use www.liu.seinstead of the IP-address this would be even better (grade1). In the search scenario are there some errors messages missing though.

Speak the user language

All buttons are labeled with what action they perform. The button label and other text and error messages speak user language.

Minimize user memory load

The address book shows all its controls in the main window, like all buttons and the search/edit/contact list group. This makes it easy to see all the search options at the same time.

Consistency

The application shows a consistent interface throughout all dialogs and error messages/alerts.

Feedback

The address book also gives good feedback as the status field at bottom of the main window shows info about every button and text field when the user moves the mouse over an object. The feedback is also excellent with the "Real time search" when typing in last name field. There should be a progress bar implemented in those situations when the app is "busy", for example when downloading info from campus server (grade2).

Clearly marked exits

The address book may be closed by typing Ctrl-W, selecting Close from File menu or click in the upper left corner (or upper right corner if the address book is running in Windows 95/98) and makes the exits in the application clearly marked.

Shortcuts

Shortcuts such as Ctrl-W (close the application), Ctrl-O (open the address book), Ctrl-C/V/X (copy, paste, cut) and keyboard navigation is possible in this application.

Good error messages

Those error messages that are presented in our current documentation do not cover some errors like that the server isn't in range when Download-button is clicked (grade 1). Others are when the Search-button are pressed with e-mail, last- and first- name fields empty (grade 1). The same when Add-button is pressed when no person in the address book is marked (grade 1).

Prevent errors

The search operation doesn't need to be performed as sequence of steps that has to be followed. The user may instead leave the search operation or close the address book at any time.

Help and documentation

Help should be implemented as a menu at the File/Edit level. The help and documentation should be reached from the Help menu (grade 1).

Evaluation 3

In Assignment 4 a heuristic evaluation of our low-fidelity prototype will be done. It is a model that analyses the usability of an interface by inspection (similar to a code walk through). It is recommended that three to five evaluators do the usability inspection independently. There are 10 main usability principles to think about.

1. Simple and natural dialogs

2. Speak the user language

3. Minimize user memory load

4. Consistency

5. Feedback

6. Clearly marked exits

7. Shortcuts

8. Good error messages

9. Prevent errors

10. Help and documentation

These are the scenarios and problems I found in our interface.

All problems are numbered 1 to 4 where1 is a minor problem and 4 is extremely serious problem.

When writing name/e-mail with stylus the program has to print the name/e-mail in a field on the screen. The user can see that the program has got it right. If not, the user has to write the wrong characters or the name/e-mail again. Without this function the user doesn't get feedback. If the user believes he is searching for "kalle" and the program reads "kaiie" there will be no match. Those problems are solved with this function. I think this is a extremely serious problem because the user might believe that a person isn't in the addressbook but in reality he is. (4)

The user can't get information about functionality or help if needed. We have to implement a help menu where the user can search for specific functions or where he can get information of how the program works. The help-menu can look like this:

This is also an extremely serious problem. The user can't use the program if he can't get the information or the help he needs. (4)

If a e-mail address is added with no @ character

A pop-up menu with the following message will appear: "The e-mail address have no @ character, please add one!" The user has to press the OK button. The cursor will be in the e-mail field with e-mail address marked.

This function can prevent errors but it isn't a serious problem. The user will notice it sooner or later anyway. (2)

Download button is pressed.

A pop-up menu with the following message will appear: "Loading personal information of xxxx@xxxx." This function will probably take some seconds and the user has to get some feedback while waiting. It is a serious problem because the user might think nothing happened and tries to download again. In the worst case that may cause a system crash in the program. (3)

Download button is pressed but the server doesn't answer.

A pop-up menu with the following message will appear: "The server doesn't answer. Check the IP-number and try again." The user has to press the OK button.

The user has to get feedback about it. He can't understand what had happened and what to do. I think this is a serious problem because the user might think it is some problem with his computer that it often isn't. (3)

Maybe it is better to use the WWW address instead of the IP address (EDIT menu and preferences). It will be easier for the user to understand. This is a minor problem. (1)

Search button is pressed with empty name/e-mail fields

A pop-up menu with the following message will appear: "The name or e-mail field are empty. Write name/e-mail and try searching again" The user has to press the OK button.

The user has to get feedback of what happened and what to do. This is a problem that the user might understand anyway. You can't search for nothing. The program must have good error messages. (2)

One or several selected person haven't got telephone number/e-mail when Make a call/Send e-mail is pressed

A pop-up menu with the following message will appear: "Some persons haven't got telephone number/e-mail. Do you want to add them?" The user has to press the YES or NO button.

This prevents errors. The user gets the error message before sending instead of after. If he take NO he will get them after. It's up to the user to decide how important it is that everyone gets the call or the e-mail. I believe this is a serious problem. (3)

Clear button is pressed with one or several persons selected in contact list.

A pop-up menu with the following message will appear: "Are you really sure you want to remove the selected person/persons?" The user has to press the YES or NO button.

It's important for the user to be reminded before erasing something. This prevents errors but it isn't a serious problem because all persons are still in the addressbook. The user can just make a new list. (2)

Add button is pressed with no person selected.

A pop-up menu with the following message will appear: "No person is selected. Select person and try again!" The user has to press the OK button.

The user has to get feedback of what had happened and what to do. This is a problem that the user might understand anyway. You can't add something that doesn't exist. The program must have good error messages. (2)

When adding data directly in the address field there will be a problem to add several addresses, telephone numbers or e-mail to one person because there is no triangle. My suggestion is that there should always be a triangle with several lines to write in. If a person has several data in one field the triangle is marked and if not, it isn't marked. The user can always see if there are several data in a field and he has no problem to enter several data directly in the address field. The program will be more consistent and simple. (3)

Evaluation 4

This report will show some scenarios in order to operate the addressbook prototype. Under each scenario I will also show the problems that I have found while I was evaluating the groups project. To grade the problems that I found I have used the scale of 1 (minor) to 4 (extremely serious).

Scenario 1 - Adding a person to the addressbook using a stylus

When the user selects a name field with the stylus a pop-up window appears to write the name of the person that is to be added. Then the user writes the name and clicks the Ok button to add the person. A problem here is that the program might have problems interpret what was written with the stylus. It would have been better if we had included a component that shows the interpretation when an Interpret button is clicked. If the user finds the interpreted name correct he or she could click Ok to add the person. If the user finds the interpreted name to be incorrect he just clicks the Clear button to write the name again.

When the person is added the user will have to click each of the fields in the addressbook to get a pop-up window to write the address, telephone number and e-mail respectively. This window should also need an interpretation component so the user could check that the written words are correct.

The found problem must be corrected to prevent errors in the program. I think that the problem described above is an extremely serious one, which I give the grade 4.

A less serious problem when trying to add a person to the addressbook is, if the user clicks the add button before writing a name in the name field, nothing will happen. A solution would be a pop-up window that will alert the user that a name is missing in the name field. This problem I grade 2 since the program can work without the pop-up window.

Scenario 2 - Calling or emailing more than one person

If the user wants to call or e-mail more than one person at the same time he just clicks on a wanted person in the addressbook and then clicks the Add-to-contact-list button. Then that person appears in the contact list. When the user has selected the persons to call or e-mail he pushes the Make Call or Send Mail button. Since a person in the addressbook might not have an e-mail address or a telephone number there will be a problem to reach that person. Our mistake is that we have not included an alert message. This is of course is a serious problem.

The message should say something like "One of the wanted persons does not have a telephone number or e-mail address. Do you want to remove that person from the contact list and call the other persons?" The user can then click Yes and the person will be removed from the contact list and the calling function proceeds. If the user clicks No he might remove the person by himself by clicking a Remove-from-contact-list button. This button is not included in our project and that is also a serious problem.

The first problem described in this scenario I grade to 4 since it will create serious problems to the telephone or e-mail programs. The second problem is also given a 3 since it would be rather user-unfriendly if the user will have to clear the contact list and start adding the persons again.

Scenario 3 - Adding IP-address in Preferences

The user will have to add an IP-address in Preferences under the Edit-menu in order for the addressbook to find the server that contains the information to be downloaded. Preferences are a pop-up window that appears when Preferences is selected under Edit. Maybe there is a problem of not speaking the users' language here. The text says: "Fill in the IP-address of the address database". I don’t think that every person knows what an IP-address is. Perhaps it would be better to ask for a www-address instead.

I grade this problem 2 since I think that the user who does not understand what an IP-address is can learn about it if he or she likes to.

Scenario 4 - Adding a person to the addressbook

When adding a person to the addressbook the user writes the name of the person in one of the name fields in the Search/edit-group. Then the Add button has to be pushed in order to add a person that doesn’t exist in the addressbook. Then the user has to write the address, telephone number and e-mail address in the fields in the addressbook. This is not an easy way to do this. It would be better to implement a pop-up window where the user can write all the information about a person and then click Ok to insert the person in the addressbook.

I grade this problem 2 since it can work the way we said, even if it might be a bit difficult.

Scenario 5 - Adding more than one address, emails or telephone numbers to a person

A person can have more than one address, e-mail or telephone number. In the project we have a feature that allows this. A triangle to the left of the person's information indicates this. Unfortunately we have not thought enough about how this can be done. As it is now I don’t think adding more than one of each can be done in a useful way.

To solve this problem a pop-up window that asks for every detail in the addressbook might be useful. There the user can add the name, address, telephone number and e-mail address. If the person has for example more than one address a drop-down menu will give the user the opportunity to add the extra address, e-mail or telephone number.

This is an extremely serious problem, which therefore get the grade 4.

Scenario 6 - Searching on e-mail

If the user wishes to search on the e-mail address entering the e-mail address in that component can do this. Then by clicking the search button the program looks for the wanted e-mail address. If the user forgets to write a @ in the e-mail address there will have to be a pop-up window that tells the user that the @ is missing.

This problem I grade 4 since the e-mail program might crash if the @ is missing.

Above the search button there is a text saying Search book. This is not a good language I think. It would be better to write Search addressbook. A user that doesn't understand the program might think of some other kind of book. I grade the problem 1 since it is a minor one.

When searching on e-mail it might happen that that e-mail doesn't exist in the addressbook. Then the user will get the opportunity to download the rest of a person's information by clicking the Download info button. Then the person will be added in the addressbook.

If the IP-address is not added in Preferences the program will not find the server. As it is now no alert message is given. Therefore we will have to implement a pop-up window that gives the user a message that the IP-address must be filled in under Edit/Preferences.

There might also be the problem that the server is out of order at the moment of downloading. Then it would be useful with a pop-up window that explains that the server is not working and that tells the user to try again later.

Those two problems I grade 3 since I believe they are quite serious. It is almost impossible for the user to understand what is wrong if these messages are missing.

Scenario 7 - Searching for e-mail or first name

When searching for e-mail or first name the user must write something in at least one of those fields. Then the user must push the Search button. If the user doesn't provide either a first name or an e-mail there need to be a pop-up window that alerts the user. We have not provided this pop-up window and that is a mistake. The pop-up window should tell the user that the first name and/or e-mail must be written in the fields before the search function will work. Another solution might be to keep the Search button shaded if no first name or e-mail is written in the fields.

I grade this problem 3 since the user might not understand what is wrong if the program doesn't respond with a message.

Scenario 8 - Starting a new addressbook

If the user wants to start a new addressbook he chooses File-menu and then there should be a New option. That is normal in all programs but unfortunately not in our addressbook. This is a serious problem which I grade 4.

Scenario 9 - Calling a person with more than one telephone number or e-mail address

In order to call a person the user will have to click the person's name. But if the person has more than one telephone number there will be a problem. The program doesn't now which number to add to the contact list. A solution can be that the user must select one of the numbers in the telephone number field to select it and then click Add-to-contact-list. Another solution can be to show a pop-up window where the user can select the number to call. I grade this problem 4.

Other bad things in the program

The program doesn't provide a Help function in the menu. That might be a problem to the user but as I think that the program is pretty easy to use I grade this problem 1.

In many programs the developers have provided a function that shows a text about a button when the user moves the mouse over the button. For example when the mouse enters the save button the text says "Save". That should be provided in our project to make it even easier to use. I grade this problem 1 since I think this function only helps a new user.

The Add, Modify, Remove, Search, Download info, Make Call and Send Mail buttons are easy to use since they are easy to see. Maybe these functions should have been provided under the Edit-menu as well. There are users that like to use the Edit-menu instead of buttons. If these functions are provided under the Edit-menu there should also be shortcuts implemented for each one. I don’t think this is a serious problem so I grade it 1.

We have not provided a menu with the New, Open and Save functions as buttons. This should of course be a function that the user can select under some kind of preferences. Some users don’t like too many buttons so it would be nice if the user can configure the program, as he likes best. I grade this problem 1 since I think it is a minor one.

To sum up

Almost all of the above mentioned problems are serious ones. They are serious problems since they pretty much are missing messages in the program. I think that our addressbook is using a natural language that is used by the user. We have minimized the user memory load and all pop-up windows are consistent. All exits are clearly marked and there are shortcuts to be used by an advanced user. The error messages that exist are good. Unfortunately we have missed a lot of error messages. The help and documentation is not provided and that might also be a problem. Feedback is given to the user in a status bar located at the bottom of the program, as is usual in many other programs.


You might now want to go to the next page, go back or go to the first page.