Usability Analysis of a Client Knowledge Portal (3 of 7)

CONTENTS
Introduction
Expert review
Task analysis
Usability testing results
Survey results
Appendix A
Appendix B

TASK ANALYSIS WITH EXPERT USERS
This portion of the analysis consisted of observing expert users “at work”. The three subjects were knowledge stewards and had published documents before. Subjects were prompted to go through the process of publishing a document as they normally would and their task process was documented. Any bottlenecks or means of circumventing problems during the publishing process provide insight into design problems, which need to be addressed. After each task analysis, redesign recommendations pertaining to the subject’s particular problems are listed. It should be noted that some of the recommendations for redesign parallel those outlined in the expert review.

Subject #1

1. The subject goes through technology/application servers/Netscape application servers/FAQ and selects “publish” button.
   
2. He calls the document “expense report”.
   
3. He is not sure a few of the document types refer to and so says that he would choose “other” as document type because he has a technical document and does not see this option.
   
4. He is not sure why file size is needed and cannot remember how he has addressed file size when he has published before. He would either go to explorer and find out the file size or leave it blank. The file size he chooses is 30K and he is not sure what to write in the window- 30, 30K, or 30,000?
   
5. For title, he says that he would not necessarily put the same name as the file name.
   
6. He changes file format to Excel.
   
7. He leaves the default “yes” selected so document comes online immediately.
   
8. He already has information in the choose categories window, but selects choose categories and says that he assumes that this enables him to put the file in multiple locations. He double clicks on an item in the left window as if this would make the item appear in the right window. He does not add any items to the right window.
   
9. He does not change the date so that the file will go on-line immediately.
   
10. He selects next and on the next page he selects browse and chooses the file. He does not select publish.
    
11. Subject is asked where he would go to find the document. He says that he would navigate to the proper location in the site and that it would be useful to have the ability to immediately be taken to this location.
    
12. He navigates through technology/application servers/Netscape application servers/FAQ and finds that his file details are there without an attachment, even though he did not select publish.
    
13. He tries to click on the file icon but it is not a link.
    
14. Subject is asked how he would find the file a month from now if he were not sure what he called the file or where it was located. He goes to search and types “expense” under title and get no results. He types “expense report” and again gets no results. He does not know if he needs to enter information into every field to get results.
    
15. He changes document type to “other” and again gets no results.
    
16. He puts his name in the “author” field and gets a listing of other files he has published.

Subject is prompted to publish a rich text file.

1. He again navigates through technology/application servers/Netscape application servers/FAQ and chooses publish.
   
2. On next page he chooses “rich text” and selects next.
   
3. He says that headline would be a title as when one publishes a file.
   
4. For body he puts text and a URL link.
   
5. For keywords to types “information application server comparison” with no commas.
   
6. Chooses “other” for document type.
   
7. For file type he chooses Excel.
   
8. He leaves date and the default “yes” selected so document comes online immediately.
   
9. He does not select choose categories because there is already information in the window.
   
10. For “attach file” he says that he would put the title of the file he is attaching.
    
11. On the next page he attaches the file.
    
12. He selects publish.
    
13. He navigates to technology/application servers/Netscape application servers/FAQ to locate the file and he chooses the file and is taken to the file details page.
    
14. He tries to open the file by selecting attachment but the file type is not recognized.
    
15. He selects edit and is taken back to the publish rich text page.
    
16. He changes the title of the document so that it has an Excel extension. He chooses publish and is taken to the screen where he must attach the file again. He tries to publish again after attaching the file and is given an error message, which says, “the work you have done cannot be saved”.
    
17. He selects the back button and cannot go back.
    
18. He goes to home and through technology/application servers/Netscape application servers/FAQ and sees that his change to the file name has been made.
    
19. The file type still will not be recognized when he tries to open the file.
    
20. He goes directly to excel and tries to open the file and it does open.
    
21. He now wants to delete the file from the site but does not see a delete button.
    
22. He chooses edit and changes the file to off-line.
    
23. He selects publish and again is asked to attach the file. When he selects publish he is given a message, which says that the file has been uploaded successfully. He navigates back through technology/application servers/Netscape application servers/FAQ and does not see the file. He says that if he wanted to put the file on-line again he would not know where to find it because it is no longer listed.

Recommended changes based on task analysis:

• There are not enough document types listed. If in the future a survey were sent out employees could be shown a list of the existing document types and asked what types they would add to the list.
  
• Users should not be asked for file size.
  
• The choose categories window needs to be redesigned completely or at the very least, a few words of explanation should be given right on the page which say that the user has the option to choose a category or another category.
  
• If the same design is used, it would be easier for users is they could double click on an item in the left window and have the item appear in the right window.
  
• It would be useful for the user to be taken to a “manage my documents” page after publishing a document so that he does not need to navigate to that area of the site to ensure that the document was published.
  
• The “next” button on the publish file page should be changed to “publish” so that the user does not move the next page unless he really wants to publish.
  
• Here the user did not attach a file and still was able to see the file details on the document. However, when he searched for the document there were no results. There should either be no file details unless a document has an attachment or search should show documents, which do not have attachments.
  
• The file icon on the file details page is misleading because it looks like a link. It should be removed.
  
• On the publish rich text page, headline should be changed to title so it is consistent with the publish file page.
  
• It is unclear to the user whether keywords should be in quotes or should be separated by commas. This needs to be specified.
  
• When a user makes a change to document, which he has published, he should not have to reattach the file unless he needs to do so.
  
• When a user receives a message indicating that changes cannot be made this implies that he was unsuccessful in making changes. However, this is not always the case because here changes were actually made to the file name. The user should not see an error message if there is actually no problem.
  
• A user should be given the option to delete a file within the channel where the file is published so that he does not need to go through the edit published file process if he simply wants to delete the file.
  
• The option to delete a file is currently not available.
  
• When a user makes a document offline, he should not be asked to attach the file and should not be given a message, which says the file has been uploaded successfully because this is a contradictory message. The message says that the file has been taken off-line.
  
• A “manage my files” page would eliminate the user needing to navigate to the file location- here the user repeatedly needed to do so and it was very time-consuming.
  
• Once a file has been taken off-line there is no way simple way to change its status to on-line.

Subject #2
The subject is prompted to go through the process of publishing a document as he normally would.

1. The subject navigates to technology/application service providers/white papers and chooses publish and then publish file.
   
2. He includes the date that he is publishing in his title because he is not sure whether this information is given automatically when he publishes.
   
3. He fills in all of his information but is unsure why he sees categories that he knows he is unauthorized to publish to. He is also unsure whether he needs to only move to the right window the folder he is publishing to or the folders at a higher level as well.
   
4. He is not sure why file size is important.
   
5. On the second publish page, he chooses browse and notes that the default file type is html and that he must change this to “all files”.
   
6. He gets a confirmation message telling him that his file has been uploaded and because he does not want to go home which is where this page will take him if he selects home, he presses the browser’s back button and get an error message.
   
7. He goes back to tools to find the file that he has published. He does not know what rich text is but assumes that since he was publishing a file that he would select publish file.
   
8. He then goes to home and selects ASP white papers which is one of his favorites and it takes him to the white papers listing and he sees that his file is there.
   
9. The subject wants to publish another file and he goes to marketing/internal communications and he does not see a publish button here (he would need to go down another level) so he goes to tools/publish file. He guesses the file size and writes “2 meg”. “Presentations” is listed under categories although he accessed the page via tools.
   
10. He finishes filling in his information and then receives an error message, which tells him that he does not have access to publish to this area. He has already gone through the entire publish process.
    
11. He suggests that instead of a cryptic message that instead the user sees a message, which explains why he does not have authority and perhaps directs him to the knowledge steward in case he would like to ask any questions regarding publishing privileges.
    
12. He points out that keywords are not standardized and so he must hope that his own choice of keywords when publishing and when searching for documents is the same as other users’.
    
13. He opens help and is confused because can’t go back to the site. He does not realize that home is separate window.

Recommended changes based on task analysis:

• Documentation needs to be provided to the user regarding “choose categories” and where the user is authorized to publish and what folders need to be moved to the right window when choosing where to publish. This should be documented in help in detail as well as in a tutorial and there should also be a brief description on the publish page.
  
• The top navigation bar needs to be on the confirmation page, which appears after a file has been published so that the user can navigate elsewhere easily.
  
• The user should be able to go back and publish another file from the confirmation page so there should be a link back to publish file.
  
• On the tools page, there needs to be a brief description of the difference between “publish file” and “publish rich text.”
  
• Choose categories should be located closer to the top of the page so that if the user does not have access to publish to a desired location he has not filled out the entire form prior to discovering that he does not have access.
  
• The error message, which appears when a user does not have access to publish, should provide more of a description as to why access is denied and who could be contacted to possibly request access.
  
• There should be a standard set of keywords that users can choose from when specifying document details to increase the likelihood that documents will be readily located using search. There should also be the ability to provide keywords that are not standardized in case there is a document, which is a unique case and does not match typical keywords.

Subject #3
The subject is prompted to go through the process of publishing a document as he normally would.

1. The subject states that he has been in the position of wanting to add a category but has not had access.
   
2. To find file size he opens explorer.
   
3. The first time he was publishing he was not sure whether he wanted to publish the document and so he put the documents offline and had other people log in as him and view the documents. Alternatively, he would send someone an email with the document attached to have him or her read it before he published it.
   
4. Since the file he is publishing will be offline he does not specify a date.
   
5. He does not understand the function of choose categories; he thinks it has something to do with the search function or perhaps it enables him to publish under multiple categories. He decides to publish under more than one category to test this hypothesis.
   
6. He looks for the file and cannot find it. He says that this happened to him before and so he put the file online and “hoped that no one would see it” so that he would not lose the file.
   
7. Suddenly his file appears and he chooses edit and makes the file online at a future date. One of the categories he had specified is no longer on the list.
   
8. He presses next expecting to be asked to upload the file and he is told that the status has been changed successfully. He thinks that he has done something wrong.
   
9. He goes to find the file again and it has disappeared. He reloads the page and it is still not there.
   
10. He goes to help to see if there is information about uploading documents and it says to pick a date when the file will be removed from the site and he does not recall seeing this option.
    
11. Help says that an upload window should appear and he notes that this did not happen for him.
    
12. He tries to publish another document and puts in online even though he does not want to do so. When he clicks next he gets an error that says that the name already exists. He thinks that his file is somewhere on the site.
    
13. He logs in again and cannot find the document, yet he is given an error, which says that there is a file existing with that name.
    
14. He decides to publish the file under a different name to “fool the site.”
    
15. He tries again to publish a file and gives it a new name and puts it online. He successfully publishes the file and is able to locate it on the site.
    
16. He tries to find a delete file option or another way to prevent others from seeing the file. He does not trust the offline feature. He cannot find a way to delete the file from the site when he looks in help.
    
17. He does not think that search will lead him to documents, which he has put offline since one of the search options is “online since.”
    
18. Previously when he tried to delete a file from the site he went to submit feedback and asked how he could delete a file and he did not get a response.

Recommended changes based on task analysis:

• It should be specified in help that average users are not able to add categories to the site but that a request could be made by sending email to system admin (assuming that this is true).
  
• There should be an option to publish documents to the site and only allow certain people to view the document.
  
• As already stated, choose categories needs to be explained more clearly to eliminate confusion.
  
• It takes a few minutes for a file to appear on the site once it has been published. This should be explained on the publish file page so that users do not think that they have made a mistake and try to publish the file again.
  
• This subject specified a few categories and when he went to edit the document one of the categories was no longer listed. This is a bug, which needs to be fixed.
  
• He goes to find the file again and it has disappeared. He reloads the page and it is still not there.
  
• Help describes a way of deleting a file from the site. This feature needs to be added or help should explain that this is currently not an option.
  
• Help says that an upload window should appear but does not explain that this only occurs if a file is to be put online immediately.
  
• While perhaps the same user should not be able to publish two documents with the same name, there is no reason why different users should not be able to do so.
  
• Since this subject is having problems publishing a document and gets an error which complains that a file already has been published under that name, he changes the name to “fool the site.” Inevitably if this scenario continues to occur this will lead to an excess amount of useless files.
  
• It is implied that search will only locate files that are published because one of the search functions says “online since.” Presumably it is true that when searching for others’ files but when searching from one’s own files even unpublished ones should be listed.
  
• Users have problems which help does not address and which they cannot resolve themselves. When their last resort is submit feedback and they get no response, this will lead the user to distrust using the site.
  

<< 1 | 2 | 3 | 4 | 5 | 6 | 7 | >>

Copyright 2002 Chromosome22. All Rights Reserved.

download the WORD .DOC (184k)