Chapter 13 is discussing the need of setting a framework of your study. This will drastically help you to execute professional methods, such as interviews etc. where you know that you will acquire information that is relevant. A good evaluation is based on certain goals that really aims to seek answers.
A good way of making sure that you cover all the essential aspects of this framework is using the DECIDE structure.
Where you first should define clear goals with your study, what should we achieve and what is really the weighted value in the end?
When exploring different questions you need to start somewhere, so bring up potential questions that could potentially answer respective areas of interest. It is however important that we choose questions where you leave some room for subjective answers. Sub-questions may be very useful as a next step, have a look at your computed questions and try to turn them into sub-questions. That is, break them down into more specific areas if this could give more comparable or problem addressable answers.
When coming to the evaluation method of your choice you should take time to think about what would work and what could not work? Maybe interviews isn’t the right choice considering the type of data that you want to achieve.
Also one of the key aspects of a good evaluation is to involve the right kind of participants.
After this we should identify possible practical issues, such as do we have resource limits? Will we have the contacts or possibilities to come in contact with the appropriate kind of people? Important to set schedules and budget constraints so that the group can function and keep track of progress through a timeline.
A big concern when conducting different kind of research methods is the aspect of ethical issues. In more professional projects and maybe more sensitive subjects the need of anonymity is important to the participants. You should be very clear from the beginning with how their data and personal info will be handled, analyzed and reported. A smart thing to do is to offer them to review the report before you make the report official. This way you will eliminate potential fears and maybe you gain more trust and that means more qualitative answers.
If making tests with people as part of an evaluation task, make sure to treat them courteously. Especially if the task itself takes more than 20 minutes, offer them breaks and other reliefs so that they will perform normally when continuing.
Evaluate, Analyze, Interpret, and Present the Data
We have to think about how we should analyze, interpret and how to present it. Will the method really measure what it is intended? and is it generalizable?
You can check these questions with different key areas
Reliability: Is the method consistent? are you allowing lots of free questions in the interview or keeping strict to a few ones that can easily be compared with each other?
Validity: How valid is the method with perspective to the goal of the evaluation?
Ecological validity: This is also a validation check but with respect to the surroundings of the subject when conducting your methods. Will a noisy cafe atmosphere interfere with your intended goals of seeing how well a student can think in calm areas? probably..
Biases: The risk of not getting you information without certain biases, for example, only interviewing a certain kind of person within your target group, or phrasing leading questions etc.
Scope: How much of the study results can be generalized?
Question: What evaluation method should we use for our project?
Chapter 15
The heuristic evaluation is a form of a usability inspection method commonly used when you want to identify usability problems with a user interface design. There is some well formed principles but exactly which heuristics and how many is not written in stone. It is rather a matter of what fits most to respective project. I think one good question for the group session is
Question: Which kinds of heuristics could help our evaluation of our product?
A good thing about setting up and acknowledging different heuristics is that we will have a stronger focus on the design and its interaction when proceeding with our prototype.
Another kind of evaluation of a software product is to do a Walkthrough. This means that a designer or lead programmer shows their program step by step for other people within the development team and then they can comment on potential errors that could occur in each step.
A cognitive walkthrough is also a way of finding interactive issues for certain software systems. Here you focus on how easy users can manage to do the tasks that they are supposed to.
Inga kommentarer:
Skicka en kommentar