A recent conversation about user testing new features and designs induced me to get a little more organized in my thinking about the right method for particular testing goals. Two key dimensions to consider are:
- User Dimension (X Axis): Is our research goal to better understand how users are behaving on the site (aka, what users are doing), or is it more about understanding their underlying goals, tasks, and assumptions (aka, why users are doing what they do)?
- Product Dimension (Y Axis): Are we more interested in how well our product is aligning with user goals and tasks in terms of features, content, and high-level structure (aka, our product Strategy*), or are we measuring how well the design is presenting the product through visual presentation, IA, and labeling (aka, our product Surface*)?
* Surface and Strategy are designations borrowed here from Jesse James Garrett's Elements of User Experience. Surface is represented by the eyeball icon; Strategy by the lightbulb.
Given those dimensions, I tried to map the relative usefulness of various research and testing methods. What are your thoughts? How would you arrange these research methods?