A Two-Dimensional Approach to Diagnose Website Performance

It has become quite the juggling act for site owners to manage and monitor a Website that caters to multiple audiences, while trying to increase and maintain the highest level of site performance. While the goal of current Website design is to create an interactive experience delivering more relevant content to users, Websites have become increasingly more complex as a result. The best methodology for evaluating and enhancing Website performance is to merge site behavioral metrics with attitudinal survey data, enabling site owners to make tactical and strategic changes that serve to minimize user pain point areas and optimize the visitor's overall Web experience.<br clear="none" /> <br clear="none" /> Traditionally, Website satisfaction research has been aimed at monitoring shifts in site satisfaction and providing high-level explanations for these shifts. For instance, satisfaction has increased in the current month as users found the content more up to date compared to last month. This approach to delivering results was useful because many site owners only needed to maintain a high-level understanding of their Website's performance. However, as sites become increasingly more complex, site owners are demanding more than just "taking of the pulse" metrics. They are looking for a complete set of "diagnostics" that empower them to make both short-term and mid-term fixes to their site.<br clear="none" /> <br clear="none" /> Website satisfaction surveys, as most research firms provide, no longer are able to meet this need and should be characterized as "feedback" mechanisms rather than Website optimization tools. In providing a quantitative overview of the Website landscape, traditional analysis focuses on conducting advanced analytics such as regression, requiring large samples and only delivering macro-level recommendations. In response to this gap, site owners increasingly have turned to usability studies for providing qualitative data they can easily wrap their heads around. However, usability studies are one-point-in-time, snapshot studies and are not optimal to meet the needs of site owners who are left still looking for a methodology that can deliver continuous improvements to their Website at a reasonable price.<br clear="none" /> <br clear="none" /> 

In an attempt to fill this ever-expanding gap for Website owners, metrics companies such as Omniture now are dabbling in the arena of conducting surveys. Their hook is different than traditional research companies; they link attitudinal and behavioral information to provide a better understanding of Website performance. However, site metrics companies are not research companies. These companies focus on providing as many data points as possible to site owners, not on data analysis leading to actionable recommendations. As such, site owners are left with little more than high-level feedback, conducted in an unscientific manner, along with behavioral data and still no clear recommendations on how to improve the site.<br clear="none" /> <br clear="none" /> 

Many times, site owners are left feeling they could have gotten the same value from a page-level feedback tool such as OpinionLab, so why pay for the additional work Omniture provides? A company such as OpinionLab provides simple feedback that is good for collecting open-ended verbatim responses, but should not be used to plan strategy around since there are no sampling checks and balances around the collection of data. Data collected in a non-scientific data manner should never be used in a quantitative manner to develop strategies since the data is not representative of site visitors.<br clear="none" /> <br clear="none" /> 

Combining both behavioral and attitudinal data for analysis forms the basis for making continuous improvements to the Website. However, how this data is collected, analyzed, and used together is the key to providing actionable information to site owners.<br clear="none" /> <br clear="none" /> <b>Case Study</b><br clear="none" /> <br clear="none" /> A Fortune 100 company recently was faced with an issue regarding its worldwide site of more than 300,000 pages. It was trying to determine why a key metric&#x2014;links leading to the correct content&#x2014;had declined from May to June 2009, but satisfaction had not declined significantly over the same period. Looking at the attitudinal data, it appeared links on the site were no longer directing users to the desired content in an effective manner. However, this was not enough information to provide site owners with direction to fix the problem.<br clear="none" /> <br clear="none" /> Analysis of behavioral data involved splitting the Website into various sections and then cross-comparing attitudinal scores section-to-section. This was not an easy task with a Website with more than 700 sections. Nevertheless, this led us to pinpoint the section where the deterioration in the "links" metric took place. Unfortunately, this section was more than 100 pages deep. The next phase involved a systematic analysis of key index pages within the poorly performing section to find a key intersecting point where users converge that faced this similar issue with the links. Of the indices pages, there was one that had a high percentage of users who fit our criterion of rating the site poorly in terms of links leading to the wrong content.<br clear="none" /> <br clear="none" /> The final piece of the puzzle was now qualitative in nature due to the small sample size; it involved an audit of the pages users visited to get to the index page, as well as high-trafficked pages users went to from the index page. Working backward and forward, we found a page that preceded the index page that had an important dead link, initially intended to lead visitors to the index page in question where we uncovered an intersect of users rating the links poorly. As such, we were able to uncover the issue down to the page-level and provide an actionable recommendation to the site owner: fix the dead link with the correct link.<br clear="none" /> <br clear="none" /> The recommended change resulted in a 10 percent increase in the scoring of the Links metric and a subsequent 4 percent increase in overall visitor satisfaction. Although the solution may have been fairly straightforward and simple, merging both behavioral and attitudinal data enabled us to identify the exact page that needed to be corrected among 300,000-plus pages of a global Website within a matter of hours for quick execution.<br clear="none" /> <br clear="none" /> <i>Jason Ten-Pow is the president of Research Operations at OnResearch Inc.</i>