Successful benchmarking with SurveyShop
August 2002
Benchmarking your achievements against similar players in the field, whether you are examining rival companies or comparing your organization to others that perform a similar function, is the ideal way to quantify your success, identify any weaknesses in your performance and determine managerial priorities for the future. The big problem is in gathering the data. Enterprises are not going to deliver sensitive information on income and expenditure into the hands of their perceived rivals or competitors.
Often, with benchmarking exercises, the recipients of the reports are not the only beneficiaries. Benchmarking does a lot to enhance the reputation and status of the organizations that commission and publish them.
Successful benchmarking exercises depend on clear boundaries, trusted intermediaries and good relationships. In the two benchmarking studies we will be looking at here, Snap SurveyShop plays a pivotal role as the neutral go-between, gathering the data but only passing on results in aggregate where no individual’s responses can be viewed or even guessed at.
Often, with benchmarking exercises, the recipients of the reports are not the only beneficiaries. Benchmarking does a lot to enhance the reputation and status of the organizations that commission and publish them. In the first example presented here, an annual survey of financial management within legal practices by a leading accountancy firm not only provides valuable insights to its clients and potential clients, but in doing so, enhances its reputation as experts and thought leaders in the field. In the second case study, a trade association has developed a benchmarking exercise, over several years, into being one of the most valued services it provides to its members as well as one that benefits its industry more widely.
Case Study 1: PricewaterhouseCooper’s Financial Management in Law Firms report.
The leading professional services organization PricewaterhouseCoopers (PwC) has a specialist team of consultants that advises law practices in the UK on all aspects of their financial management, the Professional Partnership Advisory Group or PPAG. Every year, PPAG publishes the Financial Management in Law Firms report - a definitive benchmarking report on the financial performance of legal practices throughout the country. For a number of years, Snap SurveyShop has played a central role in the collection and processing of the benchmark data. These cover a wide range of key indicators, such as remuneration, profitability, levels of support staff, levels of financing, investment in IT, training, which make interesting reading not just for partners in law firms, but for the national press, who often pick up some of the key findings each year.
SurveyShop is involved from the moment the survey form is finalized and mailed out, handling the receipt of the questionnaires, entering and cleaning them, then producing the analyzes and outputs on which the four-strong editorial team base the report.
"In the past, we have tried other methods and suppliers, but we have gone back to Snap Surveys on the basis of convenience, cost and service levels," commented Mark Waddilove, PPAG Senior Manager and a member of the report editorial team. "The whole point of using SurveyShop is to underpin the confidentiality of the exercise, as it is only they who see the individual questionnaires. Their collation of the data and production of statistical analysis makes a real contribution to our report."
In the report, the average for each benchmark is broken down by practice size and also geographic region, so that the head of a small firm of solicitors (or general practice lawyers) is not comparing his or her firm’s performance against the big city law firms in London. Trends are also presented, from reference to previous years’ data. PPAG also provides valuable, thought-provoking commentary in the report.
When the report is published, it becomes a powerful marketing tool. One copy of the full report is sent to each law firm participating in the survey. Each practice, not all of whom are PwC clients, is also invited to a series of very popular presentation dinners across the country.
"The benefit to the firms is that, for the price of completing the survey, they get a comprehensive management tool," Mark explained. "The firms use it in many different ways, such as for internal benchmarking or as an agenda builder for the financial director in the practice."
The news media love polls, and having a topical and authoritative benchmarking survey is a reliable way to achieve the kind of coverage that money cannot buy.
Key findings are also released to the press. Last year, these appeared as news stories in all of the UK’s quality daily newspapers and much of the business press too. The news media love polls, and having a topical and authoritative benchmarking survey, like the Financial Management in Law Firms report, is a reliable way to achieve the kind of coverage that money cannot buy.
Case Study 2: The Association of Leading Visitor Attractions
ALVA, the Association of Leading Visitor Attractions (http://www.alva.org.uk), represents many of the UK’s most popular and well-known tourist destinations. The qualification for entry is that the organization must welcome more than one million visitors annually. Members include The British Museum, Madame Tussauds’, the castles, stately homes and landscapes of The National Trust and English Heritage, Canterbury Cathedral, York Minster, The Tate Gallery and even one of the UK’s newest major attractions, The Eden Project: all magnet destinations for visitors to Britain, if not the reason for visiting Britain. Some years ago, ALVA’s membership realized that benchmarking would provide each of them with a valuable management tool. ALVA set about defining a benchmarking exercise, and a lasting relationship was forged with the Economic Research Association (ERA) and Snap SurveyShop, to bring this about.
"It takes a lot of work to create the definitions and ours, which we guard jealously, have been built up gradually over the years," said ALVA’s Director, Robin Broke. "You need very clear definitions, otherwise people will be giving you the wrong answers."
It is now an established annual process. Once the questionnaire is agreed between ERA and ALVA, it is sent to SurveyShop with a list of the participating sites. SurveyShop then distribute the questionnaires to the different sites, in order to maintain confidentiality, as the questionnaires are despatched with identifying data on them. Completed forms are sent back to SurveyShop, who also follow up with sites that are late submitting, or who provide incomplete or illogical data.
The results are sent to ERA as series of means, medians, quartiles, maxima and minima for each benchmark, who then produce a detailed report which also contains valuable explanation and commentary.
In addition, SurveyShop sends an individualized report to each participating member site showing their own score for each benchmark. This report was a later refinement, because members were finding it difficult to equate their performance against the benchmarks, as there are a number of calculations involved in arriving at the benchmarks from the raw data provided.
For ALVA’s Robin Broke, SurveyShop’s role is crucial: "By acting as a firewall it means only SurveyShop know the answers to the individual sites. They have the sort of employees that are very alert to their role in terms of confidentiality. But it does mean it is quite a complicated process. We have a lot of interface with them and find them very helpful indeed. That is also important when you use them as agents to get answers to any anomalies in the data, as we do. It is not just a mechanical process and you need sensible human beings that help to resolve the anomalies."
Any site attracting in excess of 100,00 visitors a year is eligible to participate.
Now in its seventh year of operation, the survey presents data on 35 different benchmarks from 65 visitor sites and includes several non-member sites. It is not only ALVA’s members that can participate: any site attracting in excess of 100,00 visitors a year is eligible to participate.
"The members and participants get massive benefit," Robin observed. "And the association gets kudos in running the only seriously credible benchmarking exercise for large tourist sites. The point of benchmarking is that it helps managers to know where the problems are, directs them to find out why they are not doing so well in some areas and helps them know where to put the management spotlight for the next year."
How SurveyShop can help
As we have seen, effective benchmarking studies depend on building the trust of participants by using a neutral intermediary. It is a role that SurveyShop is experienced in fulfilling, offering all of these major advantages:
- Survey forms can be sent directly to Snap Surveys using its postal reply services, resolving the survey logistics.
- The SurveyShop name can be used on the envelopes, if desired, emphasizing independence and professionalism.
- Fast and accurate data entry, with options for optical scanning, ensure reliable results.
- Alternatively, surveys can be completed online using SurveyShop’s Internet survey services.
- Flexibility of output means that results can be presented in aggregate form as tables, charts or even PowerPoint slides, on paper or electronically.
- Customized reports can also be sent directly to survey participants, showing their own individual scores against the norms for their group.
- Additional requests for analysis can be handled quickly and efficiently, in confidence, and delivered electronically, if required.
- Experienced staff are on hand to anticipate and resolve problems intelligently, when they occur.
Overall, by circulating the finding throughout the entire organization, it means that everyone can continue to learn and improve, through better knowledge and understanding.