Survey Abandonment is like Death to a Researcher - Upshot: Put High-Abandonment Questions at the End

Have you ever noticed that more people start your online surveys than finish them? It’s like they start clicking through the answers and then suddenly they have a heart attack and it’s curtains!

What is happening to those hapless survey takers?

Life insurance actuaries, the Census Bureau, and the Social Security Administration use a tool to understand mortality: life tables. Life tables measure the odds of dying at a particular age. Age 0 to 1 has a higher risk of death than age 1 to 5. Past age 5, our odds of death steadily climb.

Life Table

One might assume that a similar pattern would be true for online surveys, and that the odds of abandoning a survey increase with each increasing page. So, I decided to check it out. Do survey abandonment curves match similar patterns to human mortality tables? I used one of Allegiance’s client surveys to examine the trends.

In the chart below I used one axis to plot the number of survey takers who had completed each page. On the secondary axis, the red line, I plotted the odds of abandonment. The odds of abandonment is the number of survey takers lost to page x+1 divided by the survey takers remaining at page x.

Abandonment

Turns out, survey abandonment curves are not quite as smooth as human mortality curves. Survey abandonment curves are spiky. It’s like the Grim Reaper inhabits certain pages but not others.

Clearly, one can see where the problem pages are, where abandonment spikes. It turns out that this survey has a bunch of open-ended questions on pages 2, 3 and 10. Answering open-ended questions is a lot of work. Rather than work, people abandon the survey altogether.

There is no way this client will get rid of their open-ended questions. There is nothing I can do to make the questions less prone to abandonment. However, what we can do is re-order the pages and put the low-abandonment pages at the front of the survey and the high-abandonment pages at the back. The thought is that with more survey takers sticking around for longer, we may get to gather more data.

We can measure the increased data the same way the census bureau measures “life-years.” I’ll call this measurement “page-completes.” If we accumulate more page-completes by the end of the survey then we’ve made an improvement. Below is a simulation of re-ordering the pages to put the high-abandonment pages at the end.

Abandonment Improvement

Sure, the same number of respondents made it to the end of the survey, 1910. However by re-ordering the survey, the page-completes grow from 28,828 before to 32,319 after. Re-ordering may help me capture more survey data overall.

Kyle LaMalfa, Best Practices Manager and Loyalty Expert, Allegiance

Share/Save/Bookmark

Observations from BAI 2008: Banking, Innovation and the Current Leadership Gap

There is a real sense of fear in the banking industry right now, and at the 2008 BAI Retail Delivery Conference and Expo, a conference at which we just exhibited last week, that fear was almost palpable.

While in any other year the following statement would have encouraged prospective attendees to pay the registration fee without a second thought, the current climate in the banking and finance industry ensured that this year’s BAI Expo would have a rough go of it:

“In these critical times, rarely has it been so important to learn successful strategies for controlling costs and exploring new ideas for staying profitable in the future - using innovative strategies to get the most out of existing resources and ideas to prioritize for the future to ensure greater profitability and performance.”

In times of crisis it is typical for most companies, especially banks to take on a bunker mentality. There was strong evidence at BAI that banks are already doing this, as this year’s BAI show had roughly half the attendees as last year. Unfortunately, this mentality can do more harm than good. Banks don’t need to stop spending; they need to start spending smarter. There are solutions to the serious problems that banks face (i.e. solutions such as those offered by Allegiance) that can add immediate value to banks with a minimal investment, if banks will invest in them.

The good news is that this year’s BAI theme “Return on Innovation” was spot on. It took me a moment when I first saw it because I had been reading it as “Return to Innovation”. I don’t doubt that there is a good amount of out-of-the-box thinking going on right now as organizations scramble to stay relevant in a market where the rules are changing every day. There seems to be a trend right now for banks to do everything they can to innovate their way out of the crisis. But the problem isn’t innovation…. the problem is quantifying it. Are you seeing a return on investment? Innovation for the sake of innovation means nothing unless there are solid numbers to back it up. Banks are one of the most customer-centric industries that exist. Almost everything a bank does is transactional. If there is a way for you to show, in real dollars and cents that keeping even one percent of your customers would mean a significant increase in revenue would you pursue it? Of course you would. We all would.

One of the highlights of the conference was the keynote address by former Secretary of State General Colin Powell. He spoke at length about the need for leadership that promotes good ideas, that rewards creativity, but that is also grounded in common sense. It wasn’t until he was finished and the audience stood to applaud that I saw how starved people were for real leadership. Both people sitting next to me had tears streaming down their face, and as I walked out I noticed quite a few more red and puffy eyes. So, when it comes to the banking industry, there is a tremendous opportunity for banks to innovate and continue to narrow the leadership gap. And if they do that, not only will their own bank be better off, but so will the banking industry.

Jordan Schelin, Marketing Manager, Allegiance

Share/Save/Bookmark

Re-evaluating Business Standards and Practices in the Wake of the Wall Street Meltdown

One of the most important lessons that Enron, WorldCom and the recent financial/Wall Street meltdown has taught us is just how damaging unethical behavior can be to a company, and how important it is for businesses today to establish ethical standards in the workplace as well as support those who do come forward to report unethical behavior.

For instance, the Ethics Resource Center’s (ERC) 2007 National Business Ethics Survey found that over the past year, more than half (56 percent) of employees surveyed had personally observed violations of company ethics standards, policy or the law. Many saw multiple violations. And, unfortunately, more than two of five employees (42 percent) who witnessed misconduct did not report it through any company channels.”

Part of the reason for this is that many employees are not comfortable reporting bad behavior for fear of potentially losing their jobs, being considered a “snitch”, and/or fear of retaliation from the guilty party.

However, the good news is that more and more companies are beginning to recognize this, as one of the positive things we’ve seen recently is a growing desire among businesses to show their responsibility and adopt best practices, as well as ensure stricter ethical standards and practices in their workplace. In doing so, these businesses are not only looking for programs to help them establish and promote ethical standards (or are re-evaluating their existing programs), but are also looking for different ways to better assist and support employees in coming forward to report unethical behavior. And hopefully, this will help prevent a lot of unethical behavior from happening in the future.

Clay Osborne, Director SilentWhistle Sales, Allegiance

Share/Save/Bookmark