Feedback from Conjoint Interviews
Satisfies Respondents' Intrinsic Motivation
Sam Kingsley, Action Research NY
readers may be old enough to remember a time when consumers
were flattered to be stopped in a shopping mall and asked
their opinions. Or to be called on the phone -- completion
rates of 60% or more were attainable. No more. Why not?
Consider the following intrinsic
motivations for participating in a survey:
can get feedback about themselves, individually and in how
they stand compared to others.
- Influence and control—Respondents'
views of the world are understood and, their opinions valued.
respondents have a chance to learn about new products or
services being developed.
With two exceptions, current
practice in market research doesn't meet these needs of respondents.
One exception is well-designed customer satisfaction research,
which can provide respondents a sense of influence and control
and generate high levels of cooperation.
The second exception is online
research, particularly conjoint, which provides a special
opportunity to meet respondents' intrinsic needs.
- In conjoint, respondents are
asked to make thought-provoking tradeoffs based on what
they value. The answers are sometimes surprising to respondents,
and the final results aren't always obvious.
- Because individual level conjoint
results can be revealing, there isn't a need to provide
normative data, which may be difficult in some competitive
- In Adaptive Conjoint Analysis,
respondents' answers are processed as they are given, so
that individual-level results are automatically available
at the end of the interview.
The potential power
of giving feedback to respondents can be seen in a recent
application of conjoint to matching candidates with job openings.
Futurestep, a subsidiary of Korn/Ferry International, is an
online matching service that uses direct questions and adaptive
conjoint to profile potential job candidates. Candidates,
including those who may not be actively looking for career
changes, register with Futurestep because of the personal
information and feedback they receive.
Futurestep was the first company
to use ConjointOnline, a Web adaptation of the DOS interviewing
module of Adaptive Conjoint Analysis (ACA), which was developed
by Sawtooth Software, Inc. Over 70,000 ConjointOnline interviews
have been completed to date.
To recruit a base of candidates,
Futurestep launched a nation-wide radio and print advertising
campaign. The Wall Street Journal is an important partner
with Futurestep because of its recognition among potential
candidates and clients. Within a month there were over 50,000
registrants. These individuals are generally mid-career managers
In addition to the conjoint data,
Futurestep uses a battery of validated self-assessment instruments
to better match personal preferences in decision-making and
managerial styles. This data is matched with information provided
by hiring managers and potential peers of the candidate who
fill out a set of mirror instruments prior to the search.
Candidates come to Futurestep
because they receive: complimentary career and market value
feedback; guaranteed confidentiality of candidate's information;
the credibility of Korn/Ferry International and the Wall
Street Journal, non-advertised positions from blue chip
organizations as well as smaller, high-growth firms.
Some of the feedback from candidates
illustrates their level of involvement with the process.
"This whole process ...
provides an opportunity to establish a personal relationship
in a high-tech world. The business world rarely sees technology
as an enabler and technology rarely sees beyond its owns
bits & bytes."
"Thank you! What a great
way to get to know your business style. The more I know,
the better I can communicate my needs in a job interview."
"The characteristics described
above are the person I am. I did not believe the questions
asked could reveal so much. This indeed is very interesting
and exciting. Thank you."
"I really enjoyed the exercise
- I think the results are right on and consistent with other
profiling techniques I have experienced. In this case, I
think the attributes in the "job characteristics" report
were well chosen - had to be via focus groups." [Editor:
No focus groups.]
"Absolutely amazing! You
have drawn a very accurate picture of my personality, characteristics
and personal goals."
that educate and inform respondents about themselves or their
world can be motivating to many people, as the comments above
illustrate. Our firm's experience in online research with
doctors shows that once respondents agree to participate,
85% are willing to complete short surveys with little notice.
Many of the doctors comment about learning of developments
in their field via the surveys.
Comments made by respondents
indicate many are highly motivated to give honest, complete
self-portraits -- if only the questions and answer choices
would let them! This is particularly true in conjoint, because
the questions require some thoughtful reflection by respondents
on what they really value. Respondents hold Web surveys to
high standards -- this is no place for sloppy thinking, inaccurate
wording or erroneous analysis.
In ConjointOnline, respondents
are given the opportunity to review their final profile, and
to comment on whether it matches their self-perceptions. The
addition of real-time checking for inconsistent responses
(i.e., where a response might vary significantly from what
was predicted) further strengthens the bond between the respondent
and the survey process. There actually is an intelligence
in the machine!
Being able to start and complete
a task at one time is a positive factor in motivation. From
the respondent's perspective, the task isn't complete until
the answers have been correctly registered and implications
To return to the original
question, well-executed online research indicates that cooperation
levels can be improved. In many surveys there is little indication
that the research recognizes that true communication is a
As long as our profession tries
to pump more data out of people, those being pumped will increasingly
view surveys as an economic transaction, for which they should
be compensated. When there is true two-way communication,
respondents' intrinsic needs are met. The implicit contract
with survey respondents is reaffirmed, leading to a more satisfactory
relationship for both researchers and respondents.