widespread availability of Web access has led to re-evaluation
of traditional relationships between companies and their
customers. Industries which have a heavy information component,
such as news and financial services, are experiencing
large shifts to online delivery.
business logic of online job search and recruiting for
middle-management and professional positions is that the
Internet allows Futurestep to use sophisticated assessment
methodologies effectively with a large database of management
candidates. Candidates, including those who may not be
actively looking for career changes, register with Futurestep
because of the personal information and feedback they
receive. Recruiting professionals manage the process with
employers, particularly the job specification step, preparation
of an offer and negotiation.
was the first company to use ACQNET, a Web adaptation
of Sawtooth Software's ACA interviewing module. Over 70,000
ACQNET interviews have been completed to date. In the
Futurestep process, direct interview questions are used
to identify likely candidates for potential jobs. Structured
questionnaires also assess decision-making and communication
styles, and career motivators. The ACA results are used
to identify potential candidates in the early stages of
a search by eliminating those who would be unlikely to
accept offers within the hiring company's set ranges:
Is this candidate ready to move? Which job factors are
the most important to this candidate? The ACA results
are also used in the final stages of a search to provide
a client company with information about how to tailor
an offer that appeals to a candidate's preferences.
the analysis of conjoint results is conducted at an individual
level, minor adjustments were necessary in the ACA interviewing
algorithm and analysis.
traditional or not, recruiting firms generally follow
three stages in conducting a search:
FINDING means sourcing a large pool of potential candidates.
It is usually done by industry Researchers. As much
as 50% of a firm's resources may be devoted to this
stage, which is repeated for each new search conducted.
ASSESSMENT of candidates means reviewing the pool and
understanding the top candidates beyond their objective
experience levels. This gives the recruiter a chance
to probe and test the way candidates handle themselves
in the context of the client's needs.
PLACEMENT of candidates includes articulating to both
the candidate and employer why this is a good match,
plus negotiating an offer.
job search and recruiting business has traditionally relied
on a "telephone tree" of Researchers to source
potential candidates for job openings. This concentrates
over half of the resources of a firm on simply obtaining
a list of potentially qualified candidates. In a typical
recruiting firm, dozens of Researchers may contact hundreds
of candidates by telephone for a given search. Many of
the contacts may be wasted, because they surface neither
a potential candidate nor a referral to someone else in
the telephone tree.
identify a large pool of potential candidates with qualifications
that match the job specifications. The particular skills
and experiences required for the job are assessed at this
in the traditional firm, Senior Associates or Partners
review the qualifications and select a small number, perhaps
2-6 candidates, for additional interviewing. This is a
time-consuming process that usually involves a personal
contact with each candidate.
one or more candidates are presented to the employer,
who will often interview several of the candidates. At
this stage, the recruiting firm seeks to solidify the
match: both sides should be convinced of the appeal. A
specific offer will be made, usually with input from the
recruiting firm. The inefficiency at this stage occurs
when there are protracted, iterative rounds of interviews
with client managers or rounds of offer negotiations and/or
a candidate ultimately declines the offer -- a "slow-no."
reduce the inefficiency of the Finding step, a large database
of qualified candidates was developed. Futurestep launched
a nation-wide radio and print advertising campaign. The
Wall Street Journal is an important partner with Futurestep
because of its recognition among potential candidates
and clients. Within a month there were over 50,000 registrants.
These individuals are generally mid-career managers and
do candidates come to Futurestep? Because they receive:
career and market value feedback
confidentiality of candidate's information
of Korn/Ferry International and the Wall Street Journal
for exclusive, non-advertised positions from blue chip
organizations as well as smaller, high-growth firms.
of the feedback from candidates illustrates their level
of involvement with the process.
whole process ... provides an opportunity to establish
a personal relationship in a high-tech world. The business
world rarely sees technology as an enabler and technology
rarely sees beyond its owns bits & bytes."
you! What a great way to get to know your business style.
The more I know, the better I can communicate my needs
in a job interview."
is an outstanding and leading edge tool for career development."
is part of a sea change in the business of searching for
jobs (stated from the employee's perspective) and hiring
(employer's perspective). It is changing because of the
amount of information available to candidates and employers.
As we have seen in other markets, such as home mortgages
and foreign exchange, when more information is made available
in a standardized format to both buyers and sellers, "liquidity"
improves due to a greater transparency in pricing. Transaction
costs decrease and there is an overall greater level of
as uncertainty is diminished due to greater transparency
of information, the decision process is shortened. Overall,
the effects on the employment marketplace may eventually
be as large as the introduction of employment advertising!
both the traditional and Futurestep approaches there is
a requirement to verify the information that has been
submitted and check that the candidate in-person matches
the description on paper. Futurestep uses a proprietary
video-conferencing system to permit recruiters to interview
potential candidates in their own homes. To keep the comparison
between candidates more consistent, the questions are
asked from a common script. In fact, the interviews are
recorded and can be used in future searches as well. Once
Futurestep has selected one or more candidates to present
to the employer, a computer CD is prepared that contains
the recorded assessment interviews. In addition to the
ACA data collected, Futurestep uses a battery of validated
self-assessment instruments to better match personal preferences
in decision-making and managerial styles. This data is
matched with information provided by hiring managers and
potential peers of the candidate who fill out a set of
"mirror" instruments prior to the search.
identified several potential candidates for a search --
usually 2-3 individuals -- the next challenge for the
recruiting firm is to market the job to the candidate,
and vice versa.
The wealth of standardized information available
to the recruiter and employer about each candidate's
approach to work environments gives the employer a clear
basis for selecting the top candidate.
Conjoint utilities provide the values that the
candidate places on the most important aspects of the
job. This helps the recruiter and employer in several
the utility curves helps recruiters understand what
characteristics of a job might be very attractive,
or are potential sticking points.
calibration concept results, the recruiter gets an
idea of the candidate's willingness to move for any
job. Is this a "tough nut" or an "easy
sell"? While the really tough nuts (those with
little likelihood of moving) have already been selected
out, there may be some borderline candidates left
in the pool because of their clearly superior abilities.
a given job may represent different levels to different
candidates, a tailored "job spec" must be
created for each candidate in order to calculate likelihood
of accepting the job. At present, this is a manual
process that is only used for the small number of
finalists in a search. An input screen for Job X for
Candidate Y has been designed to permit the recruiter
to estimate what the job represents to that candidate,
and then to adjust the input values to see what's
driving the job's appeal: what ifs.
attributes contain both annual compensation and wealth
creation (stock or options). Therefore, it is possible
to tailor an offer by shifting the salary/wealth balance
to maximize the value to the candidate, within parameters
established by the employer.
conjoint results also lets the recruiter highlight to
the candidate those factors of the job that would be most
important to him/her, rather than spending time on unimportant
another feature of having the conjoint data collected
in advance is that candidates' answers aren't tailored
to their perceptions of a particular job opening. It serves
as a natural check on the enthusiasm with which candidates,
and sometimes recruiters, approach a search.
combined with the other structured questionnaires, it
gives the candidate and recruiter a richer language with
which to discuss a particular job as well as the candidate's
needs and values. Since this type of standardized data
collection is not typical in the recruitment industry,
training for Futurestep recruiters has been built into
Futurestep conjoint design consists of 11 attributes with
a total of 40 levels. The interview itself asks for respondent
input just as it would in ACA Version 4 for DOS:
for one attribute, all preference rankings are set a
Each attribute is rated on importance using Sawtooth's
suggested 4-point scale.
15 pairs are presented, 10 with 2 attributes and 5 with
3 attributes each.
4 calibration concepts are used.
from candidates who completed the conjoint portion show
that they are very involved in this part of the process,
and generally find it an accurate reflection of how they
would make choices. There is little drop-off in participation
at this stage. The conjoint is the last portion of the
registration, so candidates have generally invested over
45 minutes up to this point. Although exact statistics
aren't available, over 90% of respondents who get to the
end of the prior section complete the conjoint part too.
comments below highlight the need for more complete explanations
of the conjoint process than might be required in a "normal"
market research interview.
characteristics described above are the person I am.
I did not believe the questions asked could reveal so
much. This indeed is very interesting and exciting.
for the Desired Job Characteristics - a good reflection
in my case EXCEPT: major event = too high and making
an impact = too low; probably more like 8 and 8 respectively.
was surprising and revealing to me that my career questionnaire
feedback was in accordance with my career experiences
to date. I was very impressed that my job characteristics
(as defined by your assessment) is almost a perfect
match with the job that I am doing. It was also heartening
to read about the job characteristics of positions that
I am applying for actually dovetail with my desired
job characteristics. As a result, I would say that your
evaluations are 'on the money.' Thank you very much."
essentially agree with the results and explanations
associated with each area except in the section "Desired
Job Characteristics." I am disappointed in that
my responses indicate that I favor personal wealth twice
as much as making an impact. Although achieving wealth
is important, I believe that it should follow making
a positive impact, especially in a new position. I guess
I didn't answer the questions with this in mind. Is
it possible to change my responses in order to make
this point? I found this to be a very enlightening exercise."
pleased. The analysis "felt" right on target.
It also brought up a career option/class that I didn't
know existed, which matched really well with the characteristics
of my ideal job."
really enjoyed the exercise - I think the results are
right on and consistent with other profiling techniques
I have experienced. In this case, I think the attributes
in the "job characteristics" report were well
chosen - had to be via focus groups." [Editor:
No focus groups.]
have some [Job Characteristics] that are different by
only 1% or 2%. Are these statistically significant differences?"
would like to comment on the "relocation"
weighting in the job characteristics section. Because
I live within five miles of Chicago, and am willing
to work at locations within a 20 mile radius of one
of the major employment centers in the country, my reluctance
to relocate should not be weighted in the same way as
someone who lives in a much less urban area. Does your
analysis take geographic location of the respondent
into consideration?" [Editor: Yes it does.]
somewhat disagree with results of Desired Job Characteristics.
That may be how my answers fell out, so to speak, however
if I obtain and successfully perform a job that provides
more opportunity and responsibility, I, personally,
would only assume that I will impact the performance
results and the bottom line, in a positive manner, significantly."
amazing! You have drawn a very accurate picture of my
personality, characteristics and personal goals."
was a pleasant experience registering to Korn/Ferry's
Futurestep site on the Internet. I was specially impressed
by the immediate "Desired Job Characteristics"
feedback upon conclusion of the registration process."
In the DOS version of ACA, calculations can be made after
each entry by the respondent. Even on an original IBM
PC (8088), there is an insignificant delay as ACA does
ACQNET has avoided client-side execution, the results
of each page must be sent to the server for processing,
no matter how trivial. This imposes a transmission delay
for Web-based ACA of approximately 3 seconds. There are
two places in ACA which this affects:
the priors section, ACA uses either a preference rating
or ranking for levels within attributes. In the DOS
version, preference rankings are simplified for the
respondent by use of a disappearing list. After a choice
is made, that answer is removed from the screen. This
makes it possible to ask the respondent to rank order
a large number of levels.
To do this on the Web without client-side execution
would require a page to be submitted and returned for
each judgment. Instead, ACQNET implements preference
rankings by asking the respondent to put the rank order
(1 to n) into a text box. Rankings on all levels of
all attributes are obtained before the page is submitted.
While ACQNET allows the researcher to specify either
preference rankings or ratings, ratings are recommended
if there are more than 3 levels per attribute.
The pairs section presents a different kind of problem.
In DOS ACA, utilities are re-calculated after the respondent
enters a judgment on each pair, which can affect the
left-to-right balancing of the next pair. However, doing
re-calculations during the pairs section requires a
balance between the respondent's patience with delays
and the additional accuracy obtained if ACA does frequent
ACQNET addresses the need to re-calculate utilities
during the pairs without stretching the patience of
the respondent. It does so by presenting a variable
number of pairs on one Web page -- currently 3 at a
time. When the respondent submits a page of answers
to pairs, ACQNET re-calculates the utilities and selects
the next set of pairs to present.
of preferences from priors
There is another characteristic of ACA that was not particularly
suited to individual level analysis. Preferences for levels
within attributes are either set aprior (E.g., Higher
cost is always less appealing than lower cost) or asked
in the priors questions ("Which do you prefer?").
Answers to the pairs questions, however, can actually
lead to utilities that contradict the ordering of preferences
from the prior section. In the DOS version, this is permitted
and the final utilities sometimes contain reversals. Our
experience with aggregate-level analysis of ACA is that
such reversals have no measurable effect on simulation
individual level analysis, however, such reversals can
become confusing to respondents and analysts. When the
utilities for two levels are reversed in preference, ACA
will not properly balance a pair that contains those levels.
This means that the respondent could be presented with
the following type of choice:
comp: no change
Wealth creation: $0
comp: 20% increase
Wealth creation: $500K stock options
this situation, some respondents pause and ask of themselves
(or an interviewer, if present) "There's no trade-off
here. What should I do?"
preliminary analysis of two previously collected ACA data
sets was performed to determine how often a pair was presented
in which either the Left or Right side contained all the
preferred levels. In one ACA study there were practically
no reversals: 4 out of a total of 2,595 pairs presented
to 145 respondents contained no trade-off between left
and right. However in another study there were 102 non-trade-off
pairs out of 1,918 pairs judged by 100 respondents. In
other words, 5% of the pairs involved decisions where
there was no trade-off.
respondents seem to use the extreme scale values when
rating pairs where there is no trade-off. Out of the 106
reversals noted above, 74% of the pairs involved were
rated either 1, 2, 8 or 9. This may indicate that respondents
were rating their certainty of preference rather than
the strength of preference.
analysis is being considered to determine the situations
that are likely to produce preference reversals.
the Futurestep application, a decision was made that the
preference rank orders from the priors section were to
be accepted. Of the 11 Futurestep attributes, 10 were
rank ordered a priori (e.g., more salary is always preferred
to less). There was one attribute that required the respondent
to make a judgment: whether relocation to a more desirable
location was preferred to no relocation. For this application,
it was determined that the respondent's judgments of preferences
during the priors would be more accurate than preferences
derived from pairs.
contains an option to restrict the judgments in the pairs
section from reversing the ordering of preferences for
levels from the priors. If a reversal is found, the utilities
for the two levels are averaged, then the tie is broken
by adding .01 to the utility for the level that was preferred
in the priors.
isn't the answer for all businesses.
business employers are least likely to have well-developed
job specifications and are also quite cost sensitive.
Their adoption of this method for staffing would require
an increased level of formalization for the business as
well as lower cost delivery by the recruiting firm.
Futurestep model works well where there is a well-defined
skill set or product knowledge required. If a job requires
a high degree of personal fit, individualized assessment
or personal relationships for hiring, the standardized
Futurestep approach is unlikely to be adopted. Examples
would be employment by a movie studio, creative fields
such as design or advertising, or any job that requires
isn't for all candidates, but since online job searching
significantly expands a candidate's opportunities, it
is expected to be adopted with increasing frequency.
Candidates will still use multiple methods of job searching,
including networking, classifieds and Internet job postings.
Further development of tools and training for recruiters
in the "language of conjoint" will be required.
Currently under development with Futurestep is a method
to specify jobs in terms of the attributes and levels
for each potential candidate. This is starting as a
subjective process with a long-term goal of systematizing
this analytic step.
There is also an opportunity to better understand employer's
trade-offs for a candidate: what would drive a greater
dollar offer, or other benefits? What would make a candidate
exceptional -- such that an employer would willingly
pay more? ACA has the ability to make the dialogue between
recruiter and employer more systematic and less based
on assumptions. As conjoint has forced researchers to
be explicit about product attributes, so would employers
communicate their trade-offs with greater accuracy.
ahead for ACA on the Web? We see several trends:
... More access by researchers and non-researchers.
... Online design and testing of ACA via the Web is
already available. Prototyping and testing can be
... Humongous data sets can be developed, increasing
the opportunities for subgroup and segmentation analyses.
... When conjoint is used with individual feedback,
it places greater demands on researchers to provide
clear and meaningful explanations of individual conjoint