What is evaluation?
Evaluation is a process of making decisions about the overall worth
or value of something. Frequently, measurement is the means of collecting
relevant data in order to make those decisions. People, groups,
and even the “organization” may want or need to make
value judgments about the worth or value of your work.
“… that part of Human Performance Technology
(HPT) that provides information about worth or value or meaning
(Reynolds, 1993) in order to guide decision making… Evaluation
is the process of collecting information and feeding it back to
those who need the information so that the system can succeed…”
(Shrock, S.A. and G.L. Geis, Evaluation, in Handbook of Human
Performance Technology, H.D. Stoltovitch and E.J. Keeps, Editors.
1999, Jossey-Bass: San Francisco, CA., page 185)
ID.E.A.S That Work, LLC specializes in creating
and executing effective evaluation strategies and measurement processes.
Specifically, we can provide the following components of a useful
- Facilitate development of an evaluation
strategy for organizational stakeholders.
- Craft efficient measurement strategies and
processes to collect appropriate data.
- Conduct formative evaluation (continuous
improvement) of materials, courses, or facilitators.
- Design and conduct pre-test / post-test
comparisons for skills.
- Develop and administer testing to certify
- Design and develop a rigorous process for
measuring workplace application of learning.
[Back to Top]
An effective evaluation process distinguishes evaluation
(the process of establishing worth or value) from effectiveness
(comparison of actual to intended outcomes). As such, this necessitates
two phases of work:
- Phase 1 articulates what the organization needs
in order to make a value judgment about HRD activities.
- Phase 2 sets forth a specific plan to capture
or “measure” only that data the organization needs.
(NOTE: the term “needs” will be used to mean both
“wants” and “needs.” There may be real
and material differences in what these terms mean in any given
context, but the one term will be used here simply for convenience.)
Phase 1: Develop an HRD Evaluation Strategy
To articulate what the organization needs in
order to make a value judgment about HRD activities, gather information
about the following topics. Ask questions, review existing evidence,
and look for organizational clues.
- Document the current evaluation practice and its
- Investigate what the organization and its stakeholders
really need to make value judgments.
- Draft a document explaining the organization’s
needs for evaluation. Include all significant stakeholder groups,
- Circulate the draft for discussion and consensus
about its accuracy and relevance. Revise as needed.
The final output of Phase 1 is a document, accepted
by the stakeholder group, called the Evaluation Strategy. It should
articulate the stakeholders and their needs, and it should provide
strong and clear direction for the types and frequency of specific
measures. Note that the strategy document is specific for whatever
scope was defined, and that complex organizations may have several
interdependent or independent evaluation strategies.
Phase 2: Develop a Measurement Plan
The Measurement Plan follows directly from the Evaluation
Plan. It sets forth the specific data the organization needs in
order to demonstrate the effectiveness of specific HRD activities.
- Review current and common
organizational methods for collecting and reporting data.
- Given the evaluation strategy,
identify key indicators and corresponding measures. Don’t
over-complicate the indicators or the measures!
- Draft the measurement plan
(or plans, if different groups of stakeholders need different
- Circulate the measurement
plan and gain consensus.
- Develop a report format.
[Back to Top]
This systematic approach to designing and executing
an effective evaluation plan offers several important advantages.
- Aligns HRD evaluation with strategic processes
of the organization.
- Increases the likelihood that HRD will be
viewed as an integral part of the business system.
- Provides better efficiency.
[Back to Top]
[Back to Top]
document is the exclusive property of Kathy L. Dye, Ph.D. and/or
ID.E.A.S That Work, LLC and is protected by US copyright law.
Abstract: Dye’s Two-Tier Framework of Evaluation
Most approaches to evaluating HRD activities, including the beloved
Kirkpatrick 4-levels, confuse the terms “evaluation”
and “evaluating effectiveness.” This often leads to
simplistic practices that fail to give the organization (including
HRD practitioners) the information they need to make better-informed
decisions. This white paper offers a description of 7 serious problems
embedded in the traditional approach to evaluation and offers a
more relevant and useful approach. The white paper also provides
an example of how this approach could be applied to Big Bank, providing
more meaningful information with less investment in evaluation.
Would you like
to view this white paper in Acrobat Reader?
Acrobat Reader is necessary to
view this file, click
here if you do not have the reader.