Instructional Design Evaluations Analysis Strategies About Us Contact Us


What is evaluation?
Evaluation is a process of making decisions about the overall worth or value of something. Frequently, measurement is the means of collecting relevant data in order to make those decisions. People, groups, and even the “organization” may want or need to make value judgments about the worth or value of your work.

“… that part of Human Performance Technology (HPT) that provides information about worth or value or meaning (Reynolds, 1993) in order to guide decision making… Evaluation is the process of collecting information and feeding it back to those who need the information so that the system can succeed…”
(Shrock, S.A. and G.L. Geis, Evaluation, in Handbook of Human Performance Technology, H.D. Stoltovitch and E.J. Keeps, Editors. 1999, Jossey-Bass: San Francisco, CA., page 185)

ID.E.A.S That Work, LLC specializes in creating and executing effective evaluation strategies and measurement processes. Specifically, we can provide the following components of a useful evaluation process:

  • Facilitate development of an evaluation strategy for organizational stakeholders.
  • Craft efficient measurement strategies and processes to collect appropriate data.
  • Conduct formative evaluation (continuous improvement) of materials, courses, or facilitators.
  • Design and conduct pre-test / post-test comparisons for skills.
  • Develop and administer testing to certify worker competency.
  • Design and develop a rigorous process for measuring workplace application of learning.

Key Steps [Back to Top]

An effective evaluation process distinguishes evaluation (the process of establishing worth or value) from effectiveness (comparison of actual to intended outcomes). As such, this necessitates two phases of work:

  • Phase 1 articulates what the organization needs in order to make a value judgment about HRD activities.
  • Phase 2 sets forth a specific plan to capture or “measure” only that data the organization needs. (NOTE: the term “needs” will be used to mean both “wants” and “needs.” There may be real and material differences in what these terms mean in any given context, but the one term will be used here simply for convenience.)

Phase 1: Develop an HRD Evaluation Strategy
To articulate what the organization needs in order to make a value judgment about HRD activities, gather information about the following topics. Ask questions, review existing evidence, and look for organizational clues.

  1. Document the current evaluation practice and its perceived effectiveness.
  2. Investigate what the organization and its stakeholders really need to make value judgments.
  3. Draft a document explaining the organization’s needs for evaluation. Include all significant stakeholder groups, including HRD.
  4. Circulate the draft for discussion and consensus about its accuracy and relevance. Revise as needed.

The final output of Phase 1 is a document, accepted by the stakeholder group, called the Evaluation Strategy. It should articulate the stakeholders and their needs, and it should provide strong and clear direction for the types and frequency of specific measures. Note that the strategy document is specific for whatever scope was defined, and that complex organizations may have several interdependent or independent evaluation strategies.

Phase 2: Develop a Measurement Plan
The Measurement Plan follows directly from the Evaluation Plan. It sets forth the specific data the organization needs in order to demonstrate the effectiveness of specific HRD activities.

  1. Review current and common organizational methods for collecting and reporting data.
  2. Given the evaluation strategy, identify key indicators and corresponding measures. Don’t over-complicate the indicators or the measures!
  3. Draft the measurement plan (or plans, if different groups of stakeholders need different measures).
  4. Circulate the measurement plan and gain consensus.
  5. Develop a report format.

Advantages [Back to Top]
This systematic approach to designing and executing an effective evaluation plan offers several important advantages. It

  • Aligns HRD evaluation with strategic processes of the organization.
  • Increases the likelihood that HRD will be viewed as an integral part of the business system.
  • Provides better efficiency.

Resources [Back to Top]

White Paper [Back to Top]
©Each document is the exclusive property of Kathy L. Dye, Ph.D. and/or ID.E.A.S That Work, LLC and is protected by US copyright law.

Abstract: Dye’s Two-Tier Framework of Evaluation
Most approaches to evaluating HRD activities, including the beloved Kirkpatrick 4-levels, confuse the terms “evaluation” and “evaluating effectiveness.” This often leads to simplistic practices that fail to give the organization (including HRD practitioners) the information they need to make better-informed decisions. This white paper offers a description of 7 serious problems embedded in the traditional approach to evaluation and offers a more relevant and useful approach. The white paper also provides an example of how this approach could be applied to Big Bank, providing more meaningful information with less investment in evaluation.

Would you like to view this white paper in Acrobat Reader?

Acrobat Reader is necessary to view this file, click here if you do not have the reader.

ID.E.A.S That Work, LLC  •  P.O. Box 175827  •  Covington, KY  41017-5827  •  Contact Us