minority issues in evaluation:
survey
 


In a recent series of discussions on the listserv, goveval, Michael Scriven of Claremont Graduate University, proposed the following definitions of "research" and "evaluation:"
Research:   any systematic and skilled investigation, done in the cognitive domain, of the existence and nature of things. (Or, the results of such investigations.)

Evaluation:   systematic and skilled investigation, done in the cognitive domain, of the merit, worth, or significance of things. (Or, the results of such an investigation.)

Scriven went on to add:

The phrase "in the cognitive domain" means that the conclusions are stateable and the process can be justified by reasoning. It rules out tracking game, for example, done by a skilled tracker; or the judgment of skaters by professional judges.

Either may be done in the field or in a lab, library, or office. "Things" includes processes and procedures.

These are the versions that will currently be in the 5th edition of the Evaluation Thesaurus and hence I would much appreciate comments to the contrary, so that I can modify the draft.

As might be expected, a lively discussion ensued! Many suggestions and comments were offered. Given the nature of a listserv, it has been difficult to gain a sense of consensus on these definitions. As part of its efforts to highlight important issues in evaluation, the Minority Issues in Evaluation TIG has constructed the following survey — intended to quantify the degree to which participants agree or disagee with the proposed definitions.

Survey Results

The survey results are in:   23 people rated the two definitions; the results are as follows (first, the average ratings and then histograms of the actual ratings):

1 = strongly disagree     —     strongly agree = 9

Evaluation: Average rating = 6.26  
Research: Average rating = 6.35  


  7              
  6              
  5              
  4              
  3              
  2              
  1                
    3     4     5     6     7     8     9  
Evaluation
   
  9              
  8              
  7              
  6              
  5              
  4              
  3              
  2              
  1                
    3     4     5     6     7     8     9  
Research


As might be expected, given the discussion on the listservs, respondents were divided in terms of their ratings of each of the two definitions. Again, this straw poll wasn't intended to be a "scientific" poll. Also, no attempt has been made to analyze the responses. It would appear, however, that the majority of the responses were from people who hadn't posted an opinion on the listservs, although this observation wasn't thoroughly explored.



mission   |   aea '98   |   aea '96 & '97   |   what's new?   |   job bank
hot stuff!   |   links   |   listservs   |   miscellaneous   |   home


produced and maintained by obsidian communications. questions & comments: ward bell, webmaster