Which of the following should not be a criterion for a good research project

Research is one of the means by which we seek to discover the truth. It is based upon the tacit assumption that the world is a cosmos whose happenings have causes and are controlled by forces and relationships that can be expressed as laws and principles. Discovery of these controls of nature provides us with a hunting license to search for ways of controlling our environment.

Unfortunately, the research methods available to us are far from perfect. Mankind, in the slow evolution toward civilization, has to date found only three basic approaches to the discovery of truth-authority, logic, and controlled observation. Until about 300 years ago, efforts to discover the truth largely depended upon recourse to authority and logic. Since that time researchers have placed increasing emphasis upon controlled observation supported by the logic of mathematics.

Many of the natural phenomena that we would like to study do not yield readily, however, to any combination of the three known research methods. For some aspects of the truth there are no earthly authorities, only inadequate kinds of mathematical logic, and no known means of controlled observation. True, the extension and refinement of research methods are steadily enlarging the scope of research. However, the closer we come to discovery of a part of the truth, the greater the scope of the unknown seems to become.

Research rarely reveals to us the whole truth about anything.

Because of these limitations, research rarely reveals to us the whole truth about anything. Discovery of what seems to be the truth concerning the nature of some part of the process of learning often turns out later to be only a crude approximation of the truth. Discovery of a new teaching technique that seems at first to have universal application usually turns out later to have only restricted use. It seems wiser, therefore, to view research findings as a means of approximating the truth instead of revealing the ultimate truth.

How closely the results of a particular piece of research approximate the truth depends upon its quality. Since research efforts vary widely in quality, the question of how much confidence can be placed justifiably in the findings of a particular research is one of considerable importance. The main purpose of this article is to suggest a few common-sense criteria that may assist practical schoolmen to distinguish between research that merits a good deal of confidence and research whose findings should only be accepted with reservations.

1. The purpose of the research, or the problem involved, should be clearly defined and sharply delineated in terms as unambiguous as possible.
   The statement of the research problem should include analysis into its simplest elements, its scope and limitations, and precise specifications of the meanings of all words significant to the research. Failure of the researcher to do this adequately may raise legitimate doubts in the minds of readers as to whether the researcher has sufficient understanding of the problem to make a sound attack upon it.
2. The research procedures used should be described in sufficient detail to permit another researcher to repeat the research.
   Excepting when secrecy is imposed in the national interest, research reports should reveal with candor the sources of data and the means by which they were obtained. Omission of significant procedural details makes it difficult or impossible to estimate the validity and reliability of the data and justifiably weakens the confidence of the reader in the research.
3. The procedural design of the research should be carefully planned to yield results that are as objective as possible.
   When a sampling of a population is involved, the report should include evidence concerning the degree of representativeness of the sample. A questionnaire ought not to be used when more reliable evidence is available from documentary sources or by direct observation. Bibliographic searches should be as thorough and complete as possible. Experiments should have satisfactory controls. Direct observations should be recorded in writing as soon as possible after the event. Efforts should be made to minimize the influence of personal bias in selecting and recording data.
4. The researcher should report, with complete frankness, flaws in the procedural design and estimate their effect upon the findings.
   There are very few perfect research designs. Some of the imperfections may have little effect upon the validity and reliability of the data; others may invalidate them entirely. A competent researcher should be sensitive to the effects of imperfect design and his experience in analyzing the data should give him a basis for estimating their influence.
5. Analysis of the data should be sufficiently adequate to reveal its significance; and the methods of analysis used should be appropriate.
   The extent to which this criterion is met is frequently a good measure of the competence of the researcher. Twenty years of experience in guiding the research of graduate students leads the writer to conclude that adequate analysis of the data is the most difficult phase of research for the novice.
   The validity and reliability of data should be checked carefully. The data should be classified in ways that assist the researcher to reach pertinent conclusions. When statistical methods are used, the probability of error should be estimated and the criteria of statistical significance applied.
6. Conclusions should be confined to those justified by the data of the research and limited to those for which the data provides an adequate basis.
   Researchers are often tempted to broaden the bases of inductions by including personal experiences not subject to the controls under which the research data were gathered. This tends to decrease the objectivity of the research and weaken confidence in the findings.
   Equally undesirable is the all-too-frequent practice of drawing conclusions from study of a limited population and applying them universally. Good researchers specify the conditions under which their conclusions seem to be valid. Failure to do so justifiably weakens confidence in the research.
7. Greater confidence in the research is warranted if the researcher is experienced, has a good reputation in research, and is a person of integrity.
   Were it possible for the reader of a research report to obtain sufficient information about the researcher, this criterion perhaps would be one of the best bases for judging the degree of confidence a piece of research warrants. For this reason, the research report should be accompanied by more information about the qualifications of the researcher than is the usual practice.

Some evidence pertinent to estimates of the competence and integrity of the researcher may be found in the report itself. Language that is restrained, clear, and precise; assertions that are carefully drawn and hedged with appropriate reservations; and an apparent effort to achieve maximum objectivity tends to leave a favorable impression of the researcher. On the other hand, generalizations that outrun the evidence upon which they are based, exaggerations, and unnecessary verbiage tend to leave an unfavorable impression.

Of course, the seven criteria listed above are not the only earmarks of research worthy of confidence. A relatively complete list would be much more extensive. However, it is hoped that the brief list will be of some assistance in helping readers of research reports to assess their worth. Obviously, such assessments must be largely subjective, since the criteria are couched in relative terms. The writer is unaware of any present means of categorically classifying the worth of research objectively. If, however, the criteria help to make readers more constructively critical of reported findings of research, they will have served their purpose.

This article was originally published in Phi Delta Kappan, 39 (6), 284.286.

JAMES HAROLD FOX was dean of the School of Education, The George Washington University. Since 1946 he has taught a graduate course in educational research methods and procedures. He was a member of the Phi Delta Kappa Research Commission.

What are the criterion for a good research project?

Which of the following should not be a criteria for good research project?

Which of the following is not a criterion for the statement of a good research problem?

Which of the following is a criterion for a good research question *?