|
Approved
Practices In Reporting Quantitative Research
Journal of Vocational Education Research Fall, 1986 Volume 11, No. 4, Pages 1-24. by Gary
E. Moore, Michael F. Burnett, & Barbara A. Moore1
Guidelines that should be followed in reporting quantitative research are given. Common errors made in reporting research are discussed, along with specific suggestions on how to avoid the errors. Randy, a young assistant professor, just received his third manuscript rejection letter this year. "Why can't I get anything published?" he implored. He is in his third year at the university and the "publish or perish" clock is running. During his doctoral work, Randy was taught how to design and conduct research and how to perform statistical analyses but never how to report research in any way other than dissertation style. According to Boice and Jones (1984), this is a common problem in higher education. Professors are taught how to conduct research but not how to write and report research. There is a body of knowledge on how to report educational research but it exists in a piecemeal fashion. There is a clear need for a set of approved practices for reporting research in vocational education. Two audiences would benefit from such a set of approved practices: (a) those who report research and (b) those who serve as referees of research papers and articles. Many research reports in vocational education contain methodological and reporting problems that the researcher should address. In a study of 18,000 business education research reports, Dvorak (1963) compiled a lengthy list of manuscript shortcomings. In this journal, Copa (1978), McNamara and Gill (1978), and Oliver and Hinkle (1981a) have identified similar problems in conducting and reporting vocational education research. In addition, authors of journal articles sometimes receive contradictory statements from reviewers about their reporting of vocational education research. This appears to be a common problem in all of higher education. Lloyd (1985), in an essay in The Chronicle of Higher Education, is critical of the vagaries of the current peer review process. He cites several research studies that revealed that the peer review process often is subjective and concentrates on trivia. In a study of peer review practices, Peters and Ceci (1982) resubmitted articles to journals that had originally reviewed and published them. Fictitious names and universities were used. Peters and Ceci report that only 3 of the sample of 38 editors and reviewers detected the resubmissions. This allowed 9 of the 12 articles resubmitted to continue through the review process and receive an actual evaluation Eight of the nine were rejected. Sixteen of the 18 referees recommended against publishing the articles, and the editors concurred. All of the articles had previously been published in the same journals. The purpose of this article is to stimulate professionals to review current research reporting practices in vocational education. The article is limited to discussing approved practices in reporting quantitative research. Although the techniques for conducting and reporting qualitative research are similar, there are enough differences to warrant a separate article on that topic. The authors have chosen to concentrate on those topics related to reporting quantitative research that seem to present the most problems to vocational education researchers. They are as follows: Titling the research paper Writing the introduction Reviewing the literature Stating the problem Writing hypotheses Procedures to follow in the methodology section Describing the population Explaining the sampling technique Discussing the research design Describing the instruments Explaining data collection or experimental procedures Analyzing the data Writing the findings Mistakes commonly made in statistical analysis Statistically significant correlations that are not significant Multiple comparison problems The use or misuse of null hypotheses Writing conclusions and recommendations The
First Consideration In Reporting Research
Before writing the first sentence of a journal article or research paper, one question must be answered. That question is, "Is the information to be reported of importance or significance to the profession?" Journals and professional meetings normally do not allot space for articles or presentations just to fill the pages or take up time. The reporter of research must have something significant or of importance to share with the profession (Bartol, 1981; Maher, 1974). Warmbrod (1986) recommends that "we pay greater attention to the significance and importance of the problems and issues that we research" (p. 3). A researcher may attend to all of the suggestions and mechanical details contained in the remainder of this article, but if the topic being reported is of little consequence to the profession, then the reporter is wasting everyone's time. Titling
a Report
The title of a presentation or journal article often gives readers the first impression of the author's work (Resta, 1972). A wellworded title can draw large audiences to a presentation or entice professionals to read a particular journal article. Likewise, a poorly written title can detract from a piece or work. It can disguise the value of a good paper by being too flippant, too dull, too complicated, or too long. Limit Your Title to About 15 Words A title serves several purposes. It is a concise descriptor of the content of the study or research and should be able to stand alone (Silverman, 1982). It also identifies the actual variables under investigation and the relationship between them (Resta, 1972). In today's information age, full-length titles must be able to be collapsed into short titles for use in indexing (Van Dalen, 1979). In addition, short titles are often used as running heads at the top of each page of a typeset journal article (American Psychological Association, 1983). Both Resta (1972) and the American Psychological Association (1983) recommend a title length of about 15 words. Because titles should be short, it is best to avoid words that are already understood by the reader or listener, or serve no purpose. Phrases such as "A Study of," "An Analysis of," or "An Investigation of" add length and redundancy to the title and should be avoided (American Psychological Association, 1983; Resta, 1972; Sandman, Klompus, & Yarrison, 1985; Van Dalen, 1979). Obviously, the reader knows it is a study, an analysis, or an investigation. Use Simple Language When possible; substitute common terms for specialized educational jargon (Henson, 1986). An example is the use of mental and physical instead of cognitive and psychomotor. Van Til (1981) urges, "Don't use an elaborate word when a simple one will do" (p. 71). Avoid abbreviations or other unrecognizable symbols and slang. Simple language helps extend the use of the research to a broader audience. This may be important if the avenue of dissemination selected serves a diverse group of professionals. Maintain Some Sense of Scholarliness The scholarliness of a title should match the scholarliness of the journal or research presentation session. A title that is silly or too dramatic often means that the article or presentation will not receive the recognition it deserves. Listing titles on a professional vita, such as "A Wish and a Prayer" (first-year teacher problems) or "A Shot in the Dark" (using grades as selection criteria for colleges) makes the research efforts appear less rigorous than they actually may be. This does not mean that titles cannot be interesting. However, authors should decide what kind of recognition they want for the effort involved. Funny? Dramatic? Educational? Scholarly? The writer must consider both the audience and the publisher in selecting a title. Writing
the Introduction
The introduction is where the author indicates just how creative or research oriented the article or presentation will be. If the author's decision is not appropriate for the journal or audience, the report probably will be rejected. Therefore, Silverman (1982) recommends that authors look carefully at a number of published articles in a particular journal and/or conference proceedings in order to decide just how much creative freedom they can exercise and still have the work accepted. At least 2 years of past issues should be examined. Writers will have a better chance of getting their work accepted if they have a good understanding of what is acceptable for a particular journal or audience. Once the authors establish the level of creativity possible, they may begin to write the introduction. Use a Hook Mair (1983) advocates the use of a "hook" at the start of the paper. He states, "You must capture your reader's attention right away. Use a hook to get the reader's attention at the very start of your writing. Get him [sic] interested in what you have to say. Don't make him wander through a jungle of words" (ibid., p. 14). Although this approach is more common in writing for general audiences, there is no law saying that it cannot be used in scholarly writing. In Writing for Professional Publication, Van Til (1981) devotes an entire chapter to how to start an article. Some of his suggestions include beginning with a question, leading off with a gimmick, using a quotation, starting with a narrative based upon experiences, using fictional characters to establish a problem, and using "the startling fact." An example of a good "hook" in a scholarly journal is as follows: "How large should the sample be?" Again this past academic year, the authors have been asked this question for the "umpteenth time." (Hinkle & Oliver, 1983, p. 1051) This is how Hinkle and Oliver started an article in Educational and Psychological Measurement. It does catch the reader's attention. The Introduction Should Set the Stage During the introduction, the author should establish the importance of the topic (Borg, 1981). One procedure is to discuss the topic beginning from a broad perspective (global or national) and proceed down to the problem at hand (Moore, 1983). The introduction should prepare the reader for the remainder of the paper (Kowle, Luter, & McGarry, 1979). All of this should be accomplished in 1 1 1/2 pages. Some documentation is appropriate, but reams of references are not needed in this section of the paper (American Psychological Association, 1983). The Introduction Should Be Interesting and Clear Research reporting need not be dull. Continuity of thought, simple expressions, and correct grammar will encourage the reader or listener to continue. Henson (1984) warns, "The most deadly error a writer can make is trying to impress readers through the use of an inflated writing style and ornamental trappings of scholarship" (p. 637). Clarity ensures understanding, which, in turn, breeds interest. Confusion leads to frustration, and this turns off readers and reviewers (Van Til, 1981). Do Not Title the Introduction Logically, the first section of a paper is the introduction. Therefore, there is no need to label the first section of the paper as the introduction (American Psychological Association, 1983; Moore, 1983). Review
of Literature
A journal article or research presentation is not the place for an exhaustive review of the literature. Likewise, the researcher does not want to ignore totally the previous research. The American Psychological Association's (1983) Publication Manual lists the following recommendation: Discuss the literature but do not include an exhaustive historical review. Although you should acknowledge the contributions of others to the study of the problem, cite only that research pertinent to the specific issue. (p. 25) In a research report, authors should briefly summarize and highlight the important findings and conclusions of earlier work. Borg (1981) suggests that "5 to 10 most relevant previous studies should be cited, if only briefly" (p. 69). The review of literature need not be lengthy, but it should demonstrate to the reader/listener that the researcher is familiar with the significant work that has been done in the area. In many journals the review of literature is integrated into the introduction, but it could stand alone as a separate section, if appropriate for a particular journal. The
Statement of the Problem
The purpose of the problem statement in a research report is to present the boundaries of the research in a clear and concrete statement. Because the statement of the problem sets the stage for and gives direction to the rest of the report, it should be stated as early as possible in the report. The American Psychological Association (1983) recommends stating the problem as the last paragraph of the introduction. The question may be raised as to just what is a "good" problem statement. Research problems differ widely, and there is no one "right" way to state a research problem. Leedy (1980) and Moore (1983) indicate that the statement of the problem should be written either as a declarative statement or as a question. Kerlinger (1973) prefers the question style of writing a problem statement. No matter which procedure is used, however, there are certain characteristics that are consistently identified with well-written problem statements. First, the problem statement should identify the variables being investigated (Gay, 1981). If variables are not identified, the problem statement provides very little information for the research consumer. For example, the statement, "The purpose of this study was to examine public school programs for mildly mentally retarded students" leaves the reader uncertain as to exactly what aspects of these programs are being studied or examined. Is the researcher describing the characteristics of the programs, the effectiveness of the programs, the characteristics of the participants, the success of the participants, or all of these? Secondly, the statement of the problem should indicate the relationships between variables that are being investigated (Mason & Bramble, 1978). An exception to this is certain descriptive studies that attempt to describe status, only. However, according to Gay (1981), "Most meaningful descriptive studies are also concerned with relationships between variables" (p. 128). When relationships are being studied and the problem statement does not indicate this, the reader or listener is again left with uncertainties about the nature of the research. A third characteristic of a good problem statement is that it should identify the target population (Leedy, 1980). This component enables readers or listeners to determine readily the group being investigated and whether the research is relevant to their area of interest. Without the identification of the target population, the desired degree of clarity and specificity is absent from the statement of the problem. Finally, a well-written problem statement should be stated in its simplest form. According to Kerlinger (1973), "The problem should be stated clearly and unambiguously (p. 117). Leedy (1980) is more emphatic: If you really know what your problem is, state it clearly. Each word of the problem should be expressive, sharp, indispensable, definitive. Always state the problem in a complete grammatical sentence. Your problems should be stated so well, in fact, that anyone, anywhere (who understands English) could read it, understand it, and react to it without benefit of your presence. (p. 52) Hypotheses
The statement of the problem sets the stage for the rest of the research report, but the hypotheses provide specific statements about what the investigator has tested and is reporting. The hypotheses are derived directly from the statement of the problem. According to Gay (1981): A hypothesis is a tentative explanation for certain behaviors, phenomena, or events that have occurred or will occur. A hypothesis states the researcher's expectations concerning the relationship between the variables in the research problem; a hypothesis is the most specific statement of the problem. (p. 45) The following are criteria for good hypotheses: 1. Hypotheses should be stated in declarative sentence form and should state an expected relationship or difference between two or more variables (Kerlinger, 1973; Moore, 1983). 2. The direction or nature of the relationship(s) should be specified in the hypotheses. If a relationship (or difference) can be hypothesized to exist, then the nature of that relationship (or difference) can also be hypothesized (Borg & Gall, 1979; Kerlinger, 1973). 3. The independent and dependent variables should be identified and should be operationalized in measurable terms in a hypothesis (Resta, 1972). For example, rather than saying "achievement of students," the variable would be operationalized as grade point average or score on the XYZ achievement test. 4. Hypotheses should be testable. The researcher should be certain that any stated hypothesis can be tested by some objective means (Borg & Gall, 1979; Gay, 1981). 5. Hypotheses should be as concise and clear as possible (Borg & Gall, 1979). As mentioned earlier, "The simplest way is the best way" (Kerlinger, 1973, p. 18). 6. Finally, hypotheses should be supported by a rationale. There should be a basis for the formulation of the hypothesis. This basis may be derived from theory, from the findings of the related empirical research of others, or from logical argument based on expert opinion and/or personal experience (Mason & Bramble, 1978, Moore, 1983). Hypotheses are used in most types of quantitative research. However, in some research, especially descriptive studies, it may be more desirable to state objectives as questions to be answered in the study (Borg & Gall, 1979). Hypotheses are essential to all research studies, except for these descriptive studies that have as their purpose to answer certain specific questions. Method
A major concern of people who use research is, "is the research valid-both internally and externally?" The method section of an article or presentation gives the researcher the opportunity to provide evidence that the research is valid, reliable, and objective, particularly in regard to selection of the population, sampling, design of the study, use of instrumentation, data collection, and data analysis. Population The author should discuss the population under study (American Psychological Association, 1983; Moore, 1983), including why the population was chosen and how members of the population were identified. Here, limitations and problems in identifying members of the population can be provided. If the population is not adequately described, it will be difficult for the reader to determine whether the results of the research are generalizable to other groups. Sampling When the use of an entire population is not possible, the researcher uses a sampling procedure. Two important factors should be emphasized in any discussion of sampling procedures. First, the researcher should explain which of the sampling techniques was used and why it was selected (Kerlinger, 1973). Researchers often state which sampling technique was used, but not the rationale for its use. The reader may want to know why a multistage sampling technique was used instead of a proportionate random sample. Second, the researcher should explain why a certain sample size was selected. The reader/listener needs to know if the sample size was determined according to established guidelines (Cochran, 1953; Krejcie & Morgan, 1970; Oliver & Hinkle, 1982) or was selected because of convenience, finances, or personal choice. When round numbers such as 50 or 100 are reported as the sample size, readers often wonder how this sample size was determined. If time or money prevents the researcher from using the ideal sample size, this should be noted. Design of the Study The validity of a study is dependent, to a large degree, on the design of the study. The researcher should explain the design, justify why it was selected, tell how extraneous variables were controlled, and describe the procedures used (Mouly, 1978). This allows the reader/ listener to better judge the validity of the research. Instrumentation Most research involves the use of some type of measuring instrument(s), such as questionnaires, achievement tests, or opinionnaires. The author should address the issues of validity and reliability for each instrument utilized. If previously developed instruments were selected, then the author should describe how each was designed, developed, and validated in earlier studies (Gay, 1981). In addition, the author has the responsibility to demonstrate the validity of the instrument(s) for the current research, because "one validates not the measuring instrument itself, but the measuring instrument in relation to the purpose for which it is being used" (Carmines & Zeller, 1979, p. 17). In addition to the validity of measuring instruments utilized, the author has the responsibility to report the reliability of each instrument. "Reliability, as applied to educational measurements, may be defined as the level of internal consistency or stability of the measuring device over time" (Borg & Gall, 1979, p. 217). If an instrument is reliable, then the reader would expect that whatever the instrument measures is measured consistently. Reliability is normally expressed as a numerical coefficient ranging from .00 to 1.00. An instrument that is perfectly reliable will have a coefficient of 1.00. As Carmines and Zeller (1979) state, "As a general rule, we believe that reliabilities should not be below .80 for widely used scales" (p. 51). If previously developed instruments are used, then the author should report reliability estimates from recent and/or related administrations of the instrument. If, on the other hand, researcher-designed instruments are used, the author should report the reliability estimate and the method of estimating that was employed. When a field test of the instrument was conducted, the reliability should be reported both for the field test data and final data collection. Data Collection or Experimental Procedures A step-by-step description of how the data were collected, including time frames, should be presented to the reader/listener. What was done to ensure a high participation rate? In the case of mail surveys or personal interviews it is crucial to report the response rate. The procedures used to follow up nonrespondents or handle dropouts in experimental research should be described, as well as what treatments were used and how they were administered, This step-by-step procedure allows the research to be replicated (Kerlinger, 1973). Data Analysis The researcher should briefly explain the statistical test(s) used for investigating each variable under study. This explanation may describe why the tests were selected and should explain why other statistical tests were not selected. This section often is the last part of the methods sections and serves as a transition to the results or findings section (Moore, 1983). There are several sources available to help researchers identify appropriate statistical tests, such as the Oliver and Hinkle paper (1981b). It should be noted that the selection of statistical tests should be done prior to conducting the research. Findings
The findings or results section is generally the longest section of the research paper. The material contained in this section should relate directly to the problem under study. As Kerlinger (1973) suggests: While writing [the results section], the investigator should guard against wandering from the task at hand, the solution of the research problem. Everything he writes must be geared to letting the data bear on the problem and hypothesis. (p. 697) Two major types of information are contained in the findings section: narrative and statistical information. Narrative information, written in prose, is a description of the findings of the study. The statistical information normally is presented in tabular or graphic form and is used to supplement the narrative visually. Although the statistical information supplements the narrative, and vice versa, both types of information should be able to stand alone. Graphic and/or tabular displays of data should not be used to "short cut" the written part of the narrative. It is not proper to state, "The characteristics of the teachers are presented in Table 1 " without summarizing the major points of the data found in Table 1 in prose form. Ary, Jacobs, and Razavieh (1972) suggest that writers "present the data in tables and figures accompanied by sufficient text to point out the most important and interesting findings (p. 335). The American Psychological Association (1983) advises the writer to "tell the reader what to look for in tables and figures and provide sufficient information to make them readily intelligible” (p. 27). The purpose of presenting both types of information is to make it easier for the audience to understand the findings. The journal article or presentation should still make sense if the tables, charts, and/or graphs were removed. Interpreting and Using Statistics A common problem in reporting the findings of research is the misuse of certain statistical techniques (McNamara & Gill, 1978; Oliver & Hinkle, 1981 a). As far back as the mid-1960s, Hamlin (1966) warned about the misuse of statistics: "Another current obsession is with elaborate statistical analysis, used as much to obscure as to illuminate" (p. 15). In reporting research, it is important that Hamlin's observation be heeded and that statistics be used to illuminate the research and not obscure it. The research may be exemplary, but if the researcher incorrectly applies or interprets the statistical tests, the entire research effort may be viewed unfavorable. Following is a brief discussion of several statistical procedures that often are misused in reporting research. Correlation coefficients. A correlation is a measure of association between two variables. Glass and Stanley (1970) have identified several problems in interpreting and reporting correlations. One common problem in correlational research is the assumption there is a causal link between the variables. The presence of a correlation between two variables does not necessarily mean that a causal link exists between them. Other variables not included in the study may be responsible for the observed association and the complexity of the relationships between variables. In education and the social sciences research, the problems generally are so complex that they cannot be explained in terms of a single cause. Glass and Stanley (ibid.) also note that "many researchers do not stop with one fallacious conclusion, i.e., that correlation is prima facie evidence for causation, but draw a second one as well. They assume a certain direction for the causal relationship (p. 121). A researcher who does correlational research and then reports that variable X causes variable Y is guilty of misusing the correlation coefficient. A second misuse of the correlation coefficient is making bold claims about correlations that are statistically significant but are, in actuality, weak correlations. It is possible to have a correlation coefficient of .12 that is statistically significant at the .05 alpha level. The statistical significance for correlations depends upon two factors--the correlation coefficient and the number of cases (N). As N increases, the size of the correlation coefficient needed to be statistically significant decreases. With 20 cases, a correlation coefficient of .42 is needed to be statistically significant at the .05 level of significance; with 100 cases, a correlation coefficient of .195 would be significant. In reporting correlations, researchers should first consider the strength of the correlation before they consider the statistical significance. Best (1981) offers the following standards to be used in interpreting correlations: .00 to. 20 Negligible A low or negligible correlation that is statistically significant because of a large N is of little practical value to the researcher. When the correlation coefficient starts to exceed .30 and is statistically significant, the researcher may have something of practical importance to report. Multiple comparison problems. Assume a researcher administered a 50-item instrument to two different groups of people and wanted to determine if there was a difference in the way they responded. One method of analyzing the data would be to conduct 50 t-tests, one for each item on the instrument. This technique greatly enhances the possibility of having a significant t-test happening by chance. If a researcher does one t-test, the chance of having a significant finding by chance is 5 out of 100 (assuming the .05 probability level has been chosen). When two t-tests are conducted on the data, the possibility of having a significant finding by chance increases slightly. When 10 t-tests are performed, the probability of having one significant finding by chance increases even more. When 50 t-tests are performed, it is nearly certain that one or more of the significant t-values happened by chance. The use of multiple statistical tests (e.g., t-tests, ANOVAs) increases the possibility of making a Type 1 error--rejecting the null hypothesis when it is true. The researcher is actually changing the significance level of the research by using multiple statistical tests. The probability in the previous illustration of having a significant t-test that happened by chance is closer to 1.0 than .05. The power of the decision is also affected by the multiple comparison problem. Multiple comparison tests appears to be a major problem in vocational education research. Oliver and Hinkle (1981a) examined four studies in vocational education that had this problem. The solution to the problem depends upon the problem being investigated and the conditions. In some cases, a solution to the problem may be to group data. Similar items on an instrument should be grouped together, summed, and treated as one variable for statistical purposes. Oliver and Hinkle (1981a) suggest factor analysis to accomplish this. Other suggestions are to increase the alpha level to .01 or .001 to help reduce the problem or to use a test for practical significance (McNamara & Gill, 1978). A detailed description of what happens when multiple comparisons are made (either multiple t-tests or multiple ANOVAs) and solutions to this problem are outside the scope of this article. The reader is encouraged to consult Oliver and Hinkle (1981a) or McNamara and Gill (11978) for a detailed description of the problem. Null hypotheses. There are those in the profession who believe null hypotheses should be included in a paper or article. If null hypotheses are included, they should be located in the findings section along with the statistics that were used to test the null hypotheses (Borg, 1981). The practice of reporting null hypotheses is quickly becoming a thing of the past. The American Psychological Association has taken the following stance (1983) in reporting research: "Assume that your reader has professional knowledge of statistics. Basic assumptions, such as rejecting the null hypothesis, should not be reviewed" (p. 27). Knowledgeable researchers understand the underlying concept of using statistics and the null hypothesis, Therefore, it is not necessary to restate the research hypothesis in the null form and publicly make a decision about whether or not to reject the null hypothesis. This takes up valuable space in a paper and is often redundant. If a person insists on using null hypotheses, the statistical test used should lead the researcher to either reject or fail to reject each null hypothesis thesis. It is improper to report that the null hypothesis was accepted or not accepted. "Reject" and "fail to reject" are the correct terms. Writing
Conclusions and Recommendations
The conclusions and recommendations section of the research report involve the investigator's interpretation of the facts revealed in the study. As such, care must be taken to ensure that any conclusions drawn are firmly supported by the empirical evidence presented. The object of the conclusions is "to establish as clear-cut an answer to the question posed in the statement of the problem as the data of the present study analyzed in the light of the situation and of the work of previous investigators will permit" (Mouly, 1978, p. 329). Errors commonly committed in writing conclusions include: (a) confusing results and conclusions and (b) overgeneralizing results. "A result is the outcome of a test of significance" (Gay, 1981, p. 385), whereas a conclusion is the interpretation of the relevant hypothesis. Conclusions are drawn from the findings or results. Gay (ibid.) warns, "Overgeneralization refers to the statement of conclusions that are not warranted by the results" (p. 385). One of the authors of this article recently was a paper discussant at a research conference. A paper was presented that examined the effects of two instructional modes in teaching computer science at the university level. The researcher concluded that high school vocational teachers should use computers. Nothing in the research presented dealt with high school vocational teachers. The conclusion was not based on the findings of the research. Clearly, caution should be exercised in arriving at conclusions (Mason & Bramble, 1978). In addition to developing conclusions, the researcher should discuss the implications of the research findings and make recommendations for practice and/or future research (Mason & Bramble, 1978). The recommendations for practice should be based on the findings of the study. Recommendations for future research may include suggestions for replication in different settings or with different groups that will increase generalizabiIity or next-step studies designed to build on the current research and investigate other dimensions of the problem. A summary of the major points discussed in this article are contained in checklist form in Figure 1. Conspectus
Researchers in higher education (Boice, 1984; Boice & Johnson, 1984; Boice & Jones, 1984; Hartley & Knapper, 1984; Hunter, 1985; Jalongo, 1985; Levi & Grasha, 1983; Lumsden, 1984) have indicated that faculty members are taught the approved practices in conducting quantitative research with minor or no emphasis on approved practices in reporting quantitative research. Warmbrod (1977) suggests that the research process is not complete until research is reported. The authors encourage the profession to consider, discuss, and debate what the approved practices in reporting research should be. Perhaps the points presented in this article can provide a point of departure for the profession in determining approved practices for reporting quantitative research in vocational education. Figure 1. A checklist of approved practices to follow in reporting quantitative research in vocational education ___ 1.The topic being reported is of significance to vocational education. If the topic being reported is of little or no significance to vocational educators, the remainder of the items could be checked and the article or presentation would still not be acceptable. ___ 2. The title is no longer than 15 words. ___ 3. Unnecessary words, such as "a study of" or "an analysis of," are not in the title. ___ 4. Simple words are used instead of educational jargon in the title. ___ 5. An appropriate sense of scholarliness is found in the title. ___ 6. The introduction does not say "Introduction." ___ 7. A "hook" is used to grab the interest of the reader in the introductory section. ___ 8. The importance of the topic is established early in the paper. ___ 9. The introduction is interesting and written clearly. ___ 10. A brief review of the most pertinent literature is included in the introduction (or at another location, if more appropriate). ___ 11. The introductory section is 1 - 1 1/2 pages. ___ 12. The statement of the problem A. identifies the variables being investigated; 13. If hypotheses are used, they— A. are stated in declarative sentence form; ___ 14. The population being studied is identified. ___ 15. A rationale is given for the sampling technique used. ___ 16. The reasons for selecting the sample size are given. ___ 17. The design of the study is described. ___ 18. Instrument development or selection is discussed. ___ 19. The validity of the measuring instruments is discussed. ___ 20. The reliability of the measuring instruments is discussed. ___ 21.A step-by-step description of data collection or experimental procedures is presented. ___ 22. Response rates for survey studies are presented, ___ 23. Procedures for handling nonrespondents or dropouts are described. ___ 24. The rationale for the statistical procedures used is presented. ___ 25. The findings are presented in a straightforward narrative. ___ 26. Both descriptive and inferential statistics (if appropriate) are presented in the findings section. ___ 27. Tables and figures are used, as appropriate, to present the findings. ___ 28. Statistics are used and interpreted correctly. ___ 29. Conclusions and recommendations for practice are based upon the research findings. ___ 30. Caution is used in making conclusions and recommendations. References
American Psychological Association. (1983). Publication manual of the American Psychological Association. Washington, DC: American Psychological Association. Ary, D., Jacobs, L. D., & Razavieh, A. (1972). Introduction to research in education. New York: Holt, Rinehart, and Winston. Bartol, K. M. (1981, August). Survey results from editorial board members: Lethal and nonlethal errors. Paper presented at the Annual Meeting of the American Psychological Association, Los Angeles, CA. Best, J. W. (1981). Research in education. Englewood Cliffs, NJ: Prentice-Hall. Boice, R. (1984). Reexamination of traditional emphases in faculty development. Research in Higher Education, 21(2), 195-209. Boice, R., & Johnson, K. (1984). Perception and practices of writing for publication by faculty at a doctoral-granting university. Research in Higher Education, 21(2), 159-177. Boice, R., & Jones, F. (1984). Why academicians don't write. Journal of Higher Education, 55(5), 567-582. Borg, W. R. (1981). Applying educational research. New York: Longman. Borg, W. R., & Gall, M. D. (1979). Educational research. New York: Longman. Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Beverly Hills: Sage. Cochran, W. G. (1953). Sampling techniques. New York: John Wily & Sons. Copa, G. H. (1978). Key questions in evaluating the reporting of research. The Journal of Vocational Education Research, 3(2), 1-6. Dvorak, E. A. (1963). Shortcomings of graduate research in business education. Ohio Business Teacher, 23, 19-22. Gay, L. R. (1981). Educational research. Columbus, OH: Charles E. Merrill. Glass, G. V., & Stanley, J. C. (1970). Statistical methods in education and psychology. Englewood Cliffs, NJ: Prentice-Hall. Hamlin, H. M. (1966). What is research? American Vocational Journal, 41(6), 14-16. Hartley, J., & Knapper, C. K. (1984). Academics and their writing. Studies in Higher Education, 9(2), 151-167. Henson, K. T. (1984). Writing for professional publication: Ways to increase your success. Phi Delta Kappan, 65(9), 635-637. Henson, K. T. (1986). Writing for publication: Play to win. Phi Delta Kappan, 67(8), 602-604. Hinkle, D. E., and Oliver, J. D. (1983). How large should the sample be? A question with no simple answer? Educational and Psychological Measurement, 43(4). 1051-1060. Hunter, D. E. (1985, March). What prolific higher education scholars have to say about their careers. Paper presented at the Annual Meeting of the Association for the Study of Higher Education, Chicago. Jalongo, M. R. (1985). Faculty productivity in higher education. Educational Forum, 49(2), 171-82. Kerlinger, F. N. (1973). Foundations of behavioral research. New York: Holt, Rinehart, and Winston. Kowle, C., Luter, R., & McGarry, M. (1979). Getting published. Arlington, VA: American Vocational Association. Krejcie, R. V., & Morgan, 0. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30, 607-610. Leedy, P. D. (1980). Practical research. New York: Macmillan. Levi, L. S., & Grasha, A. (1983, August). Motivational processes and personal attributes of writers: An exploratory study. Paper presented at the Annual Meeting of the American Psychological Association, Anaheim, GA. Lloyd, J. E. (1985, June 26). Selling scholarship down the river: The pernicious aspects of peer review. The Chronicle of Higher Education, p. 64. Lumsden, D. B. (1984). Preparing future educational leaders to write for scholarly publication. Innovative Higher Education, 9(1), 36-41. Maher, B. A. (1974). Editorial. Journal of Consulting and Clinical Psychology, 42,1-3. Mair, G. (1983). How to write better in one hour. New York: Scarborough. Mason, E. J., & Bramble, W. J. (1978). Understanding and conducting research, New York: McGraw-Hill. McNamara, J. F., & Gill, O. H. (1978). Practical significance in vocational education research. The Journal of Vocational Education Research, 3(3), 27-48. Moore G. W. (1983). Developing and evaluating educational research. Boston: Little, Brown, and Company. Mouly, G. J. (1978). Educational research: The art and science of investigation. Boston: Allyn and Bacon. Oliver, J. D., & Hinkle, D. E. (1981a). The multiple comparisons problem in vocational education research. The Journal of Vocational Education Research, 6(4), 39-47. Oliver, J. D., & Hinkle, D. E. (1981b, December). Selecting statistical procedures for agricultural education research. Paper presented at the National Agricultural Education Research Meeting, Atlanta, GA. Oliver, J. D., & Hinkle, D.E. (1982, July). Determining the sample size: Research involving multiple samples. Paper presented at the Southern Research Conference in Agricultural Education, Athens, GA. Peters, D. P. & Ceci, S. J. (1982). Peer review practices of psychological journals: The fate of published articles, submitted again. The Behavioral and Brain Sciences, (5), 187-255. Resta, P. E. (1972). The research report. In R. L. Baker & R. E. Schultz (Eds.), Instructional product research. New York: American Book Co. Sandman, P. M., Klompus, C. S., & Yarrison, B. G. (1985). Scientific and technical writing. New York: Holt, Rinehart, and Winston. Silverman, R. J. (1982). Getting published in education journals. Springfield, IL: Charles C. Thomas. Van Dalen, D. B. (1979). Understanding educational research. New York: McGraw-Hill. Van Til, W. (1981). Writing for professional publication. Boston: Allyn and Bacon. Warmbrod, J. R. (1977, July). Remarks at the Central Region Research Conference in Agricultural Education, Ames, IA. Warmbrod, J. R. (1986, March). Priorities for continuing progress in research in agricultural education. Paper presented at the Southern Region Research Conference in Agricultural Education, Little Rock, AR. 1Gary E. Moore is a Professor and Michael F. Burnett is an Associate Professor in the School of Vocational Education and Technology at Louisiana State University, Baton Rouge, LA. Barbara A. Moore is Director of Education in the Office of Women’s Services, State of Louisiana, Baton Rouge, LA. |