Article Evaluation
Steve Varga
LIS 506

Fall 1996

Bibliographic Information
Article Recruitment of Librarians
into the Profession
Public Library Development
in the United States, 1850-1870:
An Empirical Analysis
Oregon's Public Libraries
during the
First World War
Journal

Library and Information
Science Research

Journal of Library History

Oregon Historical Quarterly

Authors/
Other works
Buttlar, Lois
Caynon, William
Williams, Robert V. Weigand, Wayne A Comments

The following table is an overview of the evaluation.  The criteria, found in the first column, was adapted from Evaluating Information A Guide for Users of Social Science Research  by Katzer, Cook, and Crouch.  It is supplemented by lecture material.  Each cell links to the individual questions used to evaluate the particular criterion. The following three columns contain a subjective rating of good, fair, or poor for each of the articles.  In the last column is a link to a paragraph or more which consists of further comments and comparisons between the articles.

Note that the last article is qualitative research, quite different from the empirical research found in the first two articles. The criteria from Katzer apply to both types of research but is augmented with elements oriented toward empirical research. The augmentation may be difficult to apply to the Weigand article.  When this is the case, appropriate comments are made.

Evaluation
Criteria
(Katzer,1982)
(augmented)



Research type

(1)
Recruitment of Librarians
into the Profession


Quantitative

(2)
Public Library Development
in the United States, 1850-1870:
An Empirical Analysis

Quantitative

(3)
Oregon's Public Libraries
during the
First World War

Qualitative



Comments

Intentions

Good

Fair

Good

Comments
Problem
Statement

Fair

Good

N/A

Comments
Definitions

Poor

Good

Good

Comments
Literature
review

Fair

Poor

N/A

Comments
Instrumentation

Fair

Good

N/A

Comments
Measurement

Poor

Fair

Good

Comments
Control

Poor

Poor

N/A

Comments
Summary

Fair

Good

Good

Comments
Interpretation

Fair

Good

Good

Comments
Summary Summary Summary

Other works:

Based on entries found through First Search, Buttlar and Caynon, co-authors of  "Recruitment of Librarians into the Profession: The Minority Perspective" (1) are the authors most published out the three articles reviewed.  Buttlar was found to have written nine other articles and Caynon, two.  Both are professors at Kent State University with Buttlar having research interest in ethnicity and multiculturalism in libraries.  Their profiles from Kent State reveal many other publications.  First Search had Robert Williams, author of "Public Library Development in the United States, 1850-1870: An Empirical Analysis" (2), with six other entries.  Those works and the journals in which they were published show an affinity toward empirical studies.  Wayne Weigand, author of "Oregon's Public Libraries during the First World War" (3), only had two other articles in the First Search database. Both, like the article reviewed, appear to be historical research.

Research type

Comments & Comparison

The first two articles can both be classified as quantitative-descriptive empirical research.  Both contain the elements expected by research methodology.  This includes problem statements, literature reviews, methodology descriptions, results, and conclusions.  Yet significant differences can be seen between them.  The Williams article is far more empirically rigorous than the Buttlar article.  The Buttlar article establishes correlations through statistical methods but in its results uses such techniques as quoting comments made by individuals to support its assertions.  In short its emphasis could be said to be descriptive where the emphasis of the public library development study by Williams is on quantitative evidence.  The Oregon article by Weigand is a qualitative historical piece.  It contains none of the elements of empirical research.  It instead reads as a narrative, telling the story of public libraries in Oregon during World War I.  References numbered in the article are found in a note section at the end of the article.  The note section provides bibliographic information as well as further comments concerning the reference.

Framework:

Intentions

Comments & Comparison

The intended audience is not explicitly stated in any of the articles but can be deduced.  For the Buttlar article, library administrators responsible for recruitment appear to be the specific target, and secondarily, all professionals in the field.  The bibliography supports this conclusion.  The Williams article requires more deduction.  The empirical approach and the subject matter indicate that social scientists interested in the historical development of libraries are the intended audience.  The strongest evidence concerning intended audience are the journals in which the articles are published.  For the Wiegand article, this suggest those interested in Oregon history are the primary target.  Librarians obviously may have an interest as well as World War I historians.  It also seems written to appeal to broader segment of the general population.  Prior knowledge of library history is not necessary to follow the article.

The objective/purpose of the first two articles are explicitly given.  Briefly, the objective of the recruitment article is to identify factors and affective approaches in influencing minorities to choose the librarian profession.  The intention of the second article is to explain library development during a specific time period using an alternate theoretical framework.  The third article's purpose, based on its title and introductory paragraph, is to examine how Oregon's public libraries responded to the patriotic fervor found in Oregon during World War I.

In general, all the articles did a fairly good job establishing their intent, in terms of audience and purpose, at the beginning of the piece.  If it was not done explicitly, it was still relatively easy to determine.

Problem Statement

Comments & Comparison

The problem statement for the first article is two-part.  First, minorities are under-represented in libraries.  This is a problem because "minorities are best served by minorities."  Such an important premise should have been supported by much more extensive evidence than it was.  The evidence that is provided is of demographic statistics concerning the proportional representation of minorities in the population versus the library profession.  The second part of problem statement is that there is a dearth of information concerning recruitment of minorities.  The first part of the statement may be too broad to be solvable but the second part certainly can be addressed.  The problem statement is open-ended but can not be called nonbiasing.  The researchers definitely have an agenda.  The language used to start the article and in the conclusion does not have an objective tone.  Yet the body of the article, where the study is defined and the statistical analysis presented, does come across as being objective.

The problem addressed in the Williams article is initially stated in general terms, " ... our understanding of the causes of this development (libraries from 1850 to 1876) is not good".  It is not good because explanations differ considerably. At this point the problem certainly is open-ended but is not in solvable terms.  The problem statement then gets very specific lapsing into describing the approach, i.e. the problem is to see if a theoretical framework could explain the development based on an empirical study.  It may have  been useful to first explore and discuss other possible approaches and approaches tried by others.  Positively, the specificity does provide the piece with a definitive direction.   The language used in the article is neutral and objective.  It is not filled with discipline laden terms which made it easy to read and absorb.

The qualitative research article by Wiegand does not contain a problem statement.  What role Oregon's public libraries played during World War I is the primary question addressed in the article.  It is written in a factual authoritarian style.  The effect of this, coupled with being written in a narrative manner, is for the article not to fit the mode of all the other research articles to which we've been exposed in this class.  It also does not identify itself as research and therefore requires a second look to see it as such.  The piece does contain some obvious bias.  The author goes to great length to explain/defend some of the actions of the public libraries such as its role in censorship.  

Definitions

Comments & Comparison

No definitions were offered in article 1.  Although the piece is not highly technical, definitions would be useful. Categories or groupings are used extensively throughout the piece including minority classification, family background, professional status, sex and age.  The author must see all of these categories as self-defining as sex and age.  For the purposes of empirical analysis this is inadequate.  For example, minority classification has significant complications such as not necessarily being mutually exclusive.  References are used exclusively in the introduction, background, and literature review sections of the article.  They are used to provide primarily statistical evidence in support of the problem statement that minorities are under-represented in the library profession.  Other citations deal with the more qualitative question of why the statistics are what they are.  The references are appropriate but appear biased in that everything presented is consistent with other references and the authors' statements.  Given the subject, this may be not be any inconsistencies but this point is noteworthy.  A couple of references were made in the result section to studies whose results were somewhat different.

Definitions are found in the public library development article.  The "social conditions theory" is a cornerstone of the article and therefore clearly explained.  The purpose of the statistical measurements used are defined.  An important section of the article describes defines the independent variables extracted census statistics.  Definitions were brief, appearing to be over-simplified.  As article one, some categorizations were considered self-evident when perhaps they should have been defined.  References are used throughout the article to build a foundation for the research, to establish the validity of the census data, and to compare results.  The reference material presented is balanced in contrast to the first article.   It is not always confirming.  For example, some reference matter call into question the validity of the data.  Overall the references for the article were are excellent and complement the statistical methods. 

Like the first article, the Wiegand article also lacked definitions.  The closest it comes is occasionally providing background information on organizations that play a role in the story, such as the ALA.  Given its qualitative non-technical approach, rigorous definitions are not necessary.  Therefore this is not a serious shortcoming for this piece.  The references are the sole source of evidence for the article.  References are used to provide bibliographic information, add further context to a particular point, provide additional resource material, describe the use and relevance a citation, and give further detail on the location and accessibility of the evidence.  This note section gives the piece credibility.  There are 26 references made in the text.  References often contained multiple citations.  To validate the authenticity of the references I was able to locate source material from the first three references in the Central Washington University library.

Literature Review

Comments & Comparison

There is a lack of uniformity among literature reviews.  The most useful kind of review is evaluative in approach with the purpose being to put the present study into perspective in terms of its research history and theoretical context (Katzer, 1982).   The literature review is substantial in the article on minority librarianship.  It is broken into two sections, "Minorities and Library Schools", and "Factors Affecting Minorities Entering the Field".   These sections focused on findings and conclusions of earlier work.  They have little evaluative substance and consist entirely of supportive material as noted when discussing references.  Yet the organization of the review material is good and serves the purpose of adequately "setting up" the presentation of the new research.

"The explanations (concerning public library development) offer by historians differ considerably" is central to the problem statement of the second article.  Yet the author does not review the literature of these explanations saying they "have been reviewed extensively elsewhere and are not repeated here".  He does develop and explain the "social conditions theory" which is the label placed on some these explanations but he does not do it through reviewing findings of these other works.  Therefore one must say the lack of literature review is a shortcoming of this research article.  The author assumes that one is aware of the prior research and/or can locate it easily if they desire.

The historical article on Oregon's public libraries does not contain a literature review section unless one considers the entire piece a literature review which in essence, it is.

Methods:

Instrumentation

Comments & Comparison

The research design of the first article intended to gather data from a sample of minority professional librarians. Nominal data is aggregated to gain an overall demographic picture and inferential statistics, the chi square test, to find correlations.  While the first article is clearly not an experimental design, as described by Campbell utilizing experimental (independent) and dependent variables (Campbell, 1963), the second article on public library development does attempt to follow an experimental design.  It establishes library development as the dependent variable defining this as the sum of the number of libraries per ten thousand population and volumes per capita in those same libraries. Seven explanatory or independent variables are defined in the table mentioned above.  The design followed appears to be the one-shot case study (Campbell, 1963) where a single group is studied once in relationship to some agent presumed to cause change.  Campbell points out the numerous problems with internal and external validity inherent in this design but given the problem, looking at library development over a century ago, there is no other choice but this design in an empirical approach.  The research design of the third article could simply be described as the compilation and validation of available evidence concerning the activities of Oregon's public libraries during WWI.  That compilation was then to be presented in narrative form.

The instrument used in the first article is a survey.  The survey, found as an appendix, is quite simple, comprised of 16 questions.  Of those 16, eight are collecting basic data information such as age and sex.  Yet in some of those questions the problem of categorization, mentioned earlier, could be confusing.  In addition, at least two questions are worded such that bias is introduced:

"In order to attract minorities, library schools should:
. . .
* have flexible admission standards
* give candidates preferential treatment"
. . .
As one might expect, the overwhelming response to the first question was "yes" and to the second "no".  The word "flexible" has positive connotations while "preferential" often raises a red flag.  These two questions should have been tossed but the rest appear acceptably objective for use in the analysis.  The population is clearly defined, minority library professionals, but the introduction of bias in the sample selection must be examined.  A stratified random sampling was  taken from professional membership list.  Using such list introducing the bias of selecting librarians with the most to gain through membership, i.e. academic librarians.  This is noted by the authors.  Also respondents (57%) were self-selecting in that they chose to return the questionnaire.  There could be reasons in common that the others did not that would significant alter the findings.  Perhaps those who strongly believe in a color-blind society may have resented being singled out for the survey.

The instrument, or source of the data, for the public library development study is the U.S. censuses of 1850, 1860, and 1870.  It is the major source of empirical data available for the purpose of the study.  Its reliability is discussed at great length in the article.  The author is very "up-front" about it. He quotes the 1870 superintendent of the Census office saying "The statistics of libraries have never been very creditable to the census of the United States". Certain information is discarded.  Descriptions of how the data was "scrubbed" to improve its reliability is discussed. The author justifies use of the data based with a familiar argument.  Statistically the statistical tools were massaging the data searching for trends, not to count beans.  Trend analysis is not sensitive to "dirty" data.  Results of trend analysis may still be reliable when the error rate of data is quite high and unacceptable for other purposes.  Although the research design is fraught with validity issues and the data used is highly unreliable, I found the author did an excellent job of working with these problems in order to produce a credible study.

Measurement

Comments & Comparison

The findings in the first article are found in three sections. Demographic characteristics and the questions concerning what libraries should do are analyzed quantitatively in the first two section. The last section reviews the comments made by the survey respondents. A major portion of the quantitative analysis text consists simply of reporting the tabulation results as percentages. "For Asians, 24% are in their 20s" is typical. Redundantly most of the same information is presented in eight tables. The tables do enhance the article but graphical presentations, such as pie charts, would have been better for many of the tables giving ratios. Besides ratios, the chi square statistic is calculated for six of the eight tables found in the study. The measurement is not described as to purpose or dependability. It is presented as a determinant of a significant relationship between two attributes. It is the proper test to use for nominal data. But the authors did not follow the convention of setting a predetermined significance level and then accepting or rejecting the null hypotheses. Instead the significance level is given as another statistic, the lower the p, the greater the significance.

The quantitative analysis of the library development article begins with a table presenting descriptive measures of central tendency. The Pearson product-moment coefficient and linear, stepwise multiple regression are the statistics used to evaluate the theoretical framework. Each are briefly described as to what they assess. This discussion could have been more thorough. The author does note that no significance-level information is given since the data is of the entire population. Discussed earlier, the author presents substantial reasons for concern over the validity of the data which brings into question any measurement. Turning to the tables, they are not near as busy as those in the first article. They are easier to follow and complement the text. Again, graphical presentations would add greater saliency.

The third article does not use statistical measurements and does not contain any illustrations. The author did have to assess the validity of the references used. He provides evidence of these assessments in the notes.

Control

Comments & Comparison

Control is poor in both of the qualitative studies. Neither of the research designs utilize a control group for comparison. The article on the minority library professionals does not consider rival explanations (which may not be applicable) while rival explanations were just said to differ in the second article. The library development study does expand on the general explanation of library development based on socioeconomic issues, as the purpose of this study appears to be refining that explanation. Control, as implied by the questions presented above, is not applicable to the third article.

Results-Conclusion:

Summaries

Comments & Comparison

The summary and conclusion of the minority library professionals article is quite short reflecting the fact that the objectives of the study are really quite modest, to generate a few demographic statistics and identify some key factors that may be considered to improve minority recruitment. It summarizes these recommendations in the conclusion. There is no suggestion for further research nor do the authors try to place the study in a broader perspective.

Williams has much more to say in the summary of the library development study. He states that the theoretical rationale holds up well and has merit while confirming that library development is a "complex phenomenon". The summary identifies what it considers to be the surprises and talks about how the results are contrary to other development theories (theories that are first identified at this point). Finally in the conclusion the author states that the problem under study is far from resolved suggesting an attempt be made to study library development at a finer resolution, i.e. looking at the county level or smaller. He points out the weaknesses of the study as an argument for further work.

The summary of the Weigand's historical article on Oregon's public libraries during WWI call the period "tumultuous" but also a time of great opportunity for the public libraries. It reviews the many ways they became involved as a public agency including ways that may not meet with approval today. It concludes by placing the events of the period in a boarder perspective, pre- war and post-war. In the short term the profile of the public libraries was raised considerably but soon returned to its pre-war status as another municipal service.

Interpretations

Comments & Comparison

Since the results/conclusion section is so light for the librarianship article,  what there is to say about interpretation is fairly well covered in the summary section above.  There is not much interpretation presented.  Most of the findings, and even the summary, involve reporting aggregate information resulting from the survey compilation. Interpretation of the results does take the form of pointing out the statistics that the authors found most significant.  Otherwise interpretation consists of basically two points, that recruitment should be done through library schools and effective recruitment must include awareness programs, funding, working with large urban and other major libraries, and lowering attrition rates.  These remarks are supported by the study and the literature.  The study did leave out an important dimension.  It looked at why minorities entered the profession.  Reciprocally it might have tried to examine why minorities do not enter the profession.  This may have more significance and shed additional light although, based on my own conversations with minority students, it may be a hard pill to swallow.  Competition, status, prestige, and salary may keep minorities away.  Why, with so many other lucrative professions available and begging for minority students in their programs, should minority students chose librarianship?  In reporting on comments made on the survey this point was made, "recruitment would be easier if librarianship had a better image and salaries were higher".  It deserves much greater attention because without changing the image of the profession, the other measures will have minimal results, as the literature indicates has been the case in the past.

The interpretations put forward in the study by Williams are consistent with the thorough and objective quality of the entire work.  The analyses leads reasonably to the interpretations.  He does make some assertions that appear to take the evidence a bit too far but then will qualify those assertions, placing them in proper perspective.  The study presents a "theoretical framework".  The interpretation addresses how well the evidence supports this framework. Therefore he does address the questions asked.  With strong adherence to research methods the interpretation is fully supported resulting in its credibility.

Because the historical article are Oregon's public libraries is a qualitative work, the whole work may be considered an interpretation.  As such it comes across very definitively.  Therefore one must be aware that they are reading an interpretation and not necessarily the facts.  Yet the interpretation is supported well with the evidence provided. When combined with the consistency in the interpretation and the reporting of the events, I found the work to also be quite credible.

Summary - "Recruitment of Librarians Into the Profession: The Minority Perspective"

Overall I judge this article to be no more than adequate as a example of social science research.  It contains the proper elements.  It uses a common accepted statistical tool, chi square tests, to establish correlations.  Yet the instrument, a survey, introduces questions concerning sampling bias and potential biasing of the survey answers by the way the questions are worded.  In its results section it identifies the correlation findings and presents the statistics.  It does not draw unwarranted conclusions based on the correlation results.  Its evaluation section focuses on the comment section of the survey, introducing salient comments made by respondents.  Such comments may be relevant but they detract from the article in terms of being empirical research.  The article provides information that may be valuable to the profession in improving minority recruitment and retention but in term of research, it is lightweight.    

Summary - "Public Library Development in the United States, 1850-1870: An Empirical Analysis"

Overall I give this article high ratings, particularly in its attempt to follow research methods as they were presented in this course.  It was found to be weak in the background material.  It was brief, definitions were limited, and the prior research was discussed "elsewhere".  Its references were objective considering non-supportive evidence.  The issues concerning the problems with validity of the data are fully discussed.  It does what it can to utilize the data in such a way as to render it valid.  Appropriate statistical tools are employed.  The examination of seven independent variables may be questionable. Interpretations and conclusions are reasonable.  In drawing conclusions the author does not claim that the study resolves the questions studied.  The conclusion also points to further research.  It is not a highly sophisticated study but is relatively rigorous in its approach and therefore a good article to examine for Research Methods I.

Summary - "Oregon's Public Libraries during the First World War"

This article was a pleasure to read.  It captured my interest for several reasons.  It had structure, grouping the libraries' self-perceived wartime responsibilities into three categories.  It then proceeded to discuss those in order.  At times the article did digress from the main themes as when it spent more than two pages on the Louise Hunt episode.  It was flowed well otherwise.  Being from the Northwest, knowing some of its history (in general and as it pertains to library development), and being familiar with the geography and place names, it is difficult to render an objective opinion of this article.  I found it to be credible based on my background and the substantive reference material.  In addition I will retain the information found in this article far longer than that in the other articles.  Besides presenting good research that must be a major objective of written research, to draw interest. 

References

Campbell, Donald, and Stanley, Julian. Experimental and Quasi Experimental Designs for Research. Houghton Mifflin Co., 1963.

Katzer, Jeffrey, Cook, Kenneth H., and Crouch, Wayne W. Evaluating Information A Guide for Users of Social Research Science. 2nd ed. Reading, MA.: Addison-Wesly Publishing Co., 1982.

Tripodi, T., Fellin, P., and Meyer, H.J. The Assessment of Social Research. 2nd ed. Itasca, Ill.: FE Peacock Publishing, 1983.


Return to top

Steve Varga
Comments/etc.?

varga@u.arizona.edu


This page last Updated: Tuesday, 21-Nov-2000 15:49:04 PST