Ranking and reality--We're number one!
From PLN
Ranking and reality--We're number one!
|
Holt's Perspectives
by Glen Holt, published March 9, 2006
Drive through the streets of my city right now and look at the banners tied to the facades of its libraries. The banners say,
“HONORED TO BE RANKED NO. 1”
Like our daily newspaper (which announced the ranking in a 2-inch, single-column note), our regional business journal (which carried a nice summary article) and so many other citizens who hold high regards for our city’s library, I say, “Congratulations.”
Having completed this honorific act, I now ask the questions that ought to be asked of every statistical claim: “By whom?” “For what?” “How?” And, finally, “With what consequences?”
By whom?
The person who bestowed the ranking is Dr. John W. Miller, President of Central Connecticut University. Miller holds a bachelor’s degree in journalism from Ohio University (1969), a master’s degree in education from Northern Illinois University (1972) and a doctorate degree in education from Purdue University (1975).
For what?
Miller’s statistical exercise is intended to rank “literate cities.” He initiated the system when he served as chancellor at the University of Wisconsin-Whitewater. Two years ago, St. Louis libraries ranked third. Dr, Miller ranks the City of St. Louis the 15th most literate city among all 69 US urban places with populations over 250,000. For someone who once taught American urban history and urban studies at the university level, that claim is only a little short of the miraculous.
Here are a few statistical realities: The City of St. Louis has lost over half its population since the end of World War II (declining from over 850,000 to under 350,000). Moreover, it has a high incidence of functional literacy (the State of Missouri counts over 40% of the adult population as workplace “literacy problems”) and a growing number of new immigrants (15 percent in 2000 and projected to rise to 23% by 2010) who can neither speak nor write the language, The City of St. Louis also has a public school system that has had to be saved from shambles any number of times since the end of World War II. With such population and educational realities, there can be no doubt that the number one “libraries” ranking helped the city win its place in the upper quartile of America’s literate cities.
How?
Miller’s earlier ranking scored the City of St. Louis as the nation’s 3rd-most “literate city.” And “Number 1” in the “libraries” category in his newest report on America’s Most Literate Cities, 2005, including the sources of the statistics for the rankings.
The methodology that Dr. Miller uses (multivariate analysis) is simple enough to make it easy to understand, although like Tom Hennen’s “best public library” rankings, the actual weighting formula for Miller’s rankings remains unreported. By focusing on statistics for the “We’re number fifteen!” city of St. Louis and the “We’re number one” claim of the St. Louis Public Library, we can get an idea of what goes into conducting this ranking system.
Level of the community's educational attainment
Like so many other old and heavily segregated cities, St. Louis did poorly in this ranking, placing 58th out of a total of the 69 cities included in the count.
For social statisticians, the poor rank could be predicted from the statistics that Miller used. These are: “Percentage of the adult population with an educational level of 8th grade or less; percentage of the adult population with a high school diploma or higher; percentage of the adult population with a bachelor's degree or higher.”
In the United States, educational attainment is a function primarily associated with family income. Since lots of St. Louis citizens are poor (over 85 percent of its children are eligible for subsidized school lunches) and its public education system hovers close to losing its accreditation, we should expect relatively low educational attainment.
Bookseller activity
St. Louis tied for 7th with Portland, OR, on this indicator. This index number includes the number of retail bookstores, rare/used bookstores and independent members of the American Booksellers Association all counted per 10,000 population. No specialty, adult or religious bookstores were counted.
I’ve never thought of St. Louis as particularly strong in bookstore numbers, but then lots of bigger cities appear to have fewer and smaller bookstores. If you want a rough sense of the significance of bookstores in your city, you might want to do a Yellow Pages directory count of bookstore numbers against the number of sporting goods stores, golf courses or any other discretionary retail activity
Newspaper circulation
St. Louis ranked 8th among all the cities in Miller’s literate cities comparison. Here is his statement about the source of the figures.
For the newspaper database, the daily and Sunday circulation figures for the March 2005 reporting period from the Publisher’s Statements were obtained from the Audit Bureau of Circulations website. Those numbers were divided by the 2004 U.S. Census population figures for the cities resulting in a ratio, which was then rank ordered.
If you want to read something that is more than a little bit scary from a statistical standpoint, take a look at the Audit Bureau of Circulation website, and see how newspaper folks transform a few hundred thousand counted circulation into a million by making critical assumptions about second-readers and pass-along readership. Still, the nature of the count is fixed which is not true for a lot of booster-style indices.
Miller minimizes the importance of this indicator, however, when he writes, “Newspaper circulation variables correlate with nothing other than themselves. There is virtually no relationship between number of papers circulated per person & any of the other literacy factors, including reading a newspaper on the internet.”
Periodical publications
St. Louis ranked 8th. This index number is based on statistics drawn from the web edition of the National Directory of Magazines (2004-2005) for magazines and the Standard Periodical Directory 2005 for journals. St. Louis’ history served it well in this category. Through the 19th century, St. Louis' economy boomed. Many manufacturers published their catalogs and trade journals in St. Louis as did religious organizations and missionary societies, and it still is a major printing center for everything from Sunday school weeklies to Hustler magazine. The community also has a good many not-for-profit headquarters, both national and regional, and many of these issue periodicals as well.
Internet Resources
This measurement was a new one added in 2005. St. Louis ranked 37th along with Fort Worth, TX. Dr. Miller derived this index number from four variables: “Number of library Internet connections per 10,000 library service population; number of commercial and public wireless Internet access points per capita, number of Internet book orders per capita and the percentage of adult population that has read a newspaper on the Internet.”
Note that a St. Louis Public Library measurement, “Number of library Internet connections per 10,000 library service population,” is included in weighting this category. The figure is not counted in the libraries category, discussed next. St. Louis Public Library, bolstered by two Gates Foundation grants, has several hundred computers tied to the Internet. Its figures undoubtedly pulled up the City of St. Louis ranking in this category.
Libraries
Miller ranks the City of St. Louis “libraries” number 1. This high rank is based on four numbers: “Number of school media personnel per 1,000 public school students; number of branch libraries per 10,000 library service population; volumes held in the library per capita of library service population; number of circulations per capita of library service population; number of library professional staff per 10,000 library service population.”
Since these indicators are about libraries, they are worth special attention.
First, “the number of school media personnel per 1000 public school students” is exactly that, a number. That number does not reflect the richness or poverty of school media center collections, whether or not the collections are accessible, or the involvement of the “school media personnel” in the instructional and study process of students. More than one critic has complained about the marginalization of media personnel in the St. Louis Public School’s student instructional lives, the lack of paper materials, the lack of access to electronic databases and the lack of Internet connections in public and private schools.
Second, The branch library variable is based entirely on the number of branches. With little trouble, we could find a branch planning book or article that suggests that St. Louis has too many branches for its population, with many branches too small (including a couple of mini-branches that do little more than circulate hot books). In Miller’s criteria, this situation is a plus.
Third, SLPL’s volume count is as much an accident of history as a feather in its cap. The system originated in 1865. Like a lot of older systems, this one has a lot of old books. Through much of the past decade, professional staff has done lots of weeding; more still needs to be done. Because the collection is what it is--still lots of old books that would have a better life elsewhere and university-research quality in a few subject areas)--SLPL’s turnover rate, often used as a sign of good library management, is lower than lots of other public libraries. In Miller’s criteria, this situation is a plus.
Fourth, circulations for capita is another service measure that Miller uses. This measure for SLPL is a good one reflecting institutional service priorities. Since the late 1980s, the library staff at this library has worked hard to increase circulation – and the work shows. As in most library ranking systems, in Miller’s ranking scheme, it is a plus.
Fifth, the number of library staff per 10,000 persons is the final variable that Miller cites for his “libraries” category. As with branch numbers, we could find literature that says that SLPL should have more professional librarians, and we could find many directors, especially those who operate high-circulation libraries, who would suggest that it should have fewer professional librarians. The number here, I believe is one of choice. The governing board of the library has made a determination that it wants to retain its reference capability at its Central Library. So, is the relatively high number a good thing or one that’s bad within this library’s resource context? In Miller’s criteria, this staffing situation is a plus.
So, given the composition of the numbers, and their intended purpose within Miller’s “literate cities” index, St. Louis “libraries” do rank number one. And, St. Louis Public Library can hang signs that say “HONORED TO BE RANKED NO. 1.”
Having created a “literate cities” ranking that makes front-page headlines in many cities and coverage in “national” newspapers like USA Today, Miller does the right academic thing in his introduction when he qualifies his own computations. He writes:
While other factors--SAT or mastery test scores, for example--can provide useful information, this set of factors presents...the extent and quality of the long-term literacy essential to individual economic success, civic participation, and the quality of life in a community and a nation. Because of the number and complexity of the variables, because of the variety of ways in which the resource data are gathered and presented, and because of the variability in the timeliness of the data, the ranking is necessarily an interpretation (emphasis added). The value of this study, I believe, lies less in the absolute accuracy of the rank orders and far more in what communities do with the information...
That last comment from Miller, leads to asking the question that we ought to ask of all statistics:
With what consequence?
Americans love statistics. American librarians love statistics even more. Look at the numbers of numbers that they create daily, weekly, monthly, and locally, state-wide, regionally, nationally and internationally. The US library community is inundated by statistics. Most of us admit that they don’t know much about them, but we love them just the same. Especially when they rank us in a high position in some category we care about. And, most especially, when they show us that “we’re number one” in anything. Whether the ranking comes from ALA, Tom Hennen, or Dr. Robert Miller we fall into chanting our “we’re number one” ranking for the same boosterish pride.
Once the echoes from the cheers die down, however, there are more serious considerations. What are our libraries doing to see what our current and potential customers want and need? Do we have the resources we need to deliver those services? Can we make that delivery effectively and efficiently? And, what benefits flow from our expenditure of resources and our thought and work?
To paraphrase Miller, “The value of our study lies less in the accuracy of our rankings than in what we as library professionals do with the information.” And, more specifically, what will we do with such information to make our libraries better? And, perhaps of most consequence, “What part do libraries play in creating and sustaining literate cities?” And, if we believe that they do, how should we measure that reality?
Questions like these and their honest answers, refined and restated again and again, mark a continuing renaissance that has to recur over and over as we adapt our libraries to meet the conditions of an always changing future.
Related articles
- Citizens, engage! - Jamie LaRue discusses civic engagement, central to the reality of a great public library.
- Managing number one - Frank Hermes discusses management at another public library ranked Number One.
Your turn: Talk about it
Are there disadvantages to being ranked highly? That's the focus of a discussion in the Marketing forum (click on Ranking and Reality - We're Number One !). What's your experience? Join the discussion
