Research results raise concerns

 Frank Larkins at the University of Melbourne. Picture: Aaron Francis Source: The Australian

LESS than one-third of universities were at world standard in some of the disciplines evaluated in the recent audit of research quality.

But analyst Frank Larkins's crunching of Excellence in Research for Australia results also revealed , in six of the 25 disciplines, 80 per cent of universities were world standard or above, and in 18 disciplines 50 per cent were at or above world standard.

"The areas that would worry us most are those [in] social sciences because that's where we have education, economics, cultural studies, commerce management and tourism," Professor Larkins said.

Professor Larkins said if the world benchmarks against which these below par performances were judged were sound, it was a worrying outcome, given the significant amount of undergraduate teaching in them.

Professor Larkins acknowledged the magnitude of the task of assessing research across 25 disciplines and many more fields for all universities, but was critical of the Australian Research Council's failure to publish world benchmarks against which the research was measured.

In an article in the LH Martin Institute newsletter today, he argues this has compromised the federal government's audit of research quality.

But ARC chief executive Margaret Sheil said a copy of the data was kept in the office of every deputy vice-chancellor, research, in the country. "We used citation benchmarks from [bibliometrics company] Scopus, but to publish them would have cost us extra money," Professor Sheil said.

She also said the benchmarks were not the only piece of information considered by the research evaluation committees but, had the ARC released them, people would "not [have looked] at anything else".

As such, the strategy not to publish was part of maintaining the integrity of the ERA process. Further, citation benchmarks were not used in every discipline and, where they were, they were moderated by expert panels.

Professor Larkins estimates the costs of the ERA process so far to be $100 million, including outlays from the government, universities - staff time and infrastructure - consultants and software development.

Results from the first round of the ERA audit were published in January, with research evaluated on a scale ranging from five - well above world standard - to one, well below world standard.