Background

The QS World University Rankings® have been in existence since 2004 and have become the world’s favourite reference for comparing international higher education institutions at an overall and, to a degree, faculty area level. In that time they have evolved considerably but since 2007 have maintained a stable methodology and yielded increasingly robust and insightful results.

There is, however, a clear demand for ways to compare the effectiveness of institutions in narrower subject disciplines – after all many prospective international students know first WHAT they want to study before asking the question of WHERE they want to study.

At a holistic level, any ranking that has ambitions to be truly global is limited by the availability of data from every part of its scope – this is why, in general, many indicators of university quality commonly used in domestic rankings are not included in any of these exercises. While the depth of data available from the UK, Australia and the US may be exemplary it is yet to be matched by that in India, Greece or Brazil, for example.

When attempting to exercise evaluations at a more granular level this becomes even more complex – while it may be reasonable to expect a university to have a decent understanding of its average faculty-student ratio, to break that down by faculty or department is difficult in even the most advanced cultures of data provision.

There are, however, some indicators that transcend the direct involvement of the institutions and can be better stratified by subject discipline. This page outlines the QS approach for doing so, which has been used to produce the new QS World University Rankings® by Subject.

Subject Disciplines Considered

Based on natural groupings, response levels and expert advice the following 52 subject disciplines have been identified for consideration. Those highlighted are those that have reached appropriate thresholds for publication.

 

Indicators

Gathering data for the QS World University Rankings® is hard work. Not every university is as forthcoming or well-equipped as the next. Since 2004, QS has refined the process, hired a multi-lingual, multi-cultural team, tweaked definitions and become increasingly effective and successful in gathering, verifying and analysing the data we need. We frequently review the feasibility of gathering more information and providing users with additional analyses to help them make more informed choices.

At a subject level this gets more difficult. Even the most well set up university will struggle to give us good data on faculty and student numbers by subject area in a way that will be compatible with the next, and the workload for universities that are not set up so well would be prohibitive.

QS already has datasets that enable us to drill down by subject area, namely our academic and employer reputation surveys and the Scopus data we use for our Citations per Faculty indicator in the global rankings. These have been combined to produce our subject results.

Academic Reputation (Academic)

Academic Reputation has been the centrepiece of the QS World University Rankings® since their inception in 2004. In 2010 we drew upon over 15,000 respondents to compile our results. The survey is structured in the following way:

Section 1: Personal Information
Respondents provide their name, contact details, job title and the institution where they are based

Section 2: Knowledge Specification
Respondents identify the countries, regions and faculty areas that they have most familiarity with and up to two narrower subject disciplines in which they consider themselves expert

Section 3: Top Universities
For EACH of the (up to five) faculty areas they identify, respondents are asked to list up to ten domestic and thirty international institutions that they consider excellent for research in the given area. They are not able to select their own institution.

Section 4: Additional Information
Additional questions relating to general feedback and recommendations.

A thorough breakdown of respondents by geography is available in the methodology section of our main rankings on our website http://www.topuniversities.com/university-rankings/world-university-rankings/methodology/academic-reputation-index

As part of the 2010 QS World University Rankings®, respondents were asked to identify universities they considered excellent within one of five areas: engineering and technology; natural sciences; life sciences and biomedicine; arts and humanities; social sciences and management. The results of the academic reputation component of the new subject rankings have been produced by filtering responses according to the narrow area of expertise identified by respondents.

The threshold for academic respondents that any discipline must reach for us to publish results in that discipline has been set in year one at 150. As responses build over time, new subjects from the above list may qualify.

The number of academic respondents considered for each qualifying discipline is outlined in the table on the following page. As with the overall tables, our analysis places an emphasis on international reputation over domestic – domestic responses are individually weighted at half the influence of an international response. This is a global exercise and will recognize institutions that have an international influence in these disciplines. As in the main QS World University Rankings®, weightings are also applied to balance the representation by region.

Employer Reputation (Employer)

The QS World University Rankings® are unique in considering employability a key factor in the evaluation of international universities and in 2010 drew on over 5,000 responses to compile the results for the overall rankings. The employer survey works on a similar basis to the academic one only without the channelling for different faculty areas. Employers are asked to identify up to ten domestic and thirty international institutions they consider excellent for the recruitment of graduates. They are also asked to identify from which disciplines they prefer to recruit. From examining where these two questions intersect we can infer a measure of excellence in the given discipline.

Of course, employability is a slightly wider concern than this alone would imply. Students from many disciplines may not be focused on careers directly related to that discipline. Many engineers become accountants and few history students wind up pursuing careers closely related to their program. On this basis, employers citing a preference for hiring students from “Any discipline” or from broader category areas are also be included in subject score, but at a lower individual weighting.

It is our view, based on focus groups and feedback from students, that employment prospects are a key consideration for prospective students when choosing a program and a university – regardless of whether or not they envisage their future working in the discipline they choose to study.

Employers seeking graduates from any discipline are weighted at 0.1 and those from a parent category (i.e. Social Sciences) are weighted at 0.25 relative to the weight of a direct response for the subject area.

The below table shows the total number of employers contributing to our employer index in each of the corresponding disciplines. The similarities between the numbers recorded in each of the engineering sub-disciplines are down to the fact that employers were asked to comment on engineering in general rather than the specific sub-disciplines. A small number of respondents specified their preference through the “Other” option provided in the survey leading to the slightly different total for Mechanical Engineering. The threshold for including the Employer component for any discipline is 300.

Figure 2: Employer counts by subject discipline

Subject	Employer Count
Performing Arts	300
Linguistics	300
Architecture / Built Environment	303
History	311
Geography & Area Studies	315
English Language & Literature	316
Philosophy	318
Agriculture & Forestry	337
Metallurgy & Materials	337
Environmental Sciences	338
Earth & Marine Sciences	343
Art & Design	346
Medicine	346
Physics & Astronomy	353
Biological Sciences	356
Modern Languages	391
Sociology	410
Statistics & Operational Research	412
Chemistry	430
Education	438
Psychology	462
Mathematics	473
Politics & International Studies	492
Communication, Cultural & Media Studies	553
Law	637
Economics & Econometrics	757
Accounting & Finance	816
Business & Management Studies	859
Computer Science	867
Engineering – Civil & Structural	936
Engineering – Chemical	936
Engineering – Electrical & Electronic	936
Engineering – Mechanical, Aeronautical & Manufacturing	937

As with the overall tables, our analysis places an emphasis on international reputation over domestic, with domestic responses carrying half the individual weighting of international responses. This is a global exercise and recognizes institutions that have an international influence in these disciplines. A weighting is also applied to balance representation by region.

Citations per Paper (Citations)

In the overall QS World University Rankings® we use a measure of Citations per Faculty.  This has some advantages in that it does a good job of taking into account the size of an institution yet allows us to penetrate deeply into the global research landscape. At a discipline level it is impractical to reliably gather faculty numbers broken down by discipline so we need to revert to a Citations per Paper model – this will require a minimum publication threshold to be set dynamically to avoid anomalies in the results.

Papers in Scopus are tagged with an ASJC (All Science Journal Classification) code which identifies the principal foci of the journal in which they were published (multidisciplinary journals are excluded). When aggregated together these totals and their associated citations provide an indicator of volume and quality of output in the given discipline.

One of the advantages of the “per faculty” measure used in the main rankings is that a small number of papers, achieving a high level of citations, has limited impact due to the divisor. Conventionally in citations per paper analysis, a paper threshold is required to eliminate anomalies. Of course publication patterns are very different in different subjects and this needs to be taken into account both in terms of the thresholds that are used and the weights applied to the citations indicator.

Figure 3: Paper counts and thresholds by subject discipline

Subject	Paper Count	Paper Threshold
Art & Design	0	0
Modern Languages	0	0
Performing Arts	2,266	0
English Language & Literature	5,358	0
Philosophy	6,659	10
Accounting & Finance	8,083	10
History	8,358	10
Linguistics	8,511	10
Politics & International Studies	9,568	10
Law	16,296	20
Sociology	25,868	20
Communication, Cultural & Media Studies	33,077	20
Education	34,056	20
Geography & Area Studies	35,495	20
Economics & Econometrics	37,761	20
Architecture / Built Environment	43,156	30
Engineering – Civil & Structural	43,222	30
Statistics & Operational Research	52,533	30
Business & Management Studies	62,889	30
Psychology	134,377	50
Engineering – Chemical	240,831	60
Mathematics	268,074	60
Engineering – Mechanical, Aeronautical & Manufacturing	341,568	70
Earth & Marine Sciences	345,517	70
Environmental Sciences	354,694	70
Agriculture & Forestry	440,739	80
Chemistry	458,693	80
Engineering – Electrical & Electronic	514,395	90
Metallurgy & Materials	642,831	100
Physics & Astronomy	749,862	110
Computer Science	1,211,631	140
Biological Sciences	1,488,346	150
Medicine	1,810,668	170

The above table lists the subjects we will be working with as identified based on strength of response to the academic and employer surveys. The “Paper Count” column is a total of all of the papers in the discipline that we have been able to attribute to one of the 1,000+ universities that we have mapped into the Scopus database – this provides an indicator of the “size” of the overall research output in the discipline. The “Paper Threshold” represents the number of papers (based on a square function) in each discipline that an institution must have published in the last five years in order to qualify for our tables in that subject.

There are certain subjects in which academic publications are not a feasible or appropriate measure of academic output – these subjects have either zero or a low number of papers in Scopus and are denoted in the above by a Paper Threshold of 0. Any discipline must have at least 6,000 papers identifiable in the table above for us to include the Citations indicator in the table.

There are a few other clarifying points regarding our treatment of Scopus data – most, if not all, of which also apply to our analysis for the forthcoming cycle of regional and global rankings:

1. Our analysis is based on an extract from Scopus (custom data) and not on the live database in order for us to be drawing on a consistent dataset within each cycle of research. We receive this in Feb/Mar of each year. As the Scopus database evolves the two diverge – so expect to be able to find similar numbers rather than exact matches.
2. The window for both publications and citations is five years from 2006 to 2010 inclusive
3. Self-citations are excluded from all citation counts
4. Multidisciplinary publications do not contribute towards counts for any subject discipline (although they do if you run a search in Scopus so be sure to edit your search query if you are trying to verify our numbers)
5. All affiliations we know about are considered. Universities are invited to inform us of hospitals laboratories and schools affiliated to them.

Adaptive Compilation

The publication of a given subject table is not dependent on all three indicators reaching their thresholds. In most cases, we require a minimum of two indicators in order to present a final list, though exceptions have been made as appropriate in subjects where data levels are lower. Moving forward, as we gather larger responses to our surveys more disciplines will hit the thresholds and qualify for publication, leading to rankings in a greater number of subjects emerging as time goes by.

Weightings are not applied evenly between indicators for different disciplines, but will be set relative to the pertinence of the indicator to the discipline and the depth of data available to evaluate it. The chart below provides a schematic of current proposed weightings by indicator for each discipline (to be confirmed). Weightings are based on publications patterns and level of employer interest in the given subject area – all subjects have attracted solid academic response levels. Subjects have been grouped into seven clearly identifiable permutations.

Final ScreeningAcknowledging that surveys can sometimes produce anomalous results – i.e. recognition for a given institution in a discipline in which they neither operate programs nor research – QS has conducted a final screening to ensure listed institutions are active in the given discipline. For research active disciplines (those with a red bar at 20% or higher in the above schematic) this is comparatively straightforward – we can simply exclude institutions that fail to meet a certain threshold. For the other disciplines QS has conducted a manual screening to ensure that the institutions each operate programs in the given area.AggregationSimilarly to the approach used in the overall QS World University Rankings® a z-score is calculated for each indicator with the results scaled between 0 and 100 and then combined with the weightings described in the above chart.Deployment Schedule & StrategyThe initial results will be released in batches throughout April and May 2011. Results will be published on topuniversities.com, and will contribute to interactive scorecards available for undergraduate students on www.topuniversities.com and for Masters and PhD students at www.topgradschool.com. Tables will be licensed for publication to existing QS partners. More information is available from the QS Press Office – qspressoffice@qs.com

Academic Employer Citations

English Language & Literature	90% 10%
Performing Arts	90% 10%
History	80% 10% 10%
Linguistics	80% 10% 10%
Philosophy	80% 10% 10%
Sociology	80% 10% 10%
Art & Design	70% 30%
Modern Languages	70% 30%
Architecture / Built Environment	70% 10% 20%
Geography & Area Studies	70% 10% 20%
Statistics & Operational Research	70% 10% 20%
Law	60% 30% 10%
Agriculture & Forestry	60% 10% 30%
Earth & Marine Sciences	60% 10% 30%
Environmental Sciences	60% 10% 30%
Metallurgy & Materials	60% 10% 30%
Accounting & Finance	50% 40% 10%
Communication, Cultural & Media Studies	50% 40% 10%
Politics & International Studies	50% 40% 10%
Education	50% 30% 20%
Economics & Econometrics	50% 30% 20%
Engineering – Civil & Structural	50% 30% 20%
Psychology	50% 30% 20%
Chemistry	50% 20% 30%
Physics & Astronomy	50% 20% 30%
Biological Sciences	50% 10% 40%
Medicine	50% 10% 40%
Business & Management Studies	40% 40% 20%
Mathematics	40% 30% 30%
Engineering – Chemical	40% 30% 30%
Engineering – Electrical & Electronic	40% 30% 30%
Engineering – Mechanical, Aeronautical & Manufacturing	40% 30% 30%
Computer Science	40% 30% 30%