| 10% |
The Employer Reputation component is unique amongst current international evaluations in taking into consideration the important component of employability. The majority of undergraduate students leave university in search of employment after their first degree, making the reputation of their university amongst employers a crucial consideration.
A common approach to the evaluation of employability in domestic rankings is graduate employment rate, there are two reasons why this indicator does not work at an international level – the first is that this evaluation looks at the top universities in the world – all of whom have very high employment rates – so it doesn’t provide very much discernment. The second is that, since we are looking at different countries, the results would react to local economic conditions and not necessarily just the quality of the institution. So, instead, we survey employers to ask their opinion on the quality of graduates.
Source of Respondents
- Previous Respondents
QS has been conducting this work since 2004 – all previous respondents to our survey are invited to respond again to provide us with an updated viewpoint on the quality of universities in their broad field. - QS Databases
In twenty years of operation QS has developed an extensive database of employers in key markets worldwide. - QS Partners
QS has an extensive network of partners including international media organisations and job portals, number of whom support our employer research by distributing survey invitations - Institution Supplied Lists
Since 2007, institutions have been invited to submit lists of employers for us to invite to participate in the Employer Survey. In 2010, that invitation was extended to lists of academics also. Since employers are encouraged to list a number of institutions, the risk of bias towards the submitting institution is minimal, nonetheless submissions are screened and sampling applied where any institution submits more than 400 records. In 2011, over 200 institutions supplied lists contributing over 60,000 employer contacts.
The Survey
The QS Employer Survey has been running since 1990 and contributes to a number of key research initiatives operated by the QS Intelligence Unit including the QS TopMBA Salary & Recruitment Trends Report and the TopMBA Global 200 Business Schools. Like the academic survey the questionnaire is adaptive responding to the early questions to take respondents through the MBA, Masters or First Degree tracks as appropriate. The key sections for the Rankings work as follows:
- Personal Details
Name, Company, Job Title, Department, Sector, Recruitment Responsibility - Knowledge Specification
Country – respondents are requested to tell us which country they have most familiarity with rather than the country where they are based. This enables new international faculty members to comment on their sphere of knowledge rather than speculate on an area they may yet know little about.Region – regional knowledge responses are grouped into three supersets that define the list of institutions from which the respondent can select, these are Americas; Asia, Australia & New Zealand; and Europe, Middle East & Africa - Top Domestic Institutions
Respondents are asked to identify up to ten domestic institutions they consider best for research in each of the faculty areas selected in Section 2. - Top International Institutions
Respondents are asked to identify up to thirty international institutions they consider best for recruiting graduates. The list consists solely of institutions from the region(s) with which they express familiarity in section 2. - Additional Information
We use this section to gather additional information from respondents, such as recruitment priorities, volumes and preferences.
Response Processing
Sadly, the work is not done once the survey is designed and delivered. Once the responses are received a number of steps are taken to ensure the validity of the sample.
Three Year Aggregation
To boost the size and stability of the sample, QS combines responses from the last three years, where any respondent has responded more than once in the three year period, previous responses are discarded in favour of the latest numbers.
Junk Filtering
Any online survey will receive a volume of test or speculative responses, people filling out garbage just to see what appears on the next screen for example. QS runs an extensive filtering process to identify and discard responses of this nature.
Anomaly Testing
It is well documented on the basis of other high-profile surveys in higher education that universities are not above attempting to get respondents to answer in a certain fashion. QS run a number of processes to screen for any manipulation of survey responses. If evidence is found to suggest any institution has attempted to overtly influence their performance, any responses acquired through source 4.
Results Analysis
- Devise weightings based on the regions with which respondents consider themselves familiar – weightings are (now) based only on completed responses for the given question. This is slightly complicated by the fact that respondents are able to relate to more than one region.
- Derive a weighted count of international respondents in favour of each institution ensuring any self-references are excluded
- Derive a count of domestic respondents in favour of each institution adjusted against the number of institutions available for selection in that country and the total response from that country ensuring any self-references are excluded
- Apply a straight scaling to each of these to achieve a score out of 100
- Combine the two scores with a weighting 70% international, 30% domestic – these numbers were based on analysis of responses received before we separated the domestic and international responses three years ago, but a low weighting for domestic also reflects the fact that this is a world university ranking.
- Square root the result – we do this to draw in the outliers but to a lesser degree than standardization would achieve – our intention is that excellence in one of our five areas should have an influence, but not too much of influence
- Scale the rooted score to present a score out of 100 then standardize relative to the sample of institutions being used in any given context
Isn´t be possible to have a survey worldwide with the PhD/pos doc advisors asking them about the performance (creativity, knowledge, scientific formation etc) of their students from abroad? Their opinion could also be considered as an indicator of the quality of the institution where the student got it undergraduate or graduate courses.
It’s a thought – although identifying channels to attract sufficient response could be a challenge – we will put it to our Advisory Board for consideration.
I think the ranking would be more precise if you considered the size and the areas of activity of the companies where the employers work, by giving more weight to large multinationals. Large multinationals not only tend to recruit from the best universities, but they also recruit more internationaly. Therefore, the universities chosen by someone from, for example, McKinsey or BCG, would be more representative of the world top universities than the ones chosen from an employer of a mid-sized american campany.
Correct me if I am wrong, but in your methodology, even if you give a weight of 70% to recomendations coming from international employers, if for exemple an employer only chooses USA universities, only those will gain more points. This then leads to coutries with more employers responding having better represented universities.
Other sugestion to correct this would be asking the employes to choose a minimum number of universities from a minimum number of foreign coutries ou asking to pick only international universities.
I make this comment because I feel the the employer reputation ranking is a bit biased towards the USA and english sepeaking countries, where most employers responses come from, while some universities known for being very reputable among top employers, such as Ecole Centale Paris and other french grande ecoles, many indian ITT’s, top chinese universities from the Project 985, and many other top universities from other coutries (Note that I am not considering every country, as it is unrealistic to place in the world top universities, universities from 3rd world coutries. I am talking about very good universities from Europe and other highly developed areas such as Japan, India, China, South Korea, etc.) do not appear in the ranking or appear lower than expected.
We collect individual responses, not corporate responses. Thus large companies with a global footprint inevitably carry more weight as we generally get more responses from them. An “international” employer is defined as an employer contact operating in a country other than that where the institution is based – not an employer who happens to recruit internationally as well as the institution on whose science park they have premises. For example, a PricewaterhouseCoopers contact based in London would count as international if the select Harvard but domestic if they select Cambridge – similarly in reverse.
Nonetheless the results to tend to come in strongly for top British and US institutions, partly this is perhaps a historical effect and also connected with the international portability of graduates – with English being a major advantage to institutions in the US, UK, Canada, Australia, Singapore, Hong Kong and others.
You might say that we are not just trying to identify the world’s best universities but also the best “world universities” – those universities that have a global impact far beyond their own local and domestic communities.
Another note, this time concerning the ranking in general.
I think the general ranking would be better if you give more weight to the employer reputation as opposed to criteria regarding research and citations.
Usualy most students select their universities by job-related factors, such as prestige of the university, salary potential, top companies recruting there, connections with industry and internships, etc. and the quantity of research done in the university is only the main factor to consider when a student is applying to a PhD or a master degree intended to continue studies or going to R&D.
A possibility for the future. We recognize that different readers have different priorities and are working on an interactive solution where users can assign their own weights and devise a personalized table – see http://www.topuniversities.com/scorecard for an early pilot version.
Recently the New York Times published this ranking of the employers preference:
http://www.nytimes.com/imagepages/2011/10/20/education/20iht-SReducEmploy20-graphic.html?ref=education
It may be a bit biased towards buisness schools, but, with all respect, seems to me more representative and accurate of the world top universities than, at least according to employers preference.
All a matter of opinion of course. The list looks interesting enough, but its not just biased towards business schools but also, inevitably towards the 10 countries from which respondents were sourced. There are no responses from Asia, none from Spanish speaking Latin America, none from Australia or Russia, none from the Middle East or Africa and none from India or China. Furthermore, the respondents are exclusinely chief executives and chairmen – who have an important view – but one that in many cases is not drawn from recent hands on experience of recruiting. This list draws on “hundreds” of respondents from 10 countries, ours from almost 17,000 from over 100 countries.
In my experience, people will generally favour the list or table that most closely reflects their own perception of reality and not worry themselves with response rates, response levels or methodology. This list is definitely interesting, but not interestingly definitive.