|
MBAUniverse.com
B-school Survey 2011-12
Research
Methodology Note
December 6,
2011
Any research
output is as credible as the methodology and
data behind it. Understanding that a very
large number of candidates will base their
critical decision with respect to selection
of B-school for taking admission on MBAUniverse.com
B school rankings, we were very sensitive
towards both the aspects: Methodology and
Data.
We started
the 2nd annual MBAUniverse.com B-school Survey
2011-12 by seeking guidance and inputs from
our distinguished Advisory Body. Noted management
academician Padma Shri Dr Bakul Dholakia,
former Director, IIM Ahmedabad is the chairman
of the Advisory Body. The other eminent members
of the Advisory Body are Mr K. Ramkumar, Executive
Director, ICICI Bank and Dr RK Shukla, former
Director, NCAER-Centre for Macro Consumer
Research. Together, our advisory body covered
the critical domains of understanding the
process of delivering management education,
the corporate requirements, and how to conduct
a rigorous research.
Our advisory
body formulated the overall context &
mission for the survey -- to create a highly
credible B-school Ranking that helps the three
key stakeholders: MBA Students (current and
prospective), Recruiters and B-schools themselves.
While MBAUniverse.com,
with guidance from the survey advisory body,
created the overall framework for the survey,
the actual task of Data gathering and analysis
was given to a reputed research company -
Facts & Data .
RESEARCH
METHODOLOGY:
Most critical
aspect of any methodology is that it should
be based on data and hard evidence and should
be devoid of any kind of subjectivity whatsoever.
As ranking B-schools involves quantifying
a lot of qualitative information, it is very
important to ensure that any subjectivity
or bias is not allowed to creep in.
The
stakeholders in this eco system are:
(a).
Students
(b).
MBA aspirants (or prospective students),
as they take application decisions based on
these surveys;
(c).
Current students of various B-Schools
– as their placements depend on our ranking;
(d).
Past students of various B-Schools – as their
career prospects depend on our survey.
(e).
Recruiters (as they take recruitment decisions
based on our survey), further split by:
(f).
HR Heads and senior officials across leading
organizations;
(g).
Line managers across functional areas.
(h).
B-Schools (as their ranking decide the fate
of their current and prospective students,
placements, ability to attract good quality
faculty):
(I).
Directors and leadership community
a B-schools
(J).
Professors or the teaching community.
Across all
phases of the project, adequate precaution
was taken to ensure that the ranks are based
on data from all these stakeholders.
Step
1: Generation of parameters
The parameters
that make a good B-School should be decided
by all stakeholders of this eco-system, and
can neither be left at the mercy of the intellect
of the researcher OR on desk research. One
of the common problems of research has been
that those doing research believe that they
“know” the parameters – which, data suggests
is almost always incorrect. Keeping this in
mind, Facts ‘N' Data , the research partner
for this survey, conducted an extensive survey
wherein, an exhaustive (laundry) list of all
parameters that ‘ these stakeholders believe'
make a good B-School was drawn. The next obvious
challenge was to reduce this list to ensure
that correlated parameters were taken care
of. However, this by itself is a mathematically
tedious process!
Step
2: Selecting the most important parameters
Having identified
the attributes, while a much easy approach
is to ‘logically' reduce this list, based
on researcher's experience, common sense,
industry knowledge (or whatever we prefer
to call it), Facts ‘N' Data deliberately chose
a much more robust, though difficult path
to ensure that the rankings are the most credible
ones. Based on hard data and ranks of previous
surveys done by MBAUniverse.com , Principal
Component Analysis was used to reduce this
list to most important components that explain
maximum variance. Thus, data not ‘ex perience'
dictated the final list of parameters.
Step
3: Arriving at weights of parameters
Again, while
it's very easy to allocate weights to various
parameters that ‘make sense' or ‘appear logical'
or ‘are obvious', Regression technique was
used to arrive at weights of these parameters,
based on last year's data. Since context is
dynamic and things could have changed from
last year, all the stakeholders were asked
to state the weights to these parameters as
well.
The weights
arrived at, based on the regression of last
year's data were further fine tuned based
on the stated weights of all the three stakeholders,
giving equal weight to all of them at every
level. Not surprisingly, minor aberrations
were found in the two and data suggests that
these were because of dynamism of context
(change in preference year over year).
As a result
of this step, we were able to arrive at final
weights of various parameters.
So, data suggested
that final rank was a function of Input, Output
and Process parameters. It may be noted that
MBAUniverse.com was the first one to use this
methodology, which is clear from the published
material available in public domain for over
one year now. Not surprisingly, in 2011 many
others tried to copy this methodology.
The
Mathematical Equation at macro level was found
to be:
Final
Score^ (F) = f [Input Score^ (I), Process
Score^ (P), Output Score^ (O)] ,
Mathematical
relation being found to be:
F
= (0.30*I) + (0.39*P) + (0.31*O)
Where:
Input
Score^ (I) = f [Number of applications per
seat (n), Quality of intake as determined
by the candidate's academic history and entrance
exam cut-off scores (q), % of candidates with
professional degrees (p), cumulative past
work experience of last year's class (e)]
1
Mathematical
relation being found to be:
Input
Score^ (I) = (0.300*n) + (0.302*q) + (0.149*p1)
+ (0.249*e)
Process
Score^ = f [Faculty (f) 2 , Infrastructure
(i) 3 ]
Mathematical
relation being found to be:
Process
Score^ (P) = (0.847*f) + (0.153*i)
Output
Score = f [Output of students as determined
by their placement data (s) 4 , Academic Output
as determined by the number of Research Papers,
Consultancy assignments, and MDP's (a) 5 ]
Mathematical
relation being found to be:
Output
Score (P) = (0.721*s) + (0.279*a)
Explanatory
Notes:
1.
Gender diversity was NOT found to have an
influence on the quality of B-Schools
2.
Where, Faculty is the EFFECTIVE FACULTY
PER STUDENT, ‘Effectiveness' being determined
by sum of permanent faculty (not on leave
or deputation etc) weighted by the type of
Ph.D. (Central Universities IIT's, IIM's and
Tier I universities commanding a weight of
1.47, Select Foreign Universities commanding
a weight of 1.54, Other Universities at a
weight of 1.24), further weighted by the teaching
experience (ranging from 1.11 through 1.76),
further weighted by corporate or industrial
experience (ranging from 1.19 through 1.58),
supplemented by availability of Visiting Faculty
(defined as those who took at least one complete
paper in a semester) with weights ranging
from 0.65 through 0.71 depending on their
quality as determined by the quality of University
or institute that they belong to, their industrial
and teaching experience. The total effective
pool was calculated at per capita (or per
student) basis.
3.
Infrastructure score is a complex function
of IT infrastructure and Physical infrastructure
including on-campus and off campus hostel
facilities, availability of play grounds (Criket/
Football/ Gymnasium/ Pool/ Amphitheatre, etc),
Library, Books, and Research Journals.
4.
Output of produce (or students) is a complex
function of % of students getting placed out
of those opting for placement, and the Median
salary (not mean salary as the data suggested
that mean was NOT a good indicator of central
tendency being highly prone to extreme values
- this was smoothened by the skewness and
kurtosis statistics of the data) and Academic
Output is determined by the number of Research
Papers weighed by the Quality of Research
papers (as determined by the quality index
of publications), the volume and quality of
consultancy assignments and quality of MDP's
conducted.
^Since the
institutes provided data in hard numbers the
challenge was to convert this to ‘SCORES'
that were comparable. This was done by arriving
at indexed scores based on the ‘best in class'
institute on that parameter. These indexed
scores, being relational in nature, were thus
comparable.
DATA
COLLECTION AND VALIDATION:
This survey
gathers inputs from 3 different surveys for
generating a 360% perspective of quality and
performance of B-Schools:
•
B-School Survey: This survey
forms the backbone for the MBAUniverse.com
B-school Survey 2011-12. An exhaustive questionnaire
with 50 questions was sent to those B-Schools
that offer a 2 year Full Time program that
is accredited to be equivalent to a Full Time
2 Year MBA or equivalent and that are at least
5 years old. The questionnaire contained detailed
definitions of all the data points sought
and the schools were asked to provide documentary
evidence (E.g., in case of visiting faculty,
the name of the faculty, paper taught, and
dates were all sought and validated by the
student's survey). Only that data for which
verifiable documentary evidence was available
was used and the rest was not considered.
For a majority of schools, MBAUniverse.com
went back to seek clarifications and ask for
more data and evidence. The faith of B-Schools
was evident by virtue of the fact that we
received many calls from those schools that
were not initially invited with a request
to send them an invite, many schools went
to the extent of responding with a hard bound
set of documents (running up to 400 to 780
pages), many schools requesting an extension
of deadline, a large number of calls to confirm
the receipt of responses, and calls/ mails
to enquire the date of release of results
within 2-3 days of closure of the survey!
To generate
a more robust understanding of the needs and
demands of two key stakeholders (MBA students
& recruiters), we conducted Student's
Survey & Recruiters Survey separately.
What is to
be noted is that the data and learnings from
these two surveys were NOT added or fused
together with the B-school Survey, but was
used to understand the stakeholders needs.
Broader learnings from the MBA Aspirant's
survey will be published on MBAUniverse.com
at a later date.
•
Current & Prospective Student's
Survey: This was conducted using
an online tool with the aid of a structured
questionnaire as the survey instrument. This
included a long list of response-based customizable
instrument that captured data for recall value,
perceptions and for existing and past students,
in terms of experience with named institute.
As we were capturing the IP address as well,
we realized that there were a large number
of institutes whose data was getting skewed
in their favor (such responses being identified
by same IP address and machine's MAC address/
configuration used repeatedly within a short
duration (ranging from number of responses
within an hour to a day). So as to ensure
that these respondents perceived us to be
dumb enough and to discourage them from using
alternative techniques to bypass our surveillance,
we did not warn them, but all such responses
were completely removed from the system.
•
Recruiter's Survey: Trained, qualified
and experienced researcher (with over 15 years
of experience and over 782 assignments) interviewed
a mix (47:53) of HR Heads/ Recruitment Heads
and Line Managers of a cross section of organizations
(across organization size, geography of Indian
headquarters, and industry vertical) using
a semi-structured questionnaire and probed
them on various aspects including their expectations
and experiences with fresh recruits from various
(named) B-Schools and their ratings on the
same.
The survey
was conducted with strict adherence to the
code of conduct of Market Research Society
of India and the data was treated as most
confidential, with no one, except one member
each from Facts ‘N' Data , and MBAUniverse.com
having access to it. |
.jpg)
