Simulation & Gaming:
An Interdisciplinary Journal of Theory, Practice and Research
This article was originally published in Changing trends in management – Challenges and opportunities, Jan 2003, pp. 104-115, New Delhi: Excel Books. The copyright holder, Prestige Institute of Management & Research, Indore has given permission to the author to post this article, with slight modifications, on the web site of Simulation & Gaming: An Interdisciplinary Journal of Theory, Practice and Research.
Cite this version with this URL: http://www.unice.fr/sg/resources/articles/dumblekar_2004_management.htm.
Cite as: Dumblekar, Vinod. (2004). Management simulations: Tests of effectiveness. Online posting on Simulation & Gaming: An Interdisciplinary Journal of Theory, Practice and Research web site. <http://www.unice.fr/sg/resources/articles/dumblekar_2004_management.htm>.
Management simulations: Tests of effectiveness
A simulation is a replica of reality. As a training program, it enables adult participants to learn through interactive experiences. Simulations contain elements of experiential learning and adult learning. to ascertain the effectiveness of management simulation (MS), a field study was conducted at six venues for 138 managers. Factor analysis generated 5 factors of simulation effectiveness (SimE), namely, strategy, markets, leadership, competencies and focus. The locus of control construct was tested for its ability to help the MS participant to learn. It was found that participants learn more when MS sessions were in small numbers. Externals learnt more than Internals, who found MS to be ineffective in some SimE areas. Implications for management and suggestions for future research are discussed.
KEYWORDS: andragogy; constructed experiences; experiential learning; External; Internal; locus of control; management simulation; social learning theory.
“I hear and I forget. I see and I remember.
I do and I understand.”
A simulation is a replica of actual events, presented in a manner with a specific purpose. A management simulation (MS) is a collection of business cases that narrate market, business and economic events. When used for training, participants seek to understand management concepts, and augment or experiment with the methodologies, tools, techniques and practices learnt at business schools or at other training platforms. The final goal is stated as a financial measure, with a supporting explanation. A software program translates management action (their decisions) into business results (financial and other reports). The MS has been conducted across diverse industrial platforms, management functions, trainee audiences, corporate goals, or for specific situations. First introduced in 1999, MS is conducted at many large organizations and business schools.
Extensive – but unregulated and sometimes, unsolicited – responses from participants have led to some of the following conclusions. MS facilitated a rapid multi-skilling experience for participants. The extensive and periodic data-based results, which followed the participants’ decisions, enabled swift understanding of mistakes, followed by correction and careful experimentation. Peer-level communications facilitated quick exchange and absorption of difficult ideas. The team environment ensured an appreciation of delegation, functional roles and the management of performance. Participants visualized a broader and integrated view of their roles, their functions, the team and the business enterprise. Although conventional teaching was deliberately negligible, learning resulted from and was driven by the discovery of new areas of understanding emerging from successive periods of frustration, tension and excitement.
Andragogy is the learning process of adults where they indulge in the experience, willfully. Willingness to learn is vital to their learning capability, and therefore, the effectiveness of the learning process. Persons who believe that they can learn and apply that knowledge for themselves may be expected to exert more effort to learn and therefore, gain more benefits. With its multiple features and diverse contents, MS could claim to be the appropriate learning medium for adults.
An empirical evaluation of the benefits experienced by MS trainee participants in India would have established its utility as an effective experiential learning program in management education. This field experiment was therefore undertaken to isolate the correlates of simulation effectiveness (SimE). Its objective was also to ascertain, inter alia, the answers to the following questions.
Reflecting on the characteristics of the internet age, Rosenberg (2001) defines learning as the process by which people acquire new skills or knowledge for the purpose of enhancing their performance. This performance will be measured in the generation of better products and services, lower costs, a more competitive posture in the marketplace, greater innovation, improved productivity and increased market share. He reiterates that learning must be a continuous process, not merely a series of events, and that it transcends both the classroom and the workplace.
Day and Reibstein (1997) define simulation as a facsimile of reality, which is intended to display “what would transpire if the assumed conditions were to occur in reality”. Simulations have been used in many areas of activity, principally in warfare, in courtroom trials, in video games, and to polish pilots’ flying skills. They argue that simulations offer a more effective way to understand the future than the extrapolation of trend lines, forecasting and brainstorming.
Simulations would therefore be useful to learn about complex situations (where data is incomplete, unreliable or unavailable), where the problems are unfamiliar, and where the cost of errors in making decisions is likely to be high. Therefore, simulations offer many benefits. They accelerate and compress time to offer a foresight of a hazy future. They are experimental, experiential, and rigorous. They promote creativity amongst the participants, who develop a shared view of their learning and behaviors. Above all, making decisions have no real-life cost implications.
Kinnear and Klammer (1987) refer to a computer simulation game on marketing strategy called MARKSTRAT. They found that both academicians and managers believed that it constituted a realistic environment in which participants could become actively involved in making managerial decisions. As with any MS, teams made these decisions with the specific purpose of achieving an organizational goal. Senge (1990) refers to a beer game of the Massachusetts Institute of Technology’s Sloan School of Management. He described it as a laboratory replica of a real setting, and used it to conclude that structure influenced behavior, that structure in human systems was subtle, and that leverage came from new ways of thinking.
MS provides learning through experiences that are generated through participation. “Constructed” experiences also satisfy the conditions for experiential learning. An experience is a transaction between the individual and his distinctive environment, and is understood by the observer as a student. Experiential learning succeeds because one experience quickly leads to others, causing interactions that induce more learning, by dint of one’s impulses, needs and desires (Dewey, 1938). Quoting from the work of other researchers, Kaagan (1999) summarized that leadership was best learnt when learners were granted more opportunity to experience situations, problems and challenges, and not through didactic means.
Experiential learning programs foster a new breed of workforce collaboration and cooperation to generate new ideas, products and services, and technological applications, says Schettler (2002). They are often aimed at developing soft skills such as teamwork, leadership and group problem-solving, all of which are difficult to measure. Instead of merely responding to organizational procedures, employees’ commitment to their tasks would be higher, due to such experiential activity. According to Kolb, Rubin, and Osland (2000), such program processes have two goals: one is to learn about a particular subject matter, and the other is to learn of one’s strengths and weaknesses as a learner, i.e. learning how to learn from experience.
Adult learning (andragogy) is most effective when the adult learners are involved in the learning experience, and are not merely a passive recipient of information (Pfeiffer, 1994b). Often, the adults will relate their learning to their prior exposure, background and knowledge. Every experience must have both intellectual and emotional components. In training programs, the issues, problems and examples must be realistic. The learning environment must be informal, non-threatening, and non-judgmental. The use of variety and multiple equipments will stimulate audiences, and human facilitation will enrich the experience for the adult learner. MS provides many opportunities for andragogy.
Locus of Control (LoC) is a socially learned behavior that deals with how the individual attributes causes to behaviors and events that occur in their lives or are experienced by them (Pfeiffer, 1994a). Those who believe that such events are within their control and feel that they are in control of their destinies are called Internals because they attribute such events to factors within themselves. The others believe that external factors determine and control what happens to them, and attribute such events to luck, chance and fate. They are called Externals.
The LoC concept was developed by Julian Rotter (1954), to explain his social learning theory. According to the theory, the individual’s behavior was likely to be dictated by his expectation that his efforts would result in an outcome that was desirable to him. The theory is not so much about individual abilities, as it is about beliefs and expectations gained from that action. The LoC may be due to any of several antecedents such as accumulative events such as social discrimination, prolonged disability or parental upbringing, or episodic events such as death or a serious economic change.
Whatever the antecedents of LoC, research has suggested that Internals contribute more to effectiveness in organizational roles than Externals. For instance, Internals adopt innovations and new practices, quicker. Internals use persuasive power, while Externals are likely to use power and threats. Externals may be less capable of dealing with reality, and may express unrealistic occupational aspirations. Internals take better care of their equipment, are more knowledgeable and are more co-operative. LoC may therefore emerge as a strong determinant of the individual’s ability to learn.
The study was conducted for professional managers at six MS programs conducted during October and November 2001. The managers were from four of India’s largest public sector enterprises (PSE). They had academic qualifications and experience in different disciplines such as engineering, material dispatch and logistics, accounting and finance, marketing and sales, personnel and human resources, research and development, and quality.
These MS programs were conducted at the PSEs’ respective training venues, located at Gurgaon, Pune, Noida and Ranchi. The questionnaires were distributed towards the end of the program (second day), with brief oral guidelines on how to fill them. Because they were collected just before the final results were announced, emotional bias on account of such results was avoided.
Each MS event was spread across two days of intense activity in a tightly controlled arrangement. seven-eight teams (each of three-four participants) managed their respective companies within a common industrial environment of products, raw materials, other manufacturers (participating teams), suppliers and customers. Each team managed its firm for 15-18 months (split into five-six quarters of three months each), and received detailed performance reports for study and further action. The company was described in terms of a balance sheet and other enterprise reports. The cases described business situations that prompted them to take decisions.
Each participant took charge of clearly defined functions as a member of the top management team of that company. The roles were symbolically embedded in the designations provided to the members: chief operations officer, chief marketing officer, chief financial officer, and the chief executive officer, who would be the team leader. The MS administrator would outline the responsibilities attached to these roles, but it would be up to the team members to allocate those roles amongst themselves.
Every MS was organized at the venue of the client, viz. project site, learning center or business school premises. The simulations administrator began the session with an elaborate presentation that explained the purpose of the MS, management concepts and practices, and rules and documents of the MS. He intervened with announcements and team-specific discussions.
At the end of the MS, and before the end of the second day, each team made a presentation before the entire group of participants, evaluating their decisions, in response to a structured format. This would ensure an exchange of many experiences, admissions of ignorance and failure to understand, and about techniques learnt and applied. The simulations administrator would thereafter engage in discussions about the complexity of the markets, the volatility of competitive actions, the focus on business data and goals, the need for constructive dissent leading to consensual decisions, and the overall emphasis on continuous enquiry as a basis for further learning.
To collect data on SimE, a scale of 20 items was specially developed after discussions held in April-July 2001 with many participants in two categories: professional managers and management students. The composition of the scale was strongly influenced by the comments and other feedback offered by such participants. Respondents were asked to reply on a five-point Likert scale to the questions in the scale (Exhibit 1).
For data of Locus of Control, the psychological measure developed by Rotter (1971), reported in Robbins (1993: 126 and 713), was used. This scale has 10 items, and gives a maximum score of 10 for internality for that variable.
There appeared to be some resistance to the declaration of personal (demographic) data at the initial stages of the administration of the questionnaires. In order to avoid any annoyance or embarrassment to those respondents, these variables were ignored. Female participation at most simulations has been negligible, preventing any meaningful studies in this area. Participants were in the age group of 28-55 years.
The distribution of the samples at each organization and its program, and the total scores along two variables is given in Table 1. The data was then processed using exploratory factor analysis on the bases of principal component and varimax rotation methods with Kaiser normalisation.
Statistical procedures included the generation and use of measures of central tendency and dispersion such as mean and standard deviation, correlation and z tests (Black, 2001). Computations were made using Microsoft Excel and SPSS for Windows Student Version 9, 1999.
Table 1 shows the distribution of the sample across the different organizational subgroups.
Table 1: Sample distribution (n=138) amongst organizations, SimE Scores and LoC scores
# represents the organizations where the simulations were conducted, and N is the number of participants at that session.
From the exploratory factor analysis of the 20 statements of Exhibit 1, five SimE factors were extracted. The factors with eigen values greater than one accounted for almost 60% of the total variance (Table 2).
Table 2: Total variance explained
Table 3 lists the factors, their loadings and their component statements. The statements described specific benefits in learning from MS as identified below:
Table 3: Factor analysis results: Rotated component matrix
# refers to statement numbers shown in exhibit 1. Shaded areas represent loadings.
To determine the strength of the relationships between the SimE factors and LoC, simple regression and correlation analyses were made. Table 4 shows this Matrix with r as the measure of the linear correlation between the two variables and the SimE factors. F2: Markets was found to be weakly but negatively correlated with LoC. Of the strongest correlations amongst the factors, F2: Markets was correlated with F3: Leadership; F1: Strategy was correlated with F3: Leadership and F4: Competencies.
Table 4: Pearson product-moment correlation coefficient matrix for simulation effectiveness (SimE) factors with Locus of Control (LoC)
*** significant at 0.001 level
To extend this examination further, the total sample was split along the LoC median value of 8. By eliminating all records of that median value, two sample classes were isolated. One sample class of External LoC had LoC scores of less than 8, while the other sample class of Internal LoC had LoC scores of more than 8. The degree of correlation of LoC with each of the five SimE factors and total SimE was also computed (Table 5), but none of these were found to be statistically significant. External LoC has a small but positive correlation with each of the SimE factors. For Internal LoC, this is true (and marginally so) only in the case of F3: Leadership and F4: Competencies. In respect of F1: Strategy, F2: Markets, F5: Focus, LoC scores are negatively correlated for Internal LoC. The total SimE score was positively correlated with its External LoC score, but negatively correlated for Internal LoC scores.
Table 5: Pearson product-moment correlation coefficient matrix for simulation effectiveness (SimE) factors with External and Internal Locus of Control (LoC)
* Correlation with relative LoC scores
# LoC score < 8; ## LoC score > 8
To examine the impact of LoC on SimE and its factors, a z–test was performed between the External and Internal LoC subgroups. The results of the z scores and other related outputs are shown in Table 6, which showed no significant SimE changes due to Internality.
Table 6: z-Test: Two Sample for Means
Ext LoC score < 8; Int LoC score > 8
For Ext LoC, N = 51; For Int LoC, N = 58
The study demonstrated that it is possible and useful to measure the benefits of MS. The findings confirmed many earlier statements of feedback received from the participants. It was quickly established that smaller groups of participants learn more at MS. Table 1 shows that the relatively small groups of 15 and 20 participants in B1and H2, respectively had achieved better SimE gains, with higher SimE means. The larger groups showed a wider range of absorption of SimE gains, as evidenced by their higher standard deviation measures. This was especially true of C and P, who also showed the lowest SimE scores. The larger groups also had a greater dispersion of LoC scores, especially in the case of B2, C and P.
In managing their respective firms, the team members clearly felt the impact of strategy making more than any other management action or process in the MS. This is confirmed by the fact that F1 (Strategy) enjoyed the highest loading amongst the five SimE factors. Participants operated within a well-defined industry and were regularly in receipt of financial reports of their own performance. From the beginning of the MS, they were forced to chalk out a strategy to enhance their corporate wealth, in the context of their environment. In doing so, they learnt how to think long-term, take decisions on the basis of their relatively advantage for their organization, and sought a strategic fit of their activities to the external environment (Johnson and Scholes, 1997). This finding confirms Day and Reibstein’s (1997) conclusion that participants develop successful strategies, anticipate events, and learn about strategy and about their own businesses, at each step of the simulation process. Simulation played a powerful role in developing and analyzing competitive-strategy options.
The value of strategy has been emphasized by management practitioners, too. Working with several Japanese companies that regularly created wealth and built market share – despite being handicapped by resources such as people, money and technology or formal management education – Ohmae (1982) found that the common key to their success was their organizational strategist. This strategist was usually the founder or the chief executive, who had great natural talent and an intuitive grasp of the basic elements of strategy. Using insight, he developed a comprehensive set of objectives and plans of action out of the idiosyncratic mode of thinking that merged the dynamic interaction of the company, its customers and the competition.
Understanding the marketplace, the customer and the marketing processes are the keys to the firm’s financial success in any age or economic state. Discussing the battle for control of the customer’s mind, Gettler (2001) quotes management thinker and theorist, Peter Drucker on the importance of marketing. “Marketing is the distinguishing and unique function of the business. It is the whole business seen from the point of the final result seen from the customer’s point of view. The customer is the foundation of a business, and keeps it going.”
F2 (Markets) clearly implied that the participants’ strategic behavior in marketing decisions was guided towards maximizing the corporate goal. Those decisions were regularly affected by some or all of the competitive and environmental forces of the markets, the suppliers, other producers, technological events, the economics of supplies and costs, and potential threats and opportunities (Porter, 1980). The participants chose from a basket of alternatives that influenced sales: pricing, product mix, marketing communications, promotions and campaigns, sales channel commissions, product quality and credit policy. Although many of these were operational decisions, prudence demanded that the ultimate effect of these decisions could only be felt when used in conjunction with a strategy. Admittedly, these were the most disruptive decisions, for they caused the most conflict amongst the participants.
Participants learnt business Leadership (F3), the hard way: through argument, frustration, failure, tension and communications. Leadership in MS meant a fusion of use of business data and the understanding of business issues with the need to find solutions for conflicting demands for resources, expectations and objectives. Teams that were more successful as measured by their final goal exhibited visible signs of consensual decision-making, open communications, clarity of roles and work assignments, and periodic self-assessment, and other characteristics of effective teams (French and Bell, 1995).
Participants learnt to consent to a shared picture, if not to a vision, of the desirable future that they wished to create at the end of their tenure. The joint activities ensured that teams learnt to learn together, through the means of the extensive dialogue and thinking together, as described by Senge (1990). Team members learnt to suspend assumptions, to listen to each other without bias, and discovered insights about their work. In MS teams, learning may happen faster than in conventional modes of teaching or training, due to the constant exchange of information and other forms of interaction between the members. MS may therefore be an effective platform to facilitate in the understanding of team learning, a process that Senge considers vital for successful organizations.
MS participants received not only data for making decisions but also the opinions and knowledge from the rest of the team. Team members experienced both continuity and interaction, in comparing contemporary environmental events with the status quo, and in continuous dialogue and enquiry with the team. The learning resulted from what Dewey (1938) suggested as a process of observation of surrounding conditions, prior knowledge of similar situations in the past, and finally, judgment from observation and recall.
Competency is a collection of skills that met a desirable standard (Mayer and Salovey, 1997). Intelligence is an aptitude, while achievement was an accomplishment. Accordingly, competency indicated that an achievement had met a particular standard. Competency is also acknowledged to be a component of intelligence, along with abilities, talents, skills, and gifts of the individual. F4 (Competency) is important because it ensures the accomplishment of goals.
Learning to Focus (F5) on a common goal helped participants to direct their thoughts and efforts towards a common target, an activity that was seen as important by diverse management thinkers. Focus is what Drucker (1980) called “concentrating resources on results” so that the enterprise was lean and was able to move fast to avail itself of opportunity. Focus (stick to the knitting) was one of the eight attributes that characterized some of America’s best run companies (Peters and Waterman, 1982). de Bono (1991) agrees that “reducing the complex world to a specific challenge wonderfully focuses energies and thinking”.
Internals may have found it difficult to admit to their learning, which comes from the participation in an external experience. Findings therefore demonstrate that a headstrong belief in oneself may not facilitate learning from MS. For example, the Externals admit that they could control their environment, and therefore, attempted to learn more. That MS may not usually help Internals to learn substantially better than the others was a revelation.
This study analyzed how MS benefited its participants, and confirmed many features of experiential learning and adult learning. It identified those benefits in the form of five SimE factors. Small learning groups will benefit more than those larger. Externals will learn more than Internals, and high self-confidence levels of the participants may impede their ability to learn or to acknowledge their learning).
For prospective product managers, brand managers, profit center heads and chiefs of strategic business units (all business leaders), this adult experience can substitute for many years of hard-core on-the-job training experience and prepare them for their arduous roles in the future. For the business school students on the threshold of a career, this experience can augment and extend their classroom teaching exposure through a team learning experience that could prepare them for management practice. As an effective methodology in andragogy, the MS confirmed the validity of Confucius’ observations.
Suggestions for future research
This study provided support for the validity of SimE and its factors, but other areas of MS also merit attention. A potentially fruitful area would be to measure the soft skills that were learnt. Future research must use homogeneous samples with data from the same organization or industry, and academic and career backgrounds.
It is alarming to discover that Internals may fail to learn “external” areas such as how to strategize, how to focus on corporate objectives, and how to sell. However, it is gratifying to discover that a person’s lack of confidence (externality) would not interfere with one’s ability to learn in an andragogical exercise like the MS. If these conclusions were researched further, specific MS conditions, cases or objectives could be created to benefit that class of individuals. Till then, organizations and business schools would be wise to select and send only Externals to MS so that the learning benefits are ensured.
After the assimilation of learning topics, there may be a rise in the participants’ confidence levels. It may be worthwhile to examine whether - and if so, to what extent - MS can transform Externals into Internals. As argued earlier, this transformation may thereafter hinder the learning capability of the participant. Due to this ability to deliver quick knowledge, MS may have the potential to be an effective instrument to augment the participants’ confidence in management processes.
In this study, Rotter’s LoC scale was ineffectual in determining any significant correlation with the SimE factors. This was probably because the coverage of the scale was unrelated to the MS topics. In order to assess individuals’ attributions about influences on health and illness, Wallston and Wallston (1981) developed a multidimensional health Locus of Control scale that was based on Rotter’s two-dimensional scale. It may be worthwhile to develop a LoC scale specifically for the MS, on similar lines.
This study covered the individual participant, but ignored the group. Simulation is sufficiently engaging to develop group identity and collective cognitive processes (Chatman and Barsade, 1995). Long after the simulation ended, participants continued to hold on to their chosen team roles and discuss their team’s performance. New research must therefore examine team behavior and outputs, especially their incremental knowledge, competencies and intelligence. The findings would serve to confirm the usefulness of the program for groups.
Black, K. (2000). Business statistics: Contemporary decision making. Ohio: Thomson Learning.
Chatman, J. A., & Barsade, S. G. (1995). Personality, organizational culture, and co-operation: Evidence from a business simulation. Administrative Science Quarterly, 40: 423-443.
Day, G. S., & Reibstein, D. J. (1997). Wharton on dynamic competitive strategy. New York: John Wiley & Sons.
de Bono, E. (1991). Tactics: The art and science of success. London: Fontana / HarperCollins Publishers.
Dewey, J. (1938). Education and experience. New York: Collier.
Drucker, P. F. (1980). Managing in turbulent times. London: Pan/William Heinemann.
French, W. L., & Bell, C. H. (1995). Organization development. Englewood Cliffs, NJ: Prentice-Hall.
Gettler, L. (2001). Selling in the emotional economy. Management Today, November/December, 14-17.
Johnson, G., & Scholes, K. (1997). Exploring corporate strategy. Hertz: Prentice-Hall Europe.
Kaagan, S. S. (1999). Leadership games. New Delhi: Sage Books.
Kinnear, T. C., & Klammer, S. K. (1987). Management perspectives on MARKSTRAT: The GE experience and beyond. Journal of Business Research, 15, 491-502.
Kolb, D., Rubin, I., & Osland, J. (2000). Organizational behavior, an experiential approach. Englewood Cliffs, NJ: Prentice-Hall.
Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D. Sluyter (Eds.), Emotional development and emotional intelligence: Educational implications. New York: Basic Books.
Ohmae, K. (1982). The mind of the strategist. New York: McGraw-Hill.
Peters, T. J., & Waterman, R. H. (1982). In search of excellence. New York: Harper & Row.
Pfeiffer, J. W. (1994a). Theories and models in applied behavioral science. Vol 1. San Diego, California: Pfeiffer & Company.
Pfeiffer, J. W. (1994b). Theories and models in applied behavioral science. Vol 2. San Diego, California: Pfeiffer & Company.
Porter, M. (1980). Competitive strategy. New York: Free Press.
Robbins, S. P. (1993). Organizational behavior: Concepts, controversies and applications. Englewood Cliffs, NJ: Prentice-Hall.
Rosenberg, M. J. (2001). E-learning. New York: McGraw-Hill.
Rotter, J. B. (1954). Social learning and clinical psychology. Englewood Cliffs, NJ: Prentice-Hall.
Rotter, J. B. (1971). Internal control and external control. Psychology Today, June, 42.
Senge, P. M. (1990). The fifth discipline. New York: Double Day/Currency.
Schettler, J. (2002). Learning by doing. Training, 39(4), 38-43.
Wallston, B., & Wallston, K. A. (1981). Health locus of control scales. In H. M. Lefcourt, Research with the locus of control construct: Vol 1 Assessment methods. New York: Academic Press.
Exhibit 1: Scale for simulation effectiveness (SimE)
Evaluate each of the statements below in order of their importance and utility for you within the range from 1 = least important to 5 = most important). Please place that value at the end of the statement.
This simulation session has helped me:
Vinod Dumblekar is a software-based simulations designer and trainer in management, and has trained over 9,000 participants since 1998. He conducts simulations as training programs and as competitions for managers and business school students across India. He has a post-graduate degree in management from the Faculty of Management Studies, University of Delhi and expects to will receive his doctorate in management in August 2004.
ADDRESS: 71-A, Pocket ‘A’, Sukhdev Vihar, New Delhi – 110 025, India; telephone: +91-(0)11-981-863-1280; e-mail: dumblekar AT yahoo.com.