Applying GQM Method in Balanced Scorecard System for Iranian University Performance Appraisal

Rapid development in academic instruction sector has resulted in changes in the supply and demand of higher education graduates and influenced on the quality of education. Hence, if the performance assessment of a university is done in an effective manner, the university may identify internal strong and weak points and provide the chance of improvement. The aim of this article is to apply Balanced Scorecard Systems to access the performance of a sample in Iran’s universities. On the other hand, this article describes how to apply GQM method to obtain the suitable metrics of measurement in Balanced Scorecard System assessment to improve the efficiency.


INTRODUCTION
The Balanced Scorecard (BSC) is one of the performance measurement tools determines intangible assets of the organization. This system is used more frequently in strategic planning than that in performance assessment. Balanced Score Card System is, in fact, a factor for improving control that is regarded as a base for promoting the university quality and preventing defect occurrence. Implementing Balanced Score Card System allows a university to interpret perspectives and strategies based on a special pattern and then evaluate them. Regarding the great role of the universities in educational systems, accurate performance assessment of the universities results in identifying the other strong and weak points and providing the chance of improvement. In this article, one of the universities in Iran was selected to implement BSC and four perspectives related to it titled as financial, customer, internal process and growth and learning studied. In this assessment, real data of this university was used and questionnaire tools applied to collect data about the prospect of growth and learning. Moreover, to get appropriate metrics for measuring in perspectives of BSC, GQM method was used. The results of survey showed that the performance of the studied university was in an appropriate condition. Also, these results and the manner of performance assessment may be useful for the university and academic departments that aim to implement BSC for evaluating the performance of their organizations. For the future purposes, it's suggested to implement this system with fuzzy theory to evaluate quality indices (developing motivation in staff, using novel methods).

RESEARCH METHOD WHAT IS A BALANCED SCORECARD?
The Balanced Scorecard (BSC) is a performance measurement tool originated in the business world. Performance measurement is a method of tracking performance over time to assess if goals are being met. The BSC was introduced by Robert Kaplan, and David Norton, the founders of Balanced Scorecard. In the early 1990s as a new way to measure business performance, Organizations measured their performance to monitor how they achieve their overall mission and goals. Traditionally, companies measured their performance by looking only at how they were Kaplan and Norton-the founders of Balanced Scorecard-have described this method simply not simplification:

Selecting a suitable organizational unit:
Criteria for selecting a suitable organizational unit: Before concluding that a balanced scorecard system is the best option at high level for you, some criteria need to be considered for an important decision. A variety of elements help us to select an appropriate organizational for the first Balanced Scorecard System. These criteria are shown in figure1.

Strategy
The most important criteria is that if the desired unit has a clear and coherent strategy. However, the method of a balanced system is a special method designed to help you to interpret your strategies to the objectives and measures having the capacity of controlling. Without strategy, it would be like as a set of financial and nonfinancial measures are collected so that they have no connection to each other.

Need of a balanced system
Answering to this question is needed that if the desired unit has a specified objective for implementing the Balanced System.

The possibility of providing information
This criteria refers to two subjects. First, does a desired unit have a culture of performance assessment? Although any group in a modern organization should rely on the measures and performance criteria, the assessment of this issue is relatively difficult. It is because some of the measures of a Balance System are probably new Balanced score card method Need for balanced score card Corporate area

Senior management
The key manager

Information
Existence clear management strategy and their required data resources are probably undefined. However if the desired unit has some problems in collecting the data related to the current performance measures, it will certainly have some problems in providing the data related to the Balanced Scorecard System.

Resources
To implement, a unit should be selected to have the possibility of providing information. For this reason and considering this point that if the performance appraisal of a university is done through the method of effectiveness, the university may use the internal strength points and make benefit from the internal opportunities, identify the threats and defend against them and finally before becoming harmful of internal weakness point, decrease its power. Because of the defects in the present methods of performance appraisal, the performance may not be evaluated accurately, so the managers won't be able to make decision accurately with respect to the evaluation. In order to remove this problem, the Balanced Scorecard System allows a university to interpret and evaluate the strategies and views based on a special pattern. Regarding the available resources and adequate information and precise strategies (second stage of implementation) and the requirement to the method of balanced evaluation mentioned above and the great share of university and academies (including the students, professor and staff) a suitable balanced Scorecard approach was identified. General evaluation of table1: This unit has attained a very high score (8.8 from 10) and therefore it's a good candidate for implementing the method of balanced Scorecard.

Collecting required background information to determine the strategies, views and mission s of an organization
Balance Scorecard System is a tool for analyzing the strategy. To accomplish, this hypothesis should have full access to necessary data and information in field of Missions, views, strategies, competitive position and capabilities of a central organization (Bakhtiari, 2007).

Conducting an interview to clarify the vision, mission and finally to achieve strategic objectives
According to the study and group meetings vision, mission and goals are as following:  Creating a dynamic and creative learning environment for nurturing the teachers and researchers.  Providing permanent financial resources.  Preparing upgrade software and hardware equipments.  Achieving, maintaining, and developing research situation for increasing scientific productions at national, regional, and international levels.  Improving and developing the quality of services for the students.  Continuous improvement of scientific and administrative management.

Integrating the results of interviews and determining the strategic objectives and goals
After clarifying universities' strategies and goals, any of these goals should be assigned to four perspectives of the balanced scorecard system. Balanced scorecard is changed to a framework for implementing and strategic managing at all levels of organization and it does this work by linking to the ultimate goals, innovations and standards related to the overall strategy and provides this by maintaining financial measures with the other key indicators around customer perspectives, internal business processes, organizational growth, learning and innovation and offers a view of organizational performance to the company (Bakhtiari, 2007).

Assigned objectives to each of four perspectives of the Balanced Scorecard System
Balanced Scorecard System completes financial indicators from the past performance with the future performance. The goals and indicators of Balance Scorecard System have been determined through strategy and perspectives of organization. This goals and indicators refer to the performance of organization in four aspects: Finance, customer, internal processes, learning and growth. It also provides a framework for balance assessment system. In fact, in this stage, the strategy is converted to the goals (Kaplan and Norton, 2004).

financial perspective 
Financial perspective is vital for organizations, because it will obligate us to identify and set financial goals that an organization need to achieve them. Financial perspective mentioned some cases which are very important:  The main objective of business is to create wealth creation that is determined by achieving to the goals.  The purpose of the quantitative-financial goals is to prepare the operational units for performance management and to achieve the competencies for the future success.  Financial perspective is a related performance index because it registers the success after its occurrence. Therefore, we may explain the goals of an organization as follow: 1. Reducing the cost of manpower.

Customer perspective
Who are our target customers and what we offer them through evaluating the service? Customer's perspective goals and purposes describe the strategic goal and the presentable value of an organization. Some of the goals considered by the organizations are as the following: 1. Effective and accountable operations. 2. Offering competitive prices. 3. Attracting, developing and retaining the best customers. 4. Increasing customer satisfaction. 5. Improving the perceived view of the university. 6. Delivering the innovations that make business and career value. Clients of educational organizations are employees, students, parents, government and labor market (Bakhtiari, 2007). In general, teachers of educational organizations, managers, and learners are internal clients, but governments, public sector, and parents are foreign clients. If both internal and external customers are satisfied with the performance of educational organizations, University will not lose their learners and will sustain a client perspective.

Internal processes perspective
In order to determine the strategic goals what the internal processes perspectives of organization should do? One of the various benefits resulted from using the balanced Scorecard, is identified as the results of actions in this perspective. Most performance systems in current processes are focused on gradual improvements, while the balanced scorecard method maybe led to the formation of entirely new processes for realizing the desired values of the clients. The internal processes may generally be classified in four groups: 1) Operational (a key goal in this strategy is a low lasted cost): It produces the product and service and supply them, such as production, supply, providing the opportunities for students and preparing educational and research facilities. 2) Customer management (a key goal in customer-oriented strategy): It enhances the customer's value, such as selecting students, employing, retaining and developing the instruction corresponding with them so that are related to the ultimate quality assurance. 3) Innovation (a key goal of output leadership strategy): A process that will create new products and services such as: Identifying opportunity, research and development of projects, designing / developing and implementing. 4) Legal and Social: Emphasis on effectiveness of the management, improvements in education quality, management process, real-time appraisal and services to increase effectiveness and reduce the time proposed to attract satisfaction (Neon, 2002).

Learning and growth perspective
Learning and growth perspective is the basic in the Balanced Scorecard System (Kaplan and Norton, 2004). This perspective may be regarded as a driving force for the three previous approaches in achieving high performance. The function of this perspective is to create a complete set of core technical capabilities and to improve the previous three perspectives. Upon rapid advances in information technology, competition in educational market has increased as widespread (Chen et al, 2006). Intangible assets in educational organizations, including universities, are classified in three groups: 1. Institutional Capital 2. Investment information 3. Human Capital Some conceivable goals in this perspective are: 1. Developing the diverse and effective workforce 2. promoting and developing the spirit of partnership 3. Appropriate access to information and an attractive and healthy working environment. Universities should provide opportunities for members to learn and grow. Human Capitals at the Universities are professors, researchers and staff. Using the balanced scorecard in the performance evaluation system will result in improving the assessment, saving cost and time. The use of verbs such as increase or decrease, start, create, reduce, improve, becoming and the other verbs like these may be the facilitator when setting goals. Specified goals should create necessary motivation for performing the desired actions in people. But these goals should not necessary be quantitative in their nature.

Using indicator to achieve the success criteria for any goal
As shown in Figure3, performance measures form the basic core of the balanced scorecard. For selecting measures is to remain in combination of performance evaluation system may be examined through some of these criteria. Collecting necessary data to calculate the measures is also one of the important aspects and challenges in the establishment of a balanced assessment system.

Some criteria for selecting performance measures
Designing and implementing the balanced scorecard as an organization management system orientation, requires the selection of goals, quantitative goals and executive initiatives to achieve them. Difficult decision making process appears at the stage of measures selection. These measures are as a main core in the balanced assessment system and it's considered as a reference and a focus point for all organizations.

Relationship with strategy
Relationship with strategy is considered as the most obvious and significant criterion to assess the measures but its importance is not necessary to be emphasized significantly. In many businesses, a balanced appraisal system includes number of measures play an important role in daily performance. But it does not seem to be directly associated to the strategy.

GOAL-QUESTION-METRIC (GQM) FOR BETTER METRICS
The GQM method traces its roots back through software engineering practices into the 1970s, primarily through the academic and industry research conducted by Victor Basili of the University of Maryland.GQM is a simple, threestep process for developing better metrics. The first step in the process involves defining specific goals that the organization hopes to achieve. These goals are not measurement goals, but objectives that measurement is supposed to help achieve. The goals are then translated into even more specific questions that must be answered before assessing whether the organization has achieved or is achieving the goals. Finally, these questions are answered by identifying and developing appropriate metrics and collecting empirical data associated with the measurements. The method ensures that the resulting metrics data remains explicitly aligned with the higher level goals and objectives of the measurement sponsors. Figure2 illustrates the basic GQM method.

GQM Basics (Lance Hayden, 2010)
In one of his most professional development seminars, Basili defines the GQM process as follows:

Develop a set of corporate, division and project business goals and associated measurement goals for productivity and quality
The GQM method includes a basic template concept for articulating the goals of a measurement or improvement project quickly and succinctly. Specific information is captured regarding the goal, including explicitly defining the basic attributes and criteria for success. The resulting information is incorporated into the template and used to create a basic statement of the goal. These components are shown in Table 2.

Table2: Goal Template for the GQM Method
Goal Component

Generate questions (based on models) that define those goals as completely as possible in a quantifiable way (Lance Hayden, 2010):
To develop that information, individual goals are translated into a series of questions that enable the components of the goal to be achieved or evaluated for success. These questions articulate the goal and the measurement project in terms of what objects or activities must be observed and what data must be collected to address the individual components of the goal statement. Several questions are already implied by examining the goal components:  How much is the rate of students' empowerment from the view of knowledge and science acquisition in the universities.  What is the rate of the university revenues during the last four years?  How has the proportion of masters to the students been during the last four years?

Specify the measures needed to be collected to answer those questions and track process and product conformance to the goals
Step3 is about examining how the questions could be answered, moving from the qualitative (or operational level) to a quantitative level. Once goals are refined into a list of questions (GQM process step 2), metrics need to be defined that provide all the quantitative information to answer the questions in a satisfactory way. In other hand, after questions have been developed to define the goal operationally, the goal can begin to be characterized at a data level, and metrics can be assigned that will provide answers. A key strength of GQM is that, by this point, designing metrics becomes much more intuitive, because only certain measurements will produce the data necessary to answer the very specific questions that the goal has produced. Many metrics are potentially able to answer these questions, and more emphasis can be placed on evaluating the feasibility of adopting certain metrics based on how difficult data may be to collect or how detailed the data needs to be. The questions also help the project stakeholders choose appropriate quantitative or qualitative measurement and analysis techniques in a way that is driven by the goal and not subject to arbitrary judgments about the metrics themselves.

APPLYING GQM TO BSC FOR IRANIAN UNIVERSITY PERFORMANCE APPRAISAL (RESULTS AND DISCUSSION):
Applying GQM method in BSC may be used as an appropriate tool to rank the universities from the view of their performance assessment and this action may have a significant role in promoting the quality and improving the education level of the universities. GQM pattern for financial perspective of BSC is shown in table 3. The goal of this project is to increase the income and to decrease the university expenditures from the financial view.

Question
What is the level of the university revenues during the last four years?

Metrics
Revenues derived from student tuition during the last four years.

Question
How much is the university costs during the last four years?

Metrics
The salary costs of the staff during four years Administrative, educational, survey and capital costs of the university during four years. The wage costs of the faculty members during four years.
After collecting data using metrics, they were analyzed using statistical descriptive and inferential methods. For example; table 4 shows frequency description based on the real incomes and its comparison with the tuition index in relation to last year.

Goal Components
Outcome: Increase Element: average Element: survey activities Perspective: growth and learning

Goal Statement
The goal of this project is to increase the rate of growth and learning of the university students from the viewpoint of growth and learning.

Question
How much is the rate of students' empowerment from the view of knowledge and science acquisition in the universities.

Metrics
Obtaining the students' average Obtaining the number of articles and their survey activities.

Goal Components
Outcome: Assessment Element: Entertainment places Element: Exercise places Element: service places Element: Teaching places Perspective: internal processes

Goal Statement
The goal of this project is the facilities and equipment sector of the university from the view of suitable place with respect to internal processes.

Question
Are the facilities and equipment necessary for providing a high quality of education available in the universities?

Metrics
Comparing administrative and educational places with its standards. Comparing the service sectors places of the universities with its standards. Comparing the Exercise places with its standards.

Goal Components
Outcome: increase Element: student's satisfaction Perspective: student(customer)

Goal Statement
The goal of this project is to study the student's satisfaction of the master proportion to the student.

Question
Is the proportion of the master to the student suitable in the university?

Metrics
The proportion of masters to the students has been during the last four years?

LEARNING IN IRAN'S UNIVERSITY
The results of structural equations analysis have been reflected in figure3. As it seen, growth and learning are as dependent structures.
As the diagram shows, empowerment has the most influence on the structure of growth and learning so that it has the highest correlation coefficient. Job satisfaction has the least influence on the structure of growth and learning.

CONCLUSION
In this article, one of the universities in Iran was selected to implement BSC and four perspectives related to it titled as financial, customer, internal process and growth and learning studied. In this assessment, real data of this university was used and questionnaire tools applied to collect data about the prospect of growth and learning. Moreover, to get appropriate metrics for measuring in perspectives of BSC, GQM method was used. The results of survey showed that the performance of the studied university was in an appropriate condition. Also, these results and the manner of performance assessment may be useful for the university and academic departments that aim to implement BSC for evaluating the performance of their organizations.