Evaluating the Effectiveness of Self and Peer Assessment using PBL on Student Performance and Preference: Malaysian’s Experience

This paper reports on the effectiveness of students’ performances between self and peer assessment based on student’s preferences. A total of 56 respondents from the second year of study at UTeM who enrolled for the Human Computer Interaction course took part in this study. Three research instruments were developed for the purpose of evaluating students’ performance and preferences which include a set of questionnaires, interview questions and prototype development. A t-test was conducted to analyze independent variables by self and peer assessment while student’s preferences and performances are dependent variables. Findings of this study revealed that students prefer peer assessment rather than self assessment. The use of courseware namely PBLAssess can increase student’s understanding towards the topic that has been taught. Results showed that assessment in PBL is significantly related to student’s performances and preferences


INTRODUCTION
PBL was officially adopted as a pedagogical approach in 1968 at McMaster University, a Canadian medical school. PBL has been widely used across a considerable range of subjects and professionals areas in higher education. Many have written about approaches to PBL, curriculum design, the role of the tutor and various other aspects but much less attention has been given to assessment in PBL. Many students find PBL motivational as they realize that is really about how they learn outside the classroom (MacDonald R., 2001). However, some difficulties are emerging as many people retain the assessment methods they used in their traditional approaches resulting in a misalignment between their objectives, student learning outcomes, learning, teaching methods adopted and the assessment of student learning (Macdonald, R. & Savin-Baden, M., 2004).
The domain chosen for this study is Human Computer Interaction (HCI). HCI focuses on understanding basic concepts of human interaction and its relationship in system development. The module designed for this course is used in teaching all second year FICT students at UTeM. The objectives of this course is to expose students to understand the relationship between cognitive psychology and user interface design, differentiate between mental models, conceptual model and also design user interface for various computer applications such as web sites or products.
The objective of this study is to evaluate the effectiveness of students' performance in self and peer assessment based on student preference. The next section of this paper looks at related research on this area such as, Problem Based Learning, Assessment in PBL, self assessment and peer assessment. This is followed by a detailed explanation of the methodology applied, information on PBLAssess and the research instruments used. The research results are reported and this is followed by discussion and conclusion.

LITERATURE REVIEW
The use of PBL in computer based is different from traditional environment. It also can enhance learning experience especially in the assessment part. Assessment is a tool used to reflect the education outcomes. The results of assessment are used to judge and rate not only for the students, but also for the lectures, departments and institutions. For example, in UTeM, the result gives the benefits to control learning outcomes in relation to the pre-established learning goals. They need assessment to set education standard, monitor the quality of education, monitor student's learning process and academic promotion. Several assessments forms may be used to assess learners' use of elaborated knowledge in solving problem. Since the primary goal of PBL is to instill an attitude and character that will enhance the students' abilities to work in group, to solve new complex and ill-structured problem, critical thinking and self-directed learning skill, then assessment in PBL must value these skills. Mitchell and Delaney (2004) identified four core issues that need to be addressed when assessing students in a group project setting, namely; 1. Assessment of the group performance.
2. Assessment of the individual contribution to the group.
3. Assessment of the project deliverables.
4. Assessment of the course success.
From those criteria, assessment of the group, the individual and the project deliverables in the small group PBL project course, were entered into three skills groups which were identified from the course objectives that have been implemented in their study. There are Implementation skills, Teamwork and leadership skills, and Analytical thinking, problem solving and interpersonal skills as tabulated in Table 1. The skill is adopted from Mitchell & Delaney (2004).

A. Problem Based Learning (PBL)
PBL is a total approach to education and involves a constructivist approach to learning (Marinick, 2001). According to Gary and Max (2004), PBL is a pedagogy drawn from constructivism and considered to be one of the best examples of a constructivist approach. Levin (2001), described PBL as a tool that can promote the kind of active learning experience that prospective teachers should embark on during their initial teacher preparation and experience throughout their professional lives.
There are many different ways of implementing PBL but underlying philosophies associated with it as an approach are broadly more student-centered than those underpinning traditional PBL (MacDonald, R. & Savin-Baden, M., 2004). Relative to conventional lecture based methods in which information is transferred from teacher to student, the research literature suggests that based upon the literature reviews noted below:  PBL promotes more versatile studying methods and PBL students are more likely to use the library resources to study.
 PBL develops greater knowledge retention and recall skills.
PBL is an instructional methodology solution to enhance learning by requiring learners to solve problems. It is a methodology with the following characteristics (Hung, W., Jonassen, D. H., Liu R., 2006). PBL is problem focused where learners begin learning by addressing simulations of an authentic, illstructured problem. The content and skills to be learned are organized around problems, rather than as a hierarchical list of topics. Knowledge building is stimulated by the problem and applied back to the problem. It is student centered because faculty cannot dictate learning. It is self-directed where students individually and collaboratively assume responsibility for generating learning issues. The processes through self assessment and peer assessment in PBL can assess their own learning materials. It is self reflective where learners monitor their understanding and learn to adjust strategies for learning. In PBL, tutors are facilitators who support and model reasoning processes. They also facilitate group processes, interpersonal dynamics, probe students' knowledge deeply, and never interject content or provide direct answers to questions. One of the strengths of PBL is that students are placed in the position of finding out appropriate information for themselves (MacAdrew et al., 2008).
PBL follows a simple cycle. The facilitator role of the tutor varies with each stage of cycle. The teacher must ensure that all group members know the stages of the PBL. Although the steps are listed in a straight forward fashion, there is often some overlap, rethinking as the group proceeds.

B. Assessment in PBL
Traditionally, assessment has been about finding out how much students know, usually in terms of knowledge or content. However, in PBL what we are really interested in is the students' ability to perform in a professional context, to recognize their need to acquire new knowledge, skills and to view learning holistically rather than atomistically (MacDonald, R. & Savin-Baden, M., 2004). As an assessment tool, concept maps may be used to document changes in knowledge and understanding over time (Weizman et al., 2008). Assessment is the process of gathering and discussing information from multiple diverse sources in order to develop a deep understanding of what students know, understand, can do with their knowledge as a result of their educational experiences and the process culminates when assessment results are used to improve subsequent learning (Huba & Freed, 2000). Definition from the Assessment Reform group (2002), "Assessment for learning is the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there". Assessment is compatible with PBL and the specific discipline or profession (Savin-Baden, 2004; Raine & Symons, 2005).
There are many forms of assessment in PBL. Some of the assessments have been used successfully in PBL and which also move away from the need to have outcome based examinations. Table 2 shows forms of assessment in PBL.

1) Self Assessment:
Self assessment is judging the quality of ones own work (O'Grady G., 2004). It is a process where students are involved in and responsible for assessing their own piece of work. It encourages students to become independent learners so they can increase their motivation. The self assessment allows the learners to compare the standards achieved by the other learners against their own work (Race P. et al., 2005). It usually allows them to assess aspects of their work such as the range of vocabulary, originality and structure. For the self assessment, the feedback is immediate in the sense that the students mark their performance using the detailed rubric, and based on this determine the actions they need to take to develop required knowledge and skills. It gives people the choice to access their own needs or carry out an assessment on behalf of someone else. Self assessment can take many forms including: -Writing conferences -Discussion (whole-class or small-group) -Reflection -Weekly self-evaluations -Self-assessment checklists and inventories -Teacher student interview

2) Peer Assessment:
Peer assessment is the act of students assessing one another (Boud, D., et al., 2001). Peer assessment is defined as students grading the work or performance of their peers using relevant criteria (Falchikov, N., 2001).Students must have a clear understanding of what they are to look for in their peers' work. Peer assessment is quite different from selfassessment. There is evidence that peer feedback enhances student learning (Falchikov, N., 2001) as students are actively engaged in articulating evolving understandings of subject matter. However, in peer assessment students are making assessment decisions on other students' work. The students can comment and judge their colleagues work. It also encourages student's thinking skills. Bostock (2000) points out that peer assessment encourages the students to be independent and develops skills in high cognitive areas. Peer assessment can help self assessment. Peer assessment will help students become more autonomous, responsible and involved, encourages students to critically analyze work done by others, rather than simply seeing a mark. The real purpose is to allow students to gain feedback from each other. According to the researchers, self and peer assessment can be explained in Table 3. Self assessment: Self assessment was a formative way to get students to reflect on their abilities, performance and attitudes, but had no summative value.
When/Place: Self assessment took place at the end of each monthly rotation by using the rubrics.
Peer assessment: Peer assessment had no summative value; it fostered reflection by students on how their classmates assessed their performance.
When/Place: Peer assessment took place at the end of each monthly rotation. Each student handed over this written peerassessment format to each one of his peers.

Hernandez R. (2007)
Self assessment: A reflective entry in the student's learning journal based on the effect that the peer assessment task had on his/her learning process.
When/Place: Outside of the classroom environment. Self assessment: Self assessment is the ability of a student to judge his/her performance, that is, to make decisions about one's self and one's abilities. Although the term self-evaluation is common.

When/Place: Classroom environment.
Peer assessment: Peer assessment has also been described as a strategy involving students' decisions about others' work that would typically occur when students work together on collaborative projects or learning activities.

When/Place: Classroom environment.
A comparison between self and peer assessment and the facilitator's judgement about students' performance showed a fairly large difference between the two judgments (O'Grady, G., 2004). However, when comparing self and peer assessments, peers tend to rate others higher than their own evaluation suggesting students were not simply rating on the basis of self interest. It is important to help students in developing judgement about their own and other's performance by taking peer and self assessment when assessing students.

A. Participants
The purpose of this research is to evaluate the effectiveness of assessment in PBL between self and peer assessment. Several assessment forms may be used to assess learner's use of elaborated knowledge in solving problem. Therefore PBL is an educational w w w . i j c t o n l i n e . c o m approach in which students and tutors need to know what is going on during the learning process itself. Thus, this study would focus on assessment of learning process and the quality of students' improvement. To elevate this idea, an appropriate form of assessment namely self and peer assessment is proposed.
A total of fifty six students had enrolled for the course of Human Computer Interaction (HCI) in Semester 1 2009/2010. The topic of User Interface Design (UID) which includes sub topics such as Cognition, Mental model vs. Conceptual model, User Interface Components and 3D interfaces was chosen as a prototype to be designed for PBLAssess. An early analysis conducted based on past semester performances had indicated that topics under UID had a poor performance from students. UID had the lowest standard deviation of 2.273 and mean of 3.66. Based on the result, the highest mark gathered by the students for this topic was only 9 out of 20 compared to other topics in HCI .
The prototype named PBLAssess was developed based on ADDIE model. The ADDIE Model is a term used to describe a systematic approach to instructional development. There are five phases consist of Analysis, Design, Development, Implementation, and Evaluation. The ADDIE model was applied for this study because it provides a systematic, step-by-step framework used by instructional designers, developers and trainers to ensure course development and learning does not occur in a hazardous or unstructured way (Alessi and Trollip, 2001). Hence, in order to help students begin to think like competent designers, Instructional Design (ID) instruction should be contextualized in or framed by an authentic such as in a real world and complex in designing the problem (Dabbagh et al., 2010).

B. Navigation Design
Navigation design describes the overall navigation in the system. Figure 1 shows the navigation in PBLAssess. The development process of PBLAssess includes the creation of the instructional materials. The instructional materials contain information that students had used to achieve the objectives. The PBLAssess can be designed by selecting content, media and type of interactivity that best underpins these learning objectives. PBLAssess consists of five modules: 1) Problem Scenario, 2) Lecturer Module, 3) Problem Solving, 4) Exploration and 5) Assessment. The PBLAssess environment, scenario and example of modules are shown as Figure 2, 3, 4, 5, 6 and 7. The students will be presented with a problem scenario as a trigger for discussions of the problem as shown in Figure 2. Figure 3 shows the Lecturer module. This module will show all the subtopic in User Interface Design. The extended resources provided in this module will help student in understanding the problem as shown in Figure 4. Figure 5 shows one of the methods to solve problem in PBL called FILA (Facts, Identify, Learning Issue & Action). After a first brainstorming session around the PBL case, students define learning issues for the next tutorial meeting (Singaram et al., 2012). Students need to fill in FILA table form as shown in Figure 6. The FILA table will be automatically opened in Microsoft Word document. Figure 7 shows Assessment module. There are two form of assessment in PBLAsssess which is self and peer assessment.  The extended resources provided in this module will help student in understanding the problem.

Fig. 5 Problem solving
Students can solve their problem solving skills using FILA

RESEARCH INSTRUMENTS
A t-test was conducted to analyze the independent variables which are self and peer assessment and the dependent variables includes perceived initiative students performances and preferences. The research instruments used to collect data for this study includes questionnaire and development of prototype. Below are the explanations of the requirement: 1. Questionnaire -A set of checklist questions were prepared based on three factors which are course content, learning materials and assessment and evaluation.
2. Prototype -Prototype development is the creation of the first working model of a new product or invention. The prototype called PBLAssess is developed based on ADDIE model and the topic chosen for the prototype is User Interface Design (UID). Table 9 shows the result of t-test and p-value for self and peer assessment using PBLAssess. The test employed paired sample t-test. Based on the performance of the students in t-test using self assessment (pre) and self assessment (post) assessment preferences in PBLAssess, the t-value is 9.427 and the significance of two tailed value, p is 0.000. In the next series, of peer assessment (pre) and peer assessment (post) assessment preferences in PBLAssess, the t-value is -11.955 and the significance of two tailed values, p is 0.000. The result shows, p< 0.05, thus there is a significant difference between using of self assessment and peer assessment in PBL. Hence, the null hypothesis H01 is rejected. Based on the result, peer assessment performs highest mean compare to self assessment in PBL. Therefore, the result indicates that peer assessment performed better that self assessment among students at UTeM. Peer assessment also can be used to enhance the quality of students' personal improvement and their contribution on group work (Roberts T.S., 2006).  Based on the results in Table 5, for the group which prefers peer assessment in PBL, the average of preferred peer assessment in PBLAssess is 67.14 (SD = 15.538) which is higher than the average of using self assessment in PBLAssess, 59.09 (SD = 14.110). The tvalue for group that prefers peer assessment is 2.580 and p-value is 0.023. Since the p-value is smaller than 0.05 (p<0.05), there is a significant difference between the result of using peer assessment and self assessment. Thus, there is a positive relationship between peer assessment student preferences and their performances.

A. Student Performances and Preferences
Furthermore, students with self assessment, who have been using peer assessment in PBLAssess perform the highest average with 57.50 (SD = 10.875) compared to group using self assessment which the average is 56.50 (SD = 8.835). The results also revealed that students who have been practically using self assessment approach found that they are more inclined towards peer assessment form. This finding is strongly supported by the statistical result which shows the highest average score of 57.50 as compared to group using self assessment with average score of 56.50. The t-value for the group that preferred self assessment is -0.632 and p-value is 0.000. The p-value is smaller than 0.05 (p>0.05), there is a significant difference between the result of using self and peer assessment. Thus, there is a negative relationship between linear student's preferences and their performances.

EFFECTIVENESS OF PBLASSESS
This research aimed to determine the effectiveness of using PBLAssess which has been developed using a PBL approach. The questionnaires were categorized under four construct which are the Course Content, Learning Sources, Assessment and Evaluation and Overall. The Cronbach's alpha reliability coefficient was calculated for each of the constructs and the result is shown in Table 6. The questionnaire consist of 14 items and each item was accompanied by a 5point Likert scale, with 1 denoting the most disagreeable and 5 denoting the most agreeable. The findings for effectiveness of using PBLAssess can be seen in Table  7.

A. Expectancy Students Toward Assessment
The assessment study indicated that the PBL approach significantly improved student's performances. According to the results, students in PBL are able to accurately judge the performance of their peers compared to their own performance. PBL assessment relies on evaluation principles similar to other teaching and learning modalities (Montemayor, L.E., 2004). In conclusion, the respondents have been positive and were very supportive of the assessment approach using PBL.

B. Learners Show More Positivity Toward PBL
Essentially, assessment in PBL allows student to work in a group in order to solve problems given. With the implementation of PBLAssess, students performed better on assessment using PBL approach. The results had indicated that there was a relationship between student performances and preferences when applying assessment in PBL. Hence, assessment in PBL should focus not only on the process itself, but also on the outcomes. This is in line with Uden & Beaumont (2006).In contradiction, Neo & Chyn (2002), stated that PBL assessment content, technical expertise and skills such as problem solving skills, self directed learning skills and teamwork skills should be assessed.

CONCLUSIONS
From the results of this study, it can be concluded that using peer assessment provide students the experience and opportunity to clearly express their thoughts and to refine doubts on the topic with their fellow students. Self and peer assessment promotes some substantial value in the learning process. Self assessment allows the learners to compare the standards achieved by the other learners against their own work (Race et al., 2005). It usually allows students to assess aspects of their work such as the range of vocabulary, originality and structure. It is also recognized that peer assisted learning which can have a motivating effect on the teams and mentoring between teams should be encouraged and rewarded (Frank, M., & Barzilai, A., 2004). From this study, it is hoped that assessment in PBL can be implemented to other technical subjects at UTeM especially in the field of ICT and Engineering.
The following are some questions to answer: How the scaffolding provided in assessment of the PBL could affect student's performance? What could be the impact if we embed digital games in PBL courseware in terms of their assessment and problem solving? Research on these questions will help to improve the PBL especially in assessment part.