Degree of online collaboration and team performance: a case study.
As our society has moved forward, the division of labor has become more obvious. There is very little one person can do alone to make any substantial advances in research and knowledge. The ability to work as part of a team and to coordinate the efforts of a team is becoming more important to the advancement of knowledge and the success of the employee in any job (Johnson & Johnson, 2004). To prepare students for the real work environment, our schools are responsible for promoting collaborative learning and teaching necessary collaboration skills.
Increasing the opportunity for interaction has been identified as a critical component for successful online learning (Moore, 1989; Roblyer & Ekhalm, 2000). Collaborative learning, allowing students to work in groups and encouraging them to share multiple perspectives of views, seems to be a key to promote the interaction between students (Schrage, 1990). In addition, establishing a sense of belonging to a community is essential in a collaborative learning experience (Kirschner, 2004; Palloff & Pratt, 2005).
To study the issue of online group dynamics, Hathorn and Ingram (2002) examined the characteristics of online group collaboration and concluded four different characteristics:
1. Participation. Active participation is necessary in order for students to learn (Brown, Collins, & Duguid, 1996). When participation is analyzed in online settings, it is often only measured as the number of posted messages (Mazzolini & Maddison, 2003).
2. Interdependence. Group members should support each other by making every effort to "listen" attentively and respond quickly. They should offer to share the workload and show commitment to each other. A group cannot function without interdependency, which is a key ingredient of successful online group work (McConnell, 2005).
3. Synthesis. The sum of collaboration should surpass what an individuals is able to achieve on his or her own. The negotiation of multiple perspectives leads to novel ideas and creates group synergy (Stahl, 2004).
4. Independence. Not relying on instructors for answers is necessary for students to develop problem-solving skills with peers (Kaye, 1992).
Over the past decade, there has been a great deal of interest and research in online collaborative learning. However, research findings on the relationship between educational achievement and the use of online collaborative learning methods are inconsistent (Sonnenwald & Li, 2003; Thompson & Ku, 2006; Uribe, Klein, & Sullivan, 2003). This inconsistency was caused by the complicated nature of collaborative learning and different evaluation standards researchers used to measure collaborative learning. This study extends prior research by focusing on understanding the social dynamics within the group in depth. The research question--What was the relationship of the degree of collaboration to the quality of the team project?--was investigated.
Thirteen female participants were enrolled in an online graduate level course in a special education program in 2008. This course was offered in Blackboard over the span of 16 weeks. The instructor of the course randomly divided participants into four collaborative teams as Team 1 (four members), Team 2 (three members), Team 3 (three members), and Team 4 (three members). All participants had previous online learning experiences.
The composition of the teams changed in the middle of the semester. After the second collaborative project was completed, one member from Team 2 withdrew from the class due to personal reasons. The instructor offered the remaining students in Team 2 the option to continue working as a small team or to join a different team. Both team members chose to discontinue Team 2. One of them joined Team 3 and the other joined Team 4 for the last two collaborative projects. Therefore, Team 2 was disbanded and Team 1, Team 3, and Team 4 each consisted of four members.
This research was conducted as a case study as it categorically fell into the characteristics set forth by Merriam (1998). Two different data sources--discussion archives and group projects--were collected to answer the research question.
Participants were instructed to utilize the following online communication tools to work collaboratively for this course: e-mail, group discussion board, and file exchange. If the group decided to use e-mail to communicate with each other, they were instructed to send a copy of their e-mail to the instructor of this course as well as to the first author of this study. Data generated from online communication tools of Blackboard were automatically archived.
Each team had to complete four collaborative group projects for a total of 100 points throughout the semester. Each of the four group projects was worth 25 points. These four projects were graded by the instructor and a higher score indicated a better project quality. The purpose of this data source was to evaluate the teams' performance in the course.
During week 1, students introduced themselves and got to know each other on the discussion board. We posted an online collaboration guidelines document on the discussion board, which set expectations and recommendations for online collaboration. The instructor randomly divided participants into four collaborative teams. In week two, each team reached consensus of how to work collaboratively. From week three to 16, teams completed four collaborative projects, which were submitted in week 6, 9, 14, and 16 respectively.
The coding system used in this study was adopted from Hathorn and Ingram's study (2002), which used the four attributes (participation, interdependence, synthesis, and independence) that were mentioned in the Introduction section.
The first two attributes of collaborative discussion patterns were participation and interdependence. Participation was measured by counting the number of messages and statements made by each team. Interdependence was defined as the pattern of qualified interaction among team members to bring about active responses to one another. A qualified interaction pattern involved participation of all team members. For example, a simplest qualified interaction pattern among three people could be a-b-c.
a: Shall we include laws to support getting the right services in our paper?
b: Yes, but there were many possible areas to cover the right services.
c: I agree. I think it's important to only cover the right services in educational settings.
If c in the team did not participate in the above discussion, the interaction pattern would be a-b in this case and would not be counted as a qualified interaction pattern. In addition, a more complex pattern of interaction could be demonstrated as a-b-c-x (x could be a, b, or c), which meant not only all three people contributed their ideas to the discussion, but one of them also summarized their ideas or proposed a solution to the problem. The team that generated a higher number of statements and qualified interaction patterns per person demonstrated higher levels of participation and interdependence.
The third and fourth attributes of collaborative discussion patterns included synthesis and independence. Synthesis of information was measured by counting the number of statements that contributed to generating new ideas. For a demonstration, let's examine the following response from a team member. "I like how you approached this question. We may want to include reasons to go with this law such as the need for highly qualified teachers and specialized training." The first part of her response "I like how you approached this question." acknowledged her team member's work; however, it did not generate any new ideas for the team. In comparison, the reminder of her response was counted as one synthesis statement because a new idea was generated. Independence was measured by the number of messages addressed to the instructor either by e-mail or posting. The team that generated a higher number of new ideas per person demonstrated a higher level of synthesis while the team that addressed fewer messages to the instructor per person displayed a higher level of independence.
Scores of teams' performance on the four projects were obtained from the instructor. Then the performance was ranked based on the average scores each team received on Projects 1 and 2 as well as the average scores each team received on Projects 3 and 4 due to the change of composition of the team members. A higher score meant that the team produced a better quality project than the team that had a lower score.
Degree of Collaboration
Team 2 was disbanded after Project 2 and the two remaining team members joined Team 3 and 4 respectively for the last two collaborative projects. As a result, the analysis of the degree of collaboration was broken down into two parts, as demonstrated in Table 1: degree of collaboration for Projects 1 and 2 among four teams; and degree of collaboration for Projects 3 and 4 among three teams.
Degree of collaboration was analyzed in terms of participation, interdependence, synthesis, and independence among teams. The level of participation was determined by the number of statements posted on Blackboard. For Projects 1 and 2, Team 1 was the most active team among the four teams. The four participants in the team posted a total of 952 statements (an average of 238 statements per participant). Team 3 was the second most active team. Three participants in the team posted a total of 686 statements, which equals to 228.7 statements per participant. Three participants in Team 2 posted a total of 346 statements, 115.3 statements per participant, which was slightly more than a total of 339 statements from Team 4, 113 statements per participant. For Projects 3 and 4, Team 3 surpassed Team 1 to post the highest number of statements, a total of 675 (168.8 per participant) compared to 663 (165.8 per participant) from Team 1, while Team 4 only posted 219 statements (54.8 per participant).
The level of interdependence among team members was determined by the number of qualified interaction patterns. For Projects 1 and 2, Team 3 generated a total of 18 qualified interaction patterns while Team 1 and Team 4 each generated a total of 15 and 9 qualified interaction patterns, respectively. Team 2 did not generate any qualified interaction patterns at all, revealing the lowest interdependence among four teams. All interactions in Team 2 happened only between two of the three team members, and never was the entire team involved to share ideas or make decisions. For Projects 3 and 4, Team 3 once again generated more qualified interaction patterns than the other teams, a total of 10 qualified interaction patterns compared to 7 from Team 1 and only 1 from Team 4.
In terms of synthesis, for Projects 1 and 2, three participants in Team 3 made a total of 293 statements (an average of 97.7 statements per participant) that helped them generate new ideas and solve problems, while four participants in Team 1 made a total of 352 statements (88.0 statements per participant). A total of 221 statements (73.7 statements per participant) helped Team 4 generate new ideas. In contrast, Team 2 made a total of 69 statements (23.0 statements per participant) that generated new ideas. For Projects 3 and 4, Team 1 made a total of 294 statements (at an average of 73.5 statements per participant) that helped them generate new ideas, which was slightly more than the 292 statements made by Team 3 (73.0 statements per participant). Team 4 did not generate many new ideas for the last two projects; only 62 statements were made (15.5 statements per participant) to generate new ideas.
Lastly, the level of independence was measured by the number of messages each team sent to the instructor. For Projects 1 and 2, Teams 1, 3, and 4 worked almost entirely independently. Team 1 sent two e-mails (0.5 per participant) to the instructor to ask suggestions, Team 3 sent one e-mail (0.3 per participant) to the instructor to clarify instruction, and Team 4 did not contact the instructor at all. In contrast, Team 2 sent a total of 9 messages (3.0 per participant) to the instructor via e-mail and posting, indicating a lower level of independence. They contacted the instructor to clarify project instructions and file complaints of one team member not doing her share of the work. For Projects 3 and 4, Team 4 contacted the instructor three times (0.8 times per participant) via e mail about their limited availabilities due to unexpected family situations; Team 3 contacted the instructor twice via e-mail to verify their interpretations of the project requirements (0.5 per participant); and Team 1 worked entirely independently. In comparison, Team 4 displayed the lowest level of independence.
The ranking for the degree of collaboration was adopted from Thompson and Ku's study (2006) by the sums of rankings for the level of participation, interdependence, synthesis, and independence, and smaller sums indicated higher degrees of collaboration. To rank the level of participation, synthesis, and independence among the teams, all numbers were normalized per participant for each team. To rank the level of interdependence, the total number of the qualified statements generated by each team was used. For the first two projects, the sums of rankings for all four teams were: 8 (Team 1), 15 (Team 2), 6 (Team 3), and 11 (Team 4). Since Team 3 had the smallest sum, it had the highest degree of collaboration among the four teams. Team 1 was ranked at second place and Team 4 was the third. Team 2 had the largest sum and appeared to be the least collaborative among four teams. For the last two projects, the sums of rankings for the remaining three teams were: 6 (Team 1), 6 (Team 3), and 12 (Team 4). Team 1 and Team 3 were both ranked at the top, displaying a higher degree of collaboration than Team 4.
Quality of Group Projects
Each team had to complete four collaborative group projects for a total of 100 points throughout the semester. The team rankings of the project quality were determined by the average of Project 1 and Project 2 as well as Project 3 and Project 4 scores. For the first two projects, Team 1 received the highest average score (24.8), followed by Team 3 (23.8), Team 4 (23.5), and Team 2 (22). For the last two projects, Team 3 received the highest average score (23), followed by Team 1 (21), and Team 4 (20.5).
In terms of degree of collaboration, for the first two projects, Team 3 was the most collaborative team followed by Team 1 and Team 4. Team 2 barely interacted with each other, displaying the lowest degree of collaboration.
The results show that the more collaborative teams--Team 3 and Team 1 produced the better quality projects than the less collaborative teams--Team 4 and Team 2. However, the most collaborative team, Team 3, did not produce the best quality projects. The best quality projects were created by Team 1, the second most collaborative of the four teams. For the last two projects, both Team 1 and Team 3 were the most collaborative teams, while Team 4 was the least collaborative team. Both Team 1 and Team 3 produced better quality projects than Team 4. The rankings for all four teams on degree of collaboration and project quality are shown in Table 2.
Overall, there is a strong relationship between degree of collaboration and the quality of the collaborative project. Interestingly, Team 3 was more collaborative than Team 1, but Team 1's average score of the first two projects (24.8) was higher than Team 3's average score (23.8). One possible explanation is that Team 1 and Team 3 did not have the same number of team members during these two projects. When being assessed on interdependence, Team 1 of four team members did not have the equal opportunity to reach the qualified statements as Team 3 of only three team members. The probability of an a-b-c-d interaction pattern to occur is less than the probability of an ab-c interaction pattern. Having teams of different sizes was one of the limitations of this study. It was difficult to conclude who was a more collaborative team between Team 1 and Team 3. However, both Team 3 and Team 1, the two most collaborative teams, produced better projects than Team 2 and Team 4, the two least collaborative teams. For the last two projects, the most collaborative teams, Team 3 and Team 1, produced better projects than Team 4, the least collaborative team.
This study confirmed that a more collaborative team participated more in group discussion, initiated more interactions among team members, generated more new ideas through discussion, and solved problems more independently with less guidance from the instructor, and ultimately achieved better learning results. Based on these findings, one recommendation to improve team performance in an online course is to encourage higher degree of collaboration among team members. Strategies such as providing team members with online collaboration guidelines, self and peer evaluation forms, and detailed rubrics will likely maximize the benefits of online collaboration.
In regards to future research, Web 2.0 tools might be an area of interest. Emerging technologies such as wikis and blogs, which encourage user participation, dynamic content, and collective intelligence, have gained popularity among teachers and students. Future research might investigate how to design, implement, and evaluate online collaborative learning using Web 2.0 technologies.
Brown, J. S., Collins, A., & Duguid, P. (1996). Universities in the digital age. Change, 28(4), 10-19.
Hathorn, L. G., & Ingram, A. L. (2002). Cooperation and collaboration using computer-mediated communication. Journal of Educational Computing Research, 26(3), 325-347.
Johnson, D. W., & Johnson, R. T. (2004). Assessing students in groups: Promoting group responsibility and individual accountability. Thousand Oaks, CA: Corwin.
Kaye, A. (1992). Learning together apart. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing (pp. 117-136). Berlin, Germany: Springer-Verlag.
Kirschner, P. A. (2004). Introduction to Part I of two-part special issue: Design, development, and implements for collaborative learning. Educational Technology Research and Development, 52(3), 39-46.
Mazzolini, M., & Maddison, S. (2003). Sage, guide or ghost? The effect of instructor intervention on student participation in online discussion forums. Computers & Education, 40(3), 237253.
McConnell, D. (2005). Examining the dynamics of networked e-learning groups and communities. Studies in Higher Education, 30(1), 25-42.
Merriam, S. B. (1998). Case study research in education: A qualitative approach. San Francisco, CA: Jossey-Bass.
Moore, M. (1989). Three types of interaction. The American Journal of Distance Education, 3(2), 1-6.
Palloff, R., & Pratt, K. (2005). Collaborating online: Learning together in community. San Francisco, CA: Jossey-Bass.
Roblyer, M. D., & Ekhaml, L. (2000). How interactive are YOUR distance courses? A rubric for assessing interaction in distance learning. Online Journal of Distance Learning Administration, 3(3).
Schrage, T. M. (1990). Shared minds: The new technologies of collaboration. New York, NY: Random House.
Sonnenwald, D. H., & Li, B. (2003). Scientific collaboratories in higher education: Exploring learning style preferences and perceptions of technology. British Journal of Educational Technology, 34(4), 419-431.
Stahl, G. (2004). Building collaborative knowing: Elements of a social theory of CSCL. In J. W. Strijbos, P. A. Kirschner, & R. L. Martens (Eds.), What we know about CSCL and implementing it in higher education (pp. 53-85). Norwell, MA: Kluwer.
Thompson, L., & Ku, H. Y. (2006). A case study of online collaborative learning. Quarterly Review of Distance Education, 7(4), 361-375.
Uribe, D., Klein, J. D., & Sullivan, H. (2003). The effect of computer-mediated collaborative learning on solving ill-defined problems. Educational Technology Research and Development, 51(1), 5-19.
University of Northern Colorado
* Ling Thompson, Instructional Designer, College for Professional Studies, Regis University, Denver, CO. E-mail: firstname.lastname@example.org
Table 1 Degree of Collaboration Team 1 Team 2 Project 1 and Project 2 (N = 4) (N = 3) Total Statements 952 346 (Participation) Interaction Pattern 4 a-b 21 a-b (Interdependence) 3 a-b-a 8 a-b-a 4 a-b-c 2 a-b-a-b 3 a-b-c-d 2 a-b-a-b-a 1 a-b-a-c 0 qualified 1 a-b-c-a-d 1 a-b-c-c-a 1 a-b-c-a 1 a-b-a-c-a-c 1 a-b-a-b-c 1 a-b-a-c-a-b 10 a-b-c-d-x 15 qualified New Ideas 352 69 (Synthesis) Message to the Instructor 2910 (Independence) Team 1 Project 3 and Project 4 (N = 4) Team 2 Total Statements 663 N/A (Participation) Interaction Pattern 7 a-b N/A (Interdependence) 4 a-b-a 2 a-b-c 1 a-b-c-a 1 a-b-a-b 1 a-b-a-c 1 a-b-c-d 1 a-b-a-b-a-c-d 1 a-b-c-a-c 1 a-b-c-a-c-a-d 1 a-b-c-a-b-c-a 1 a-b-c-b-a-c-a-c-b-d 3 a-b-c-d-x 7 qualified New Ideas 294 N/A (Synthesis) Message to the Instructor 0 N/A (Independence) Team 3 Team 4 Project 1 and Project 2 (N = 3) (N = 3) Total Statements 686 339 (Participation) Interaction Pattern 7 a-b 7 a-b (Interdependence) 3 a-b-a 2 a-b-a 4 a-b-c 4 a-b-c 2 a-b-a-c 1 a-b-a-b-c 2 a-b-a-b-c 4 a-b-c-x 10 a-b-c-x 9 qualified 18 qualified New Ideas 293 221 (Synthesis) Message to the Instructor (Independence) Team 3 Team 4 Project 3 and Project 4 (N = 4) (N = 4) Total Statements 675 219 (Participation) Interaction Pattern 2 a-b 4 a-b (Interdependence) 2 a-b-a 1 a-b-c-b 3 a-b-c 1 a-b-c 2 a-b-c-d 1 a-b-a-c-b-c 2 a-b-c-a-d 1 a-b-c-d 1 a-b-a-b-c 1 qualified 1 a-b-c-a-b-d 1 a-b-c-a-b 5 a-b-c-d-x 10 qualified New Ideas 292 62 (Synthesis) Message to the Instructor 2 3 (Independence) Table 2 Rankings for Degree of Collaboration and Project Quality Project 1 and Project 2 Team # Participation Interdependence Synthesis 1 (238.0) 2 (15.0) 2 (88.0) 2 3 (115.3) 4 (0.0) 4 (23.0) 3 2 (228.7) 1 (18.0) 1 (97.7) 4 4 (113.0) 3 (9.0) 3 (73.7) Project 3 and Project 4 Team # Participation Interdependence Synthesis 1 2 (165.8) 2(7) 1 (73.5) 2 N/A N/A N/A 3 1 (168.8) 1 (10) 2 (73.0) 4 3 (54.8) 3 (1) 3 (15.5) Project 1 and Project 2 Degree of Overall Team # Independence Collaboration Ranking 1 3 (0.5) 8 2 2 4 (3.0) 15 4 3 2 (0.3) 6 1 4 1 (0.0) 11 3 Project 3 and Project 4 Degree of Overall Team # Independence Collaboration Ranking 1 1 (0.0) 6 1 2 N/A N/A N/A 3 2 (0.5) 6 1 4 3 (0.7) 12 3 Project 1 and Project 2 Project Team # Quality 1 1 (24.8) 2 4 (22.0) 3 2 (23.8) 4 3 (23.5) Project 3 and Project 4 Project Team # Quality 1 2 (21.0) 2 N/A 3 1 (23.0) 4 3 (20.5)
|Printer friendly Cite/link Email Feedback|
|Author:||Thompson, Ling; Ku, Heng-Yu|
|Publication:||Quarterly Review of Distance Education|
|Date:||Jun 22, 2010|
|Previous Article:||Instructional design issues in a distributed collaborative engineering design (CED) instructional environment.|
|Next Article:||Strategic Applications of Distance Learning Technologies, by Mahbubur Rahman Syed.|