Case Study 1: PDI ENTREPRENEURSHIP AND BUSINESS PLANNING
One of the cases of our wiki research involved a collaborative writing in a Product Design and Innovation (PDI) studio course. PDI is an interdisciplinary studio-based curriculum that integrates technical, social, and creative approaches to the design and analysis of innovative product concepts. The studio on business planning is the sixth in a series of studio courses each emphasizing a different facet of design. PDI students are required to enroll in all of the studios in sequential order, so they travel through the curriculum as a cohort and, hence, know each other very well by the sixth studio.
The instructional goals of the wiki assignment for this course were twofold. First, the instructors hoped to foster deep collaboration among students, as discussed above. Specifically, the pedagogical goal was to encourage students to read and comment upon other team members' writing in process, rather than waiting for a formal, final “hand off” for review and revisions after the primary writing was “finished.” Here, the goal was less to circumvent students’ tendency to “divide and conquer” and more to promote earlier and more frequent peer review. Second, the instructors wanted to use the wiki as a tool for assessing “degree of participation” in the collaborative writing assignment, allowing us a more nuanced understanding of who contributed what than could be established by knowing only the primary student responsible for each section. In other words, we hoped to use the wiki to assess not only who took primary responsibility for each section, but also who contributed how much to other students’ sections.
The wiki assignment was integrated into the studio's final project, which required two to three-person student teams to develop customized business plans for original product concepts. The business plans were to be 10-30 page documents, based on a common template provided by the instructors and adapted to individual projects by students employing the Clearspace wiki. The class was introduced to the wiki and the associated collaborative writing assignment immediately after the common business plan template was distributed, approximately four weeks prior to the final project due date. The wiki assignment required teams to adapt the common template to their individual projects by suggesting specific changes to the template and justifying those changes using the wiki. In order to initiate dialogue on the wiki, each team member was instructed to contribute at least one original idea to the template modification phase and to respond to at least two other contributions made by other team members. The outcome of the assignment was to be both a working outline for the teams’ business plans as well as documentation of the thinking that went into their customization of the template. Students were encouraged to continue using the wiki as a collaborative workspace upon completion of the first-stage assignment, but were not required to do so.
Student Engagement & Experiences
Student engagement with the assignment was highly variable, with some teams satisfying the bare minimum requirements and following the instructions to the letter and other teams using the wiki regularly from the initial assignment until near-completion of their business plans. Out of seven team projects, one team reviewed the wiki’s capabilities and immediately requested permission to switch to Google Documents (which they already had been using since the beginning of the project); two teams satisfied only the minimum requirements of the week-long assignment; one team continued using the wiki for one week after completion of the initial assignment (discussing collaborative process and adding initial content and notes); one team used the wiki actively for two additional weeks before switching to Google Documents; and two teams used the wiki consistently over the entire course of the project. Level of engagement with the wiki spanned the spectrum, from simple updates as to work completed and pending on one end of the spectrum to meta conversations about business plan content on the other. Task-scheduling and team management postings occupied a middle spot along this spectrum.
Although student engagement with the assignment was variable, students’ experience with the wiki, and particularly the chosen interface, was strikingly consistent: they didn’t like it. As one student from one of the most engaged teams put it in a comment within her team’s wiki space, “I feel like this interface will seriously inhibit our ability to collaborate [sic] and work together since only one person can be working on a section at the same time. The interface actually encourages us to divide up the work rather than work in a more collective manner.” It seemed, the more students knew about collaborative authoring software, the more frustrated they were with the assignment requiring them to use a tool they otherwise would not have chosen.
Instructor Assessment of Project
From the perspective of the instructors, the assignment as carried out posed two serious problems and yet also provided valuable educational opportunities. The first major problem was students’ perception that the assignment inhibited their progress in writing their business plans, not because they had to slow down and reflect upon others’ contributions but because it added an administrative “make work” step to what was already — in their assessment — a highly collaborative process. Both of the course instructors encountered teams struggling with the assignment. These students seemed to think they had to project through the wiki a writing process that did not accurately represent how they worked. Their concerns were partly allayed when the instructors clarified that they did not have to prove to us a certain mode of working, but rather that they were to use the wiki to document how they were already working. The clarification helped, but students felt the assignment still entailed an extra and perhaps unnecessary step if the goal was to enhance collaboration.
The second major problem with the assignment was that, when completed, the wiki entailed far less reflection on business plan content than we hoped to achieve. Even students who used the wiki consistently over the four weeks (between the initial assignment and the final project due date) used it primarily as a document manager — a common space to hold the latest version of each section — and not as a tool for “deeply collaborative” writing. Whether this is due to limitations in the tool, as suggested by the student quoted above, limitations in how the project was assigned, or something else, the instructors could not surmise. At least in our studio course environment, where students already have considerable time to work together collaboratively on common assignments, the wiki’s contribution to deep collaboration is not immediately apparent.
Nevertheless, the wiki and the wiki assignment did offer some clear benefits to the course. To the instructors, the wiki provided a tool for relatively easily assessing who contributed what and when, which was one of our instructional goals. At end of term, we can readily go back into the space and determine what each student contributed, both in terms of original document text and in terms of feedback to other team members. While wikis tend to be celebrated for their contribution to more lofty goals, the ability to assess individual participation in team projects is not trivial in many educational contexts. Another clear benefit of the wiki assignment in the context of our course was that it encouraged students to face the reality of writing — that every document starts with a blank page — much earlier than they otherwise might have. By putting the (nearly) blank page in a public forum for everybody to see, students confronted their own contributions and those of their teammates in a material way: it was there or it wasn’t. This probably helped students to work earlier and to contribute small pieces more often, which was another of our goals.
Case Study 2: INTRODUCTION TO ENGINEERING DESIGN
The second case used in this research was a team-authored "Introduction" section to the final project report for an Introduction to Engineering Design (IED) course. IED is required for all sophomore engineering students at RPI. There were several writing assignments in this class, the most comprehensive of which was the final project report. This report included standardized sections, including Introduction, Methods, Results, and Conclusions. It also included appendices, usually containing technical drawings, pictures, and calculations. The final reports were 40 to 60-page documents that included graphics, pictures, calculations, programming code, and results data tables, in addition to standard textual descriptions. The Introduction section was selected for the wiki exercise because it usually involves background information that is gathered early in the project, and thus is not dependent on final results. Additionally, selecting the Introduction was meant to promote early discussion among team members regarding the effectiveness of the wiki in fostering collaborative writing without the complications of handling a lot of non-text components. Based on the previous effort to incorporate wikis in the Introduction to Air Quality course in Fall 2007, it became apparent that students experienced difficulty handling such files.
The use of wikis in the this class was promoted for several reasons. Major goals of the course were to encourage teamwork while learning the process of design and technical writing. The teams that participated in this exercise consisted of five to six students from various engineering disciplines (primarily mechanical, civil, and environmental engineering). The wiki exercise was intended to contribute to meeting the course's major goals, as well as to increase the technical proficiency of the students in using novel communications tools. Each student was expected to contribute to the writing of his or her team's Introduction section and to participate in on-line discussions of their writing process. Although only the Introduction was required to be written using the wiki, the students had the option of using the tool to complete the entire writing assignment.
The instructional goals of the wiki assignment were similar to those of the first case described in this research. The first goal was to provide a forum for discussion, document sharing, and collaborative writing that would enable deep collaboration among team members. The second goal was to provide the instructor with a means to assess the amount and quality of student contributions to the final product. The third goal was to effectively force the students to start their writing early enough to receive and accommodate instructor feedback.
The extent to which each of these goals were achieved varied widely by team. One team, for example, seemed to engage in more regular and substantive discussions while another team barely used the wiki.There were a total of five teams.
The students were required to write the "Introduction" section, including a "Benchmarking" sub-section, of their Final Project Report using the wiki. The instructor selected the "Introduction" section for this exercise because it would minimize possible complications with the handling of graphic images and other non-Windows documents (e.g., Computer-Aided Design, or CAD, drawings) that are commonly present in other sections of the report. Students had a period of two weeks to work on this section and were expected to engage in group discussions in parallel to the building of a wiki document. The two-week time window was a guideline to encourage localized “traffic” in the wiki website that could be used to assess the effectiveness of the tool for communication in a controlled manner. This concentrated composing period allowed the instructor to assist the students better with planning and to give the students earlier feedback on their written material.
The instructor provided the students with a detailed prompt describing the assignment. This document gave clear guidelines for the students to follow and to use as a reference. The assignment prompt also included some discussion on the research goals that the instructor and her collaborators had in mind. The intention was to enlist the participation and support of the students as active contributors to this research enterprise. For example, the governing hypotheses to be tested by the wiki exercise was explained to the students on the assignment prompt as follows:
“For this assignment, we ask that you help us test the following hypotheses:
1) The overall quality of writing improves with the input of more people.
2) Deep collaboration improves the quality of writing produced by groups.
You will experiment with this process by engaging in deep collaboration on the wiki itself.”
The intricacies of the methodology are discussed elsewhere in this paper; however, the primary goal for including this information on our assignment description was to inform the students that the purpose of this exercise was not trivial and had the potential to impact future students. It is difficult to say whether this approach was as effective as leaving the underlying purpose unstated.
In order to facilitate the process and shorten the “learning curve,” students received two short tutorials on how to use the wiki website, which were designed exclusively for this course. After the first tutorial “Writing Using Wikis,” which occurred the day the wiki assignment was first introduced, a number of students were able to sign on and get started; however, a glitch in the system prevented some students from successfully signing on at that time. Once that technical difficulty was resolved, all students were able to sign on and explore the wiki site. The second wiki tutorial occurred when the students received the wiki writing assignment, which was about a month after the first tutorial. By then, the technical problems with the wiki had been resolved and the students were free to start writing the required section of their final project report.
Student Engagement and Experiences
Working with information and communications technology is commonplace for our students and the majority of them managed to navigate and explore the wiki website fairly proficiently and quickly. While the initial deployment of the wiki website was not perfect, the technical difficulties were quickly resolved once they were identified. In this particular case, the initial glitch in the system was identified by the students but was not quickly brought to the attention of the instructor. (The problem entailed an inability to register as a user in the wiki website outside an initial window of time. Having a restricted time to register had not been part of the researchers’ plan, so it was not anticipated.) Once the instructor and the researchers recognized the problem, it was resolved satisfactorily. The level of engagement varied from student to student and involved a wide spectrum of behavior. A few students did not participate in the exercise (as evidenced by their lack of entries in the document creation); however, the majority of the students participated actively in the exercise.
When asked to reflect on their experience, students shared a general view of dissatisfaction with the wiki, with few exceptions. The main concerns had to do with the less-than-optimum format for the chosen wiki in the given class conditions. For example, students often worked at the same location, making the use of a wiki almost unnecessary. Similarly, in some cases, teams designated one or two members to do all the uploading of documents on the wiki site. This behavior made it particularly difficult to assess the individual writing contribution of each member. This shortcoming was worked around using a peer-evaluation form where students provided feedback to the instructor on student participation/contribution within each team.
Instructor Assessment of ProjectOne of the advantages of using the wiki is that there was a record of usage to easily track who were the most assiduous users as well as the nature of their contributions to the document's content and discussion. The tool was useful to the instructor, allowing her to gain insight on the effectiveness and dynamics of the student teams as well as helping in assigning credit more fairly to individual team members. It was also a good way to always have access to the latest version of the document being generated and to have access to the background and behind-the-scenes discussions regarding the exercise. This accessibility enabled timely feedback and direct communication with students with specific problems (e.g., limited participation, focusing on the wrong issues, etc.).
Overall, there was a lack of flexibility in document formatting that dissuaded students from using the wiki to generate complete reports. As this technology progresses, formatting flexibility should increase and user satisfaction should also increase. Another major problem was that some students did not follow the instructions regarding participation. Specifically, assigning one or two students to do all the uploading of documents onto the wiki subverted one of the assignment's main goals: to assess individual contributions. This problem could have been averted by connecting wiki contributions directly to individual student grades and outlining this correlation in the assignment prompt. Also, although the researchers provided two training sessions to the students so they could learn the basics of using and navigating the wiki, a more hands-on exercise would probably have been more effective.
Finally, having the team members meet regularly in a common location seemed to make use of the wiki less compelling. If the team participants were in different locations, they would have had a greater incentive to use the wiki to communicate and build their documents. Using wikis for document generation using geographically disperse team members and/or asynchronous collaboration would probably best capitalize on this tool.
Continue reading the Results and Analysis section.