Designing protocol studies of the writing process: An Introduction

Taylor Stukes

Flower, Linda S., Hayes, John R. and Swarts, Heidi. “Designing protocol studies of the writing process: An introduction.” R. Beach and L. Bridwell. New Directions in Composition Research. New York: Guliford, 1984.

What can protocol studies contribute to our understanding of the writing process? What kinds of studies can be done?

Definitions

Experimental Protocol: an innovative & powerful research tool developed by cognitive psychologists. Works by sequentially recording a subject’s attempts to perform a task

Protocol Analysis: Theory-driven form of research demanding vigorous analysis

·         Strengths: weed out half-formed initial hypotheses & add substance and depth to a theory-driven analysis

·         How do they work?

o    Develop hypothesis à Derive it from a coding scheme à Find hypothesis disconfirmed

Techniques of Protocol Gathering

·         Thinking Aloud Writing Protocols: Subjects are recorded as they state their thoughts during a timed writing task. After, the recordings are transcribed for coding.

·         Protocol analysis gives detailed data about the subject’s processes of planning, goal-setting, decision-making, and revising

·         Retrospective Studies: Subject reports on writing process after the fact; many limitations accompany this type of study

Preliminary Parsing

1.       There is no paragraphing in a protocol, so typist should number lines and pages. Clauses are the most basic way to measure the subject’s utterances.

2.       Match the notes against the protocol (what are they reading v. what are they writing)

3.       Chart the first appearance of ideas, notes & sentences in relation to their order in the final text

4.       Episodes: Units of activity in which subject’s compose

Coding Writing Processes – driven by the research question being asked & the researcher’s definition of the phenomenon in question

·         Planning: includes generating & organizing ideas, goal-setting (content and process goals)

·         Translating: includes creating formal written text & versions the subject tries out orally

·         Reviewing: includes reading, evaluating, and making changes in the text

Using Protocols in Writing Research

1.       Exploratory Studies: explore uncharted territory, can generate more specific questions

2.       Studies that give a problem structure with a taxonomy or hypothesis

3.       Comparative Studies: Examine differences between age groups, skill levels, etc.

4.       Studies that model writing processes

Reliability in Protocol Analysis

·         Like looking at clouds, people will see different things (68)

·         Agreement among investigators, developed coding scheme for judges to use

·         Ensure judges are familiar with researcher’s theory or context