A Faculty-Student Dialogue on Assessment: Rubrics as Vehicles ...

A Faculty-Student Dialogue on Assessment: Rubrics as Vehicles ...

A Faculty-Student Dialogue on Assessment: Rubrics as Vehicles for Communicating Learning Expectations in the Field Eric J. Pyle James Madison University Assessment vs. Evaluation Assessment involves comparing information gathered from subjects relative to some established goal or objective (Kizlik, 2009). These goals, objectives, or outcomes are set in advance, and should be clear to both instructors and students. Evaluation allows the establishment and communication of the worth of an activity (Kizlik, 2009). This worth can be determined by the extent to which decisions of instructional approaches, arrangements, organization, etc., are effective in aiding students into reaching the desired outcomes.

Goals vs. Objectives Goals reflect the overall outcome expected of an learning activity, such a capacity to generate geologic descriptions of areas; Objectives are the specific means by which students can demonstrate that they have met the goal expected of them, such as generating a geologic map, cross-section, and lithologic descriptions for a given area. Objectives Objectives consist of three parts: Condition refers to the context of the learning experience, the setting, the materials, available or to be used; Performance is a statement of what students are supposed to do, a description of the task or demonstration; Criterion is the indication that the particular objective has been met, and to what degree it has been met.

Example: Given a base map and aerial photograph of an area, the student will produce a geologic map and crosssection that describes the relative sizes and orientations of the lithologies and structures for the region. Chiappetta & Koballa, 2006 Assessments and Objectives Knowledge specific knowledge, recall, terminology, etc.; Skills specific performances, equipment, tool-use, etc. Dispositions scientific habits of mind, persistence, question-posing, etc. The struggle is to ensure that the entire range of learning described in the goals is expressed through appropriate objectives and related assessment tasks validity. Types of Assessment Objective/Knowledge items - multiple choice,

fill-in-the-blank; uses binary, nominal criterion Prescribed/Lexical tasks - problems, short answers; procedures as well as the task are of value; uses ordinal level criterion measure; Open-Ended/Prototype tasks - essays, artifacts; uses interval level criterion measure on 0-100 scale. The struggle in each form is to provide a clear and consistent determination of mastery of the related objectives reliability The special case of field learning Expectations are not just limited expressions of knowledge but the application of that knowledge; Expectations include acquisition, demonstration, and mastery of skill sets;

Expectations include consistency, persistence, attention to detail, and synthesis of working hypotheses. The special case of field learning: Knowledge Cognitive Earth Science level Sample Verbs Define, describe, Knowledge identify Interpolate, Comprehension estimate, predict Compute, modify, Application relate, use Diagram, divide, Analysis infer Arrange, generate,

Synthesis design Contrast, interpret, Evaluation appraise Based on Bloom (1956) concept rock texture, RFM identification Draw contour lines from elevation data Graph a topographic cross-section Plot fold axis on a map Construct a geologic map from field data Assess landslide hazards from map data The special case of field learning: Psychomotor Skills Sample Indicators Earth Science action level

Imitation Manipulation Precision Articulation Naturalization Based on Dave (1975) Crude reproduction of action based on observation and minimal practice Performance from instruction with attention to form Accuracy, proportion, and exactness in performance, with minimal error Coordinating a series of acts with harmony and consistency Determination of mineral sample physical properties, such as hardness, streak, or observing

cleavage Measurement and data encoding using a Brunton compass or Jacobs staff Collection of physical and chemical data at several points along a stream Generating a map from a series of station measurements, plotted on a base map Generation of finished maps that Smooth, natural reflect multiple layers of data performance with minimum collection and procedures and of psychic energy coordinate well with field notes and diagrams. The special case of field learning: Dispositions Dispositional Level Receiving Responding

Valuing Organizing Sample Indicators Earth Science action Follow directions, locate, identify Completes assigned tasks at or above level required, or for selfsatisfaction Accepts, prefers, and commits to scientific values Following along with a field trip guidebook as a part of a field trip Once a local geologic map has been studied, a student seeks out a regional geologic map to see larger context More than one measurement of a particular parameter is sought in each location Personal values are

brought into line with scientific values Each field investigation is approached with a set of questions framed on methodologies and possible outcomes Lifestyle adoption indicative of a ased on Krathwohl, Bloom, & Masia (1973) Characterizing preference for scientific Active seeking of communications with other students and faculty on Combining Domains in Assessments

Structure Sample Task Domain-Based Rationales Objective/ Knowledge Based on observations of current stream conditions and local erosion and sedimentation patterns, make a prediction of how the stream changes when flow reaches flood conditions. Prescribed/ Lexical Knowledge recall and appropriate Correctly applying application of terminology; terminology in a lithologic Skills communicating descriptions in description using texture, written or oral form;

and mineralogy, and internal Dispositions Use of a variety of descriptive features or structures. terms in a manner that reflects possible contexts. Knowledge beliefs from prior knowledge, applied to novel situation; Skills measurements and observational descriptions of setting; Dispositions use of more than one parameter in making the prediction. Knowledge synthesizing an analogy representing the distribution and orientation Constructing an accurate of materials; cross-section from a map, or Skills drafting the cross-section with Opendistinguishing the correct consistency of measurement, to scale, from Ended/ cross-section from the map; Prototype distracters, stating reasons Dispositions Cross-section contains all for rejection. necessary detail, on Smith (1995), Lawrence

& Margolis (1999), Murphy (2002), and Sibley (2005)drafted in a manner that communicates clearly the interpretations Application: Field Notes Rubric Condition: note-taking of field situations, with general expectations of description and supporting other documentation, such as maps or lithologic descriptions Where is the Dialogue? Exchange begins with sharing the rubrics ahead of time can be stated a priori by instructors, which allows reliable Criterion measurement; or with students, which allows for closer validity of Negotiated measurement to particular or unique contexts.

Student first (and continuing) response is usually a request for clarification is finer grained not what do you want, but what do you mean Reply by, allowing more specific examples to be generated. Next exchange is the assignment of scores are broken down by the level of mastery for each dimension or Scores objective; Narrative feedback becomes focused on stated criteria. Next exchange is student formative response what do they do next time to improve, or in some cases, revision of prior submission; Less prominent is the student of response of why did I get this grade, when subscores relative to descriptors are provided. Other Applications in Field Courses: Maps and Cross-Sections Other Applications in Field Courses:

Lithologic Descriptions, Memoirs Uses and Extensions These rubrics have been used with two successive JMU field courses; In general, their use allows for quicker and more consistent turn-around of student assignments; But continued discussion among participating faculty is needed to fine-tune language relative to expectations; The Knowledge-Skills-Dispositions format is also employed in student evaluations of instruction, specific to the individual assignments in the field course. Data analysis is on-going; Rubrics are frequently uses on assignments in BA-Earth Science classes at JMU; A general rubric is in use with the JMU program evaluation Students participate in a performance task, with differential expectations based on student standing in program. References Bloom, B. S. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain. New York: David McKay Co., Inc.

Chiappetta, E. L. & Koballa, T. R. (2006). Science instruction in the middle and secondary schools: Developing fundamental knowledge and skills for teaching, 6 th ed. New York: Allyn & Bacon. Dave, R. H. (1975). Developing and writing behavioral objectives. R. J. Armstrong, ed. Educational Innovators Press. Kizlik, B. (2009) Measurement, Assessment, and Evaluation in Education. Website: http://www.adprima.com/measurement.htm, retrieved 3/10/09. Krathwohl, D. R., Bloom, B. S., and Masia, B. B. (1973). Taxonomy of education objectives, the classification of educational goals, handbook II: Affective domain. New York: David McKay Co. Inc. Lawrence, S., and Margolis, E. (1999). Concepts and cognitive science. In E. Margolis and S. Lawrence (eds.), 3-83. Concepts. Cambridge, MA: MIT Press.

Murphy, G. L. (2002). The big book of concepts. Cambridge, MA: MIT Press. Pyle, E. J., & Brunkhorst, B. (2009). Developing and applying the knowledge, skills, and dispositions needed for effective Earth science teaching. In A. Collins & N. Gillespie (Eds.) The Continuum of Secondary Science Teacher Preparation: Knowledge, Questions, and Research Recommendations. Rotterdam: Sense Publishers. Sibley, D. F. (2005). Visual abilities and misconceptions about plate tectonics. Journal of Geoscience Education, 53(4), 471-477. Smith, E. (1995). Concepts and categorization. In E. Smith and D. Osherson (eds.), Thinking: An invitation to cognitive science, vol. 3., 3-33. Cambridge, MA: MIT Press. Pyle, E. J. (2009). The evaluation of field course experiences: A framework for development, improvement, and reporting. In S. J. Whitmeyer, D. Mogk, & E. J. Pyle (Eds.) Field Geology Education: Historical Perspectives and Modern Approaches. GSA Special Paper 461. Boulder, CO: Geological Society of America.

Recently Viewed Presentations

  • Cam 1: Cyflwyniad i Gronfa Ddata Diffiniad: Mae

    Cam 1: Cyflwyniad i Gronfa Ddata Diffiniad: Mae

    www.rightmove.co.uk. Tasg 1. 1. Dewiswcheichlleolidad. 3. Chwiliwch o fewn 5 milltir. 2. Dewiswch y math o adeilad. 4. Teipiwchamrediadeichpris. 5 ...
  • Chapter 4 Learning Objectives - University of San Diego

    Chapter 4 Learning Objectives - University of San Diego

    Other countries with similar factor endowment find it difficult to produce. Specialization, economies of scale and demand conditions Role for subsidies or selective government intervention 3. Porter's Diamond Michael Porter of Harvard Business School published a book in 1990: Competitive...
  • Centre de Recherches en Physique des Plasmas EPFL,

    Centre de Recherches en Physique des Plasmas EPFL,

    U. Warwick, UK. IPP. Greifswald, Germany. NIFS, Japan. LLNL, US. F-Z . J. ... New digital real-time network to control coils and EC systems potential to use massively multichannel diagnostics. TCV parameters and capabilities. Scientific mission and guidelines.
  • Architecture of Distributed Systems

    Architecture of Distributed Systems

    Exercise. Consider the models on the following slides and answer the following questions. Each model is provided with a hyperlink to acknowledge its source and for additional information.
  • Weds. 2 /22 English I-A SLIDES ARE POSTED

    Weds. 2 /22 English I-A SLIDES ARE POSTED

    Examples would include scientists, technicians, scholars… or ranchers. For example, to find out technical information about a plant, ask a biologist. To find out about a law, ask a lawyer. To find out what breed of cow winters over best...
  • Click to change title - The Standard

    Click to change title - The Standard

    Each company is solely responsible for its own financial condition. Standard Insurance Company is licensed to solicit insurance business in all states except New York. The Standard Life Insurance Company of New York is licensed to solicit insurance business in...
  • Přednáška 11 - Katedra informatiky FEI VŠB-TUO

    Přednáška 11 - Katedra informatiky FEI VŠB-TUO

    (potential infinity) But at any point we will have acquired only a finite amount of experience and have taken only a finite number of steps. Thus, for a Kantian like Hilbert, the only legitimate infinity is a potential infinity, rather...
  • New Ecologies Between Rural Life and Visual Culture in the ...

    New Ecologies Between Rural Life and Visual Culture in the ...

    The former post office at Kilnaboy, Co Clare.September 2007. Lucy R. Lippard argues that if artists are participants as well as directors in a collective or collaborative process of defining a sense of the 'local', new imaginative spaces can open...