ETAD, Learning, Math, Professional Development

Where We Are and Where To Go Next: Creating a Survey

My learning journey in EDCUR 809 has taken me to assignment 5…creating a survey. I’ll be honest my first impressions of this assignment were Easy, How hard can a survey be? Create a few questions, get back some data, make a nice Excel bar graph and Voilà ..survey and analysis done.
Well I was wrong. Really wrong.

I reflected to a colleague that this truly was an “end of ignorance” experience. Now that I know more it was not easy. Every question caused me to think, wonder, and reflect on the question “Will this provide us with the information we need?”

I began by deciding to create a survey for the PreKindergarten to Grade 5 Mathematics Learning Community. I wanted to create a tool to assess the degree to which the math learning community members have reach the medium term (year 2)outcomes and provide recommendations for next year. I wanted a survey that would shed light on where we are and where we are going as a professional learning community. My first step was to revisit the outcomes through the lens of assessment. The outcomes for year 2 are:

  • increase understanding of mathematics content, pedagogy, and instruction
  • increased enjoyment of teaching mathematics
  • increased understanding of content development across mathematics curricula (grades prior or succeeding current teaching assignment)

To strengthen the survey assessment and analysis I also wanted to consider Guskey’s (2000) five levels of professional development. These levels are used by my system’s Staff Development Team to evaluate the effectiveness of our professional development. The levels are as follows:

  1. Participants’ Reaction
  2. Participants’ Learning
  3. Organization support and change (are barriers being reduced)
  4. Participants’ use of the new knowledge and skills
  5. Student learning outcomes

Specifically, I felt that my survey would focus on levels 1, 2, and 4. (However, based on feedback I revised my survey to provide an opportunity to gain insight into number 5 as well)

With both my outcomes and the five levels of professional development in mind I created the initial survey. From there I shared it with colleagues. They provided me with feedback that lead me to revisions. Next, I piloted the survey with 4 members of the Mathematics Learning Community to get their feedback. This lead to more revisions, more pondering about what data this survey would provide and questions about if the data would be adequate to shed light on the my inquiry into effectiveness and actualization of outcomes.

Initial Version of the Survey

Initial Math Learning Community Survey

Revised Version of the Survey

Revised Version of the Math Learning Community Survey

(please note that this survey will be put into an online tool. The open question text boxes provided in the document are place holders and do not reflect the amount of space participants will be given to complete their responses.)

This survey will be put into an electronic form to be sent out to all community members around the middle of May. From there the plan is to bring together a small group of members to help with the analysis of the survey, provide additional insight into the actualization of outcomes and make recommendations for next year. I hope that the survey and the follow-up analysis will provide the information we need to move forward effectively.

Image: Objects In The Rearview Mirror by KrissyVenosdale- http://www.flickr.com/photos/venosdale/6876773480

ETAD, Math, Professional Development

Program Evaluation- PreK-Grade 5 Math Learning Community

My continuing journey in EDCUR 809 has led me to assignment 4- Performing an Evaluation Assessment.  This assessment is a necessary step in Program Evaluation as it is used to determine the feasibility and direction of any future evaluation.  In EDCUR 809 I have an opportunity to lay out a possible evaluation but it is not necessary to conduct the actual evaluation.  My learning is about the how of setting up an evaluation not the actual evaluation itself.  This assignment asks me to choose a program to lay out a plan for evaluation and  I have chosen a program that is very close to my heart…the PreK- Grade 5 Mathematics Learning Community.  This community has just begun its second year of existence and consists of teachers from PreKindergarten to Grade 5.

The Goal of the Mathematics Learning Community

In order to increase student engagement and achievement in mathematics, we will work to develop communities of professional learners.  Our learners will own their learning, openly reflecting on student learning, and implement powerful mathematical instructional practices that meet the needs of all their students.

Engaging in an Evaluation Assessment requires me to focus and reflect on 5 important steps and how they would be connected to the Math Learning Community.

1.  Engaging Stakeholders– There are many important stakeholders who would need to be engaged in this evaluation in order to provide a through evaluation

    • Teachers– Teachers are important stakeholders in the program.  Their input and voice into the effectiveness of the professional development will be invaluable.  Qualitative and quantitative data can be collected though their engagement in the evaluation.  Teachers will also play a role in the creation of assessment and evaluation tools.
    • Consultant Team– The consultant team will be involved in the co- creation of assessment and evaluation tools to gain insight into teacher engagement and changes to practice.  They will also play a role in the creation of assessment and evaluation tools to assess student learning and engagement.
    • Students– Students are the most important stakeholders in this program.  Teacher learning in the communities is in service to the learning of students.  Data around the effectiveness of teacher learning will be gathered through assessment tools which measure student engagement and achievement in mathematics.

2.  Focus the Evaluation– This evaluation would focus on assessing the degree of actualization of the Math Learning Community outcomes for teachers and students.  Through this assessment and evaluation decisions can be made around whether or not to continue funding the program (senior administration and trustees) and to make adjustments to the program to make it more effective (consultant team).

Teacher Outcomes

Short Term

  • feeling supported in learning
  • sense of belonging in a community of professional learnersMath Community (2)
  • reflecting on their practice and student learning
  • increased understanding of grade specific curriculum

Medium Term

  • increase understanding of mathematics content, pedagogy, and instruction
  • increased enjoyment of teaching mathematics
  • increased understanding of content development across mathematics development

Long Term

  • strong professional judgement in mathematics
  • confidence in delivering differentiated instruction informed by assessment and an understanding of essential understanding
  • leading and supporting mathematics with colleagues
  • feeling a part of a community that is self-led and self-sustained

Student Outcomes

Medium Term

  • increased enjoyment of mathematics learning
  • increasing understanding of mathematics and an ability to communicate

Long Term

  • confidence and competence in mathematics
  • engaged in mathematics learning
  • increased achievement in mathematics

This program evaluation could be completed at the end of the 2012-2013 school year to evaluate the effectiveness of the program to date in terms of actualization of short-term and medium term outcomes.  This evaluation would be critical in informing planning for the 2013-2014 school year and lay the foundation for future evaluations of long-term outcomes at the end of the 2013-2014 year and beyond.

Evaluation Program Design

This evaluation will follow the CIPP Model, as it allows for the partitioning of various components of the program and create a long-term evaluation plan.

3. Collect the Information– This evaluation would focus on the collection of data from reflections, observations, and products.  It will include but not be limited to:

  •  Surveys with teachers and students around their feelings about mathematics instruction and learning (existing surveys and one completed at the end of the 2012-2013 school year)
  • Photos/ Videos from workshops and classrooms (existing and ongoing)
  • Testimonials (written or from interviews) from teachers and students around their learning in mathematics (existing and to be collected at the end of the 2012-2013 school year)
  • Analysis of student created products and conversations in mathematics (existing and to be collected at the end of the 2012-2013 school year to inform next steps)

4. Analyze and Interpret– The data will allow us to answer the overarching questions of  Is this program effective for increasing student learning and achievement in mathematics?  and Will teacher directed, professional learning communities create teacher confidence and knowledge in teaching mathematics?  If this evaluation were to actually take place it would be important to ensure that the analysis of the data take place by an individual who is not a part of the consultant planning team in order to ensure that evaluator bias who not affect the analysis and evaluation.  The data evaluation could be shared with Coordinators, Superintendents, teacher participants and the board of trustees.

5.Use the Information  – This evaluation will be used to inform decision-making around professional development in mathematics, and this model of a professional development learning community.