As I stated in my last post I am currently taking an Advanced Video Design for Learning class.  My first assignment is to find an educational program and critique it in terms of its goals, objectives and production techniques.  In essence my assignment is to look at the video through a different lens.  To not just watch the program and consume the information within but see how the video was created.  To do this assignment I needed to start with a video format I was very familiar with (dramatic movies) before I could move into an educational program.  For me this first step allowed me to focus on a familiar genre as I applied my new learning.  Now as I move into my own self created step 2 I will dig into an educational program.

The program I chose to critique is A Visit to A Data Driven School by the ASCD.  For me this video represents a very typical and widely used educational program format.

Critique

Goals of this video:

• Demonstrate the use of different strategies to assess the use of instructional strategies and teaching techniques to improve instruction
• Demonstrate one way to create a more detailed picture of student achievement and school system effectiveness

Production Techniques used:

• The Camera-
  • Pans were frequently used to allow the viewer to move between the teacher and the students.  This allowed us to see the interaction and reactions between the instruction and the learner.
  • Zoom Ins and Zoom Outs were used extensively throughout this video.  At times I felt that they were over used and distracting.
  • Movement- at one point in the video the camera is moved around the teacher while at the same point maintaining focus on her.
• Audio- Voice over was used extensively in the transitions between scenes.  As well voice overs were used to provide additional information as to what was occurring from the perspective of various participants.  This video also uses a narrator to set the scene and provide background information deemed necessary for the viewer.
• Shots
  • Classroom- These scenes are designed to show what it looks like in a classroom environment.  They show the lesson and the interaction between the students and the teacher.  In the classroom scenes the shots varied form Close Ups (CU), to Long Shots(LS) of the classroom, to Point of View Shots (POV) of the teacher and the students, to Extreme Close Ups (XCU)  of the assessments the students were completing.
  • Teacher Interview- These scenes provided the viewer with the teacher’s perspective and the “why” of what she was doing during the lesson.   The setting for these interviews was in the classroom which allows the viewer to connect more easily the “what” of the classroom scenes, with the “why” of the interview.  Keeping the setting consistent in both was a good way to support the viewer in making this connection.  These Shots were all Close Ups (CU).
  • Expert Interview- Executive Director of Curriculum- The purpose of these scenes is to establish the purpose of these strategies and techniques from an expert perspective.  She provides the research based information about the strategies being demonstrated and generalize them for all teachers.  The setting chosen for this interview was in a more formal looking office.  The background was poorly lit and dark which draws one into focusing more on the expert. Again the shots in this interview were all Close Ups (CU) which allows the viewer to really hone in on what she is saying by drawing our focus to her face.
  • Superintendent Interview- Again the location for this was a more darkly lit  environment and the shots were all Close Ups (CU).  The superintendents purpose was to establish how and why these techniques are important to the larger district.
  • Teacher entering data into the computer- These shots were consisted of Medium Shots (MS), Over the Shoulder Shots,  and Close Ups (CU).  They were used to demonstrate the next step in the data collection which connects the classroom to the district.  The setting for these scenes was in what appears to be a different location from the classroom.  In this way it signifies that this work is done outside of the classroom student contact times.
• Titles and Subtitles-  This video uses a title at the beginning to establish the topic.  As well, subtitles are used to identify the various interview participants and establish their expert status by providing their job titles.
• Editing- This video is heavily edited. We are constantly switching from shots of the students, to the teacher, to the whole classroom, to the interviews, to the filling out of forms, and back again   At times I think the editing is excessive and not necessary.    The constant motion takes away from my ability to see and reflect on what is happening in the moment.

Through this critiquing process I am beginning to see the connection that exists between the goals of a video and the production techniques used.  The productions techniques need to be chosen carefully so that they enhance, and support the goals.  If used properly they can add to the learning experience, evoke emotion, and support the message.  If used improperly they are distracting and detract from the message.

Not sure I will ever look at a video the same way again.  :)

This frustration lead me to the realization that I needed to connect first with what I knew and was most familiar with. Despite the fact that I am a teacher I do not spend the majority of my time watching education videos. Rather I spend most of my viewing time watching and enjoying movies. So what if I started with what I was familiar with? What if I tried this assignment first with what I know?

This lead me to the movie “Dead Poets’ Society” and in particular the Carpe Diem scene. It seemed fitting for me as an educator but more importantly it was a scene that I have always connected with…Seize the Day.

So what are the goals of this clip from the movie? I believe the goals are to:

• Contrast the difference between the traditional education the students are used to and the inquiry based journey they are about to embark on.
• Introduce the audience to an important theme in the movie – Sieze the Day
• Connect the audience emotionally to that theme – Carpe Diem.

As I watched this clip unfold I took note of several production techniques used to reach those goals effectively:

• The use of three locations highlighted the difference between the traditional classroom and the new learning environment they were beginning to encounter. They physically move from their desks to the hall. This is a both a metaphor and a foreshadow of them moving from traditional students to engaged learners.
• Camera perspectives -During the scenes in the hall various shots were used very effectively:
  • At the beginning of the hall scene an Establishing Shot is used to give the audience a basic orientation to the geography of the scene. This shot occurs from above and looks down on the students as they take their positions in the hall.
  • subjective shots were used to show us what the characters (both the students and the people in the photographs) would see. We see the photographs from the point of view of the students, but we also see the students from the point of view of the photographs. At one point, the director even includes a shot from behind the glass in the display case. In this way he personifies the people in the photographs. They come alive. They are looking at the boys and pleading with them to Seize the Day. This ability of the camera shots to create a sense of life in the photos allows the audience to believe that they are actually whispering Carpe Diem to the students.
  • Pans and Zoom Ins were used to allow the audience to see the faces of the men in the photographs as we can hear the voice of the teacher whispering seize the day…make your lives extraordinary. These camera treatments create movement which add to the feeling that the photos are talking to the boys and imploring them to seize the day.
  • Shots showing the glass are shown at the beginning of the scene. As the audience we look into and out of the display case. As the scene progresses the glass is removed from the shots. For me this signifies the removal of this barrier and the closer connection between the students standing in the hall and those posing in the photographs.
  • Medium Shots (MS) are used extensively throughout the scene in the hall at the beginning. We see the students and the teacher from the waist up. As the scene progresses we move to more Close Ups (CU) which allows the audience to really see the emotion in the students faces and the impact this lesson is having on them.
• Editing- This clip shows three distinct scenes; The traditional classroom, the hall for the Carpe Diem scene and the hall which brings us back the traditional school setting. The transitions between these three scenes are very abrupt and quick. There is no fade out, the audience is just taken from one to the other. I believe this adds to the audiences ability to contrast the abrupt differences between the traditional education of the first and last scene, and the engaged learning in scene 2.
  • The Carpe Diem scene was edited in such a way that we go back and forth between the faces of the boys and the faces in the photographs. We get to see the emotion this lesson is creating in the boys as we look at their faces. We are also allowed to create that emotion within ourselves by seeing the faces of the photographs. In this way, the editing is highly effective in creating a feeling and emotion in the audience.
• Audio- the audio in the hall scene is very quiet, which is in contrast to the classroom scene. There is little background sound and no music. All we hear are the words of the teacher as he engages the students.

I realize that this clip is not an educational program per se but I needed this first step to make sense for myself of this new learning in video design. I found that I was much more successful by starting with what I know and am most familiar with. The next step for me is to now see if I can take my emerging understanding and apply it to an educational program.

To be continued…

http://us.123rf.com/400wm/400/400/Kmitu/Kmitu0708/Kmitu070800013/1374218-globe-with-water-reflection-against-black-background.jpg

One of the greatest and most unexpected gifts of my graduate studies has been that it has pushed me to reflect deeply on what I know, believe, and think.  I have always considered myself to be a reflective person but this has pushed me further.  Graduate studies has provided me with new constructs to think about, deeper understandings of theories, and learning that has challenged everything I thought I knew.  It has been and continues to be, a fantastic opportunity not only to learn more about education but to learn more about myself.

I equate this multifaceted opportunity to a journey.  I have embarked on a voyage across the landscape of academia with a destination of deeper understanding of my craft on the horizon.  As I navigate across the landscape there have been several places where I have stopped, investigated, and toured around.  Some of these stops have been brief; I took in the sights, snapped a few photos and carried on with my journey.  Other places have felt so comfortable, so right, so much like home that I find myself constantly going back to them only to discover new landmarks to visit.  These places have become a part of who I am as a learner, an educator, and a person that they now make up a part of who I am.

I am a Constructivist!

Constructivist belief is one such place on my journey.  I find myself constantly drawn to constructivist ideology and thinking.  It feels so right to me.  I believe in constructing knowledge for myself and providing the opportunities to support my learners in constructing their own knowledge.  I believe that there is not one “right” way to do anything.  Instead there are multiple paths that can take one to the destination.   I have always felt tension when knowledge has been presented to me as an absolute; with no exceptions, room for questioning and in a “one right way” fashion.

My constructivist epistemology has led me to several beliefs which I hold dear:

• Learners need to be active participants in learning.
• Learning and “truth” are unique to the individual.  Everyone constructs their own knowledge based on their schema, culture, and experiences.  I believe that knowledge is value dependent, subjective and highly contextualize.
• Learning is socially constructed.  I learn best with others.  Talking, sharing, questioning, and analyzing with others helps me to create meaning for myself.

I have a Cultural Bias!

A few of years ago, I had an opportunity to discover the influence that my western culture has had on my ways of knowing, being, and doing.   I was standing with a friend looking at a photograph on the wall.  My friend, a consultant with our First Nations, Inuit and Metis Unit, and herself a Metis woman, asked me what I saw in the picture.  I paused, took a careful look and began to tell her what I saw.  My description was a list of items from the picture; rock, tree, man, car….on and on I went until she asked me to stop.  Then she began to share with me what she saw.  She saw the elements in the photo…earth, air, fire, water.  She connected the image with the wisdom the Elders had shared with her in regards to car.  She saw more than a steel mode of transportation.  She saw the metal as a part of Mother Earth, born from the rocks and minerals she created and nurtured.  She saw the asphalt on the street and wondered about the earth that lay underneath, trapped under tar.  In that single moment I realized that we did not see things the same way and that my truth was not hers.  She saw connections, she saw the whole, she saw much more than my laundry list of discrete items.  In that moment my mind was opened to a new way of seeing, knowing, and being.

I now believe that knowledge is not only socially constructed but it is highly dependent on cultural ways of knowing, being, and doing.  In First Nations and Metis cultures everything is interconnected, humans hold no more importance than worms in the ground and we are all connected to Mother Earth. Observations of relationships are a significant component of this epistemology.  (Kovach, 2009) In many ways my constructivist epistemology fits so well here.  I too see relationships but how I process and make sense of what I see continues to be different.

Another ideology I connect strongly to within First Nations and Metis culture is the ideas of ownership.  I believe that knowledge should be freely shared with others.  This spirit of generosity connects us with our community, allows us to contribute, and honors the generosity of others who we learn from.   We do not own knowledge…we are a part of it, just like we do not own the earth, or the animals in it.  We are all interconnected.

I am an Action Researcher!

I connect strongly with the ideas of Action Research.  I like to be an active participant in the work.   For me, the value in educational research lies in the day-to-day, minute to minute impact it can have on student learning.  In those moments I connect strongest with the purpose of the research.   I enjoy taking theory and putting it into practice, taking big ideas and making them work in the classroom.  The classroom continues to be the place where I feel most at home.

The Journey Continues!

As I continue to navigate my learning journey I know there will continue to be places I encounter that will encourage me to stop, think, connect and reflect.  Each place I visit shapes and changes me as a person and as a learner.  This personal impact has been the most amazing and unexpected surprise in this adventure.  The journey has not been about learning more educational “stuff”.  It has and continues to be about discovering myself.

In the primary years students should be given regular and consistent opportunities to subitize in order to build their skills, improve number sense and lay the foundation for future mathematical learning.  In kindergarten numbers to 5 should be focused on for instant recognition.  Once students are familiar with familiar representations of 1 to 5, larger collections can be used to encourage students part-part-whole thinking.   For example, on the card below students may instantly recognize a three and a four and then add the numbers together to know that there is a collection of seven dots on the card.

As the collections get larger students can be encouraged to use their estimation skills to think about “how many” and “how do you know”.  Our Saskatchewan Curriculum refers to this fundamental skill through several outcomes from Kindergarten to Grade 2:

• Kindergarten- NK.2 Recognize, at a glance, and name familiar arrangements of 1 to 5 objects, dots, or pictures.
• Grade 1- N1.2 Recognize, at a glance, and name familiar arrangements of 1 to 10 objects, dots, and pictures
• Grade 2- N2.1 Demonstrate understanding of whole numbers to 100 (concretely, pictorially, physically, orally, in writing, and symbolically) by: 
  • representing (including place value)
  • describing skip counting
  • differentiating between odd and even numbers
  • estimating with referents
  • comparing two numbers
  • ordering three or more numbers.

This video is an excellent example of a kindergarten teacher who is using Quick Images to build on her students subitization skills, and create opportunities for mathematical conversation.

Information about Subitizing

Subitizing-What is it? Why Teach it?

Pinterest Board on Subitizing

Resources to support teaching Subitizing

Dot Cards and Ten Frames

Sparklebox Dot Cards

My learning journey in EDCUR 809 has taken me to assignment 5…creating a survey. I’ll be honest my first impressions of this assignment were Easy, How hard can a survey be? Create a few questions, get back some data, make a nice Excel bar graph and Voilà ..survey and analysis done.
Well I was wrong. Really wrong.

I reflected to a colleague that this truly was an “end of ignorance” experience. Now that I know more it was not easy. Every question caused me to think, wonder, and reflect on the question “Will this provide us with the information we need?”

I began by deciding to create a survey for the PreKindergarten to Grade 5 Mathematics Learning Community. I wanted to create a tool to assess the degree to which the math learning community members have reach the medium term (year 2)outcomes and provide recommendations for next year. I wanted a survey that would shed light on where we are and where we are going as a professional learning community. My first step was to revisit the outcomes through the lens of assessment. The outcomes for year 2 are:

• increase understanding of mathematics content, pedagogy, and instruction
• increased enjoyment of teaching mathematics
• increased understanding of content development across mathematics curricula (grades prior or succeeding current teaching assignment)

To strengthen the survey assessment and analysis I also wanted to consider Guskey’s (2000) five levels of professional development. These levels are used by my system’s Staff Development Team to evaluate the effectiveness of our professional development. The levels are as follows:

1. Participants’ Reaction
2. Participants’ Learning
3. Organization support and change (are barriers being reduced)
4. Participants’ use of the new knowledge and skills
5. Student learning outcomes

Specifically, I felt that my survey would focus on levels 1, 2, and 4. (However, based on feedback I revised my survey to provide an opportunity to gain insight into number 5 as well)

With both my outcomes and the five levels of professional development in mind I created the initial survey. From there I shared it with colleagues. They provided me with feedback that lead me to revisions. Next, I piloted the survey with 4 members of the Mathematics Learning Community to get their feedback. This lead to more revisions, more pondering about what data this survey would provide and questions about if the data would be adequate to shed light on the my inquiry into effectiveness and actualization of outcomes.

Initial Version of the Survey

Initial Math Learning Community Survey

Revised Version of the Survey

Revised Version of the Math Learning Community Survey

(please note that this survey will be put into an online tool. The open question text boxes provided in the document are place holders and do not reflect the amount of space participants will be given to complete their responses.)

This survey will be put into an electronic form to be sent out to all community members around the middle of May. From there the plan is to bring together a small group of members to help with the analysis of the survey, provide additional insight into the actualization of outcomes and make recommendations for next year. I hope that the survey and the follow-up analysis will provide the information we need to move forward effectively.

Image: Objects In The Rearview Mirror by KrissyVenosdale- http://www.flickr.com/photos/venosdale/6876773480

Assignment 4 in my EDCUR 809 class has me learning how to create a logic model.  The Mathematics Professional Development Logic Model is document I created to help me to see the scope, sequence, and interdependence of mathematics professional development in my school system.  The model outlines the situation of mathematics education, the inputs into the system, the outputs and impact of the actions taken.  My actual program evaluation will only focus on one aspect of this large picture of Mathematics Learning and Support in our system- The PreK- Grade 5 Mathematics Learning Community.  My plan is to focus on the Learning Community and the corresponding teacher and student outcomes.

The Goal of the Mathematics Learning Community

In order to increase student engagement and achievement in mathematics, we will work to develop communities of professional learners.  Our learners will own their learning, openly reflecting on student learning, and implement powerful mathematical instructional practices that meet the needs of all their students.

Engaging in an Evaluation Assessment requires me to focus and reflect on 5 important steps and how they would be connected to the Math Learning Community.

1.  Engaging Stakeholders- There are many important stakeholders who would need to be engaged in this evaluation in order to provide a through evaluation. 

• Teachers- Teachers are important stakeholders in the program.  Their input and voice into the effectiveness of the professional development will be invaluable.  Qualitative and quantitative data can be collected though their engagement in the evaluation.  Teachers will also play a role in the creation of assessment and evaluation tools.
• Consultant Team- The consultant team will be involved in the co- creation of assessment and evaluation tools to gain insight into teacher engagement and changes to practice.  They will also play a role in the creation of assessment and evaluation tools to assess student learning and engagement.
• Students- Students are the most important stakeholders in this program.  Teacher learning in the communities is in service to the learning of students.  Data around the effectiveness of teacher learning will be gathered through assessment tools which measure student engagement and achievement in mathematics.

2.  Focus the Evaluation- This evaluation would focus on assessing the degree of actualization of the Math Learning Community outcomes for teachers and students.  Through this assessment and evaluation decisions can be made around whether or not to continue funding the program (senior administration and trustees) and to make adjustments to the program to make it more effective (consultant team).

Teacher Outcomes

Short Term

• feeling supported in learning
• sense of belonging in a community of professional learners
• reflecting on their practice and student learning
• increased understanding of grade specific curriculum

Medium Term

• increase understanding of mathematics content, pedagogy, and instruction
• increased enjoyment of teaching mathematics
• increased understanding of content development across mathematics development

Long Term

• strong professional judgement in mathematics
• confidence in delivering differentiated instruction informed by assessment and an understanding of essential understanding
• leading and supporting mathematics with colleagues
• feeling a part of a community that is self-led and self-sustained

Student Outcomes

Medium Term

• increased enjoyment of mathematics learning
• increasing understanding of mathematics and an ability to communicate

Long Term

• confidence and competence in mathematics
• engaged in mathematics learning
• increased achievement in mathematics

This program evaluation could be completed at the end of the 2012-2013 school year to evaluate the effectiveness of the program to date in terms of actualization of short-term and medium term outcomes.  This evaluation would be critical in informing planning for the 2013-2014 school year and lay the foundation for future evaluations of long-term outcomes at the end of the 2013-2014 year and beyond.

Evaluation Program Design

This evaluation will follow the CIPP Model, as it allows for the partitioning of various components of the program and create a long-term evaluation plan.

3. Collect the Information- This evaluation would focus on the collection of data from reflections, observations, and products.  It will include but not be limited to:

•  Surveys with teachers and students around their feelings about mathematics instruction and learning (existing surveys and one completed at the end of the 2012-2013 school year)
• Photos/ Videos from workshops and classrooms (existing and ongoing)
• Testimonials (written or from interviews) from teachers and students around their learning in mathematics (existing and to be collected at the end of the 2012-2013 school year)
• Analysis of student created products and conversations in mathematics (existing and to be collected at the end of the 2012-2013 school year to inform next steps)

4. Analyze and Interpret- The data will allow us to answer the overarching questions of  Is this program effective for increasing student learning and achievement in mathematics?  and Will teacher directed, professional learning communities create teacher confidence and knowledge in teaching mathematics?  If this evaluation were to actually take place it would be important to ensure that the analysis of the data take place by an individual who is not a part of the consultant planning team in order to ensure that evaluator bias who not affect the analysis and evaluation.  The data evaluation could be shared with Coordinators, Superintendents, teacher participants and the board of trustees.

5.Use the Information  – This evaluation will be used to inform decision-making around professional development in mathematics, and this model of a professional development learning community.

First of all, there IS reading in mathematics…reading of textbooks, word problems, literature, the whiteboard…the ability to read supports the student’s ability to take in information, comprehend problems and creating meaning.

Secondly, reading at it’s very essence is thinking.  It is the interpretation of a set of symbols (letters and words), and using our understanding of the symbols to create meaning.  This process must involve thinking.  Likewise, mathematics is the interpretation of a set of symbols (numbers, objects, representations, letters, and words) to create meaning, and gain understanding.   This process must also involve thinking.   Since both subjects are looking to strengthen thinking it only makes sense that we use the strategies and supports for strengthen student thinking, and comprehension in reading to strengthen thinking and understanding in math.  Creating consistency between the strategies can foster students ability to make connections and allows them to build on an existing foundation within a new context.

In reading we use the Super 7 Reading Strategies to support thinking and comprehension.  In mathematics these same strategies can be built upon to support mathematical thinking comprehension.

So next time you are explicitly teaching comprehension strategies to your students in reading consider the possibility of expanding on those strategies in mathematics.  As Maggie Siena  (2009) so eloquently puts it “we can become more effective teachers of mathematics by drawing from our successful experiences with teaching literacy.  It’s the art of lighting two candles with one flame” (p.2).

cc.- http://www.flickr.com/photos/steveritchie/6067642964/

Siena, M. (2009). From reading to math. Sausalito, CA: Math Solutions.

Adapted from Stufflebeam’s CIPP Evaluation Model (1983)

Once again I would suggest that the CIPP Model by Stufflebeam would be an appropriate model for conducting an evaluation on this program.  My recommendation is based partially on the belief that this model would be a good fit and partially because it is one the model we have studied that makes the most sense to me…just being honest.  Stufflebeam’s model is straightforward and focuses evaluators on four core concepts- Context (Goals), Input (Plans), Process (Actions), and Product (Outcomes).  By applying these four concepts to the Aboriginal Prenatal Exercise Program I believe program designers would be able to gain a valuable insight into effectiveness of the program to date and necessary information into how to improve.  In this way the evaluation would be formative in nature.

I believe this model will work in this case for several reasons:

• Evaluation within each concept will provide information into the effectiveness/ quality of the program goals, plans, actions and outcomes.  This would provide insight into what works and where potential changes need to be made.  
• Data from each component could be gathered and evaluated.
  • Goals- Program Goals around decreasing Gestational Diabetes among Aboriginal women could be evaluated in comparison to research and similar studies across North America.   Questions around Is the research sound?  Are the connections made valid? Is this inquiry worthy of the time/ resources needed to actualize? could be investigated.
  • Plans- In evaluating the plans laid out by the program evaluators could look for coherence between the objectives of the program and each of the programs components.   The program appears to components around exercise, nutrition, pregnancy, education, and creation of a supportive community seem to be in place.
  • Actions-   Many actions were taken in this program to actualize the plans put in place.  Data from each action taken gathered.  In this component I would look at collecting data from the numerous stakeholders in the program.  In particular data from the program participants would be invaluable.  Questions around Did this program meet your needs?  What did you learn from the program?  What parts of the program were most valuable to you?  What changes would you make? What factors led you to attend or not attend? could be investigated through interviews, surveys, and/or  focus groups.
  • Outcomes- This part of the evaluation would led one to look into the medical data of participants, their children, and non participants as it exists now and into the future.  By looking into this data one could see if the goal of reducing the incidence of Diabetes during pregnancy  into the future and among the offspring of the women is being met.  By studying the health stats of non participants as well, evaluators can see how the results of non participants compare to those of participants.  Much of the data needed to make a summative evaluation would not be available this early into the program however current health data could be used for formative evaluation.
• Each component would allow for multiple sources of data to be collected and analyzed   In this way the evaluation could be very thorough and comprehensive.  Input from the many stakeholders, and their perspectives should be used to inform this evaluation.  It would be very important to consult with not only the medical personal involved in this program but also the community members whom this program effects.  In particular attention should be given to gather and honor the perspectives of Aboriginal Elders and Knowledge Keepers within each component of the model.

I am sure that if I was to actually conduct this evaluation more questions would emerge and the plan for evaluation would evolve to become more complete and comprehensive.  This assignment has created an opportunity for me to continue my learning with Program Evaluation.  One thing I have learned is that the knowledge of the content is not a prerequisite for beginning an evaluation.  I know very little about diabetes yet by applying the limited information I was given to the CIPP model I was able to “see” a plan for evaluation.  This makes me wonder if a richer knowledge to the content would have led me to a different evaluation plan.

The evaluation I chose to dig into was completed for the Literacy and Numeracy Secretariat of Ontario titled The Impact of the Literacy and Numeracy Secretariat: Changes in Ontario’s Education System.  The mandate of the Literacy and Numeracy Secretariat is to actualize the provincial goal to  improve students reading, writing, and mathematics skills through the building of instructional and leadership capacity of at all levels.  This evaluation focused on reviewing of the actions taken by the Secretariat and determining if the education system, the educators and the students have benefited from those actions.

The model of evaluation used appears to be the CIPP model by Stufflebeam.  In this model evaluators look into the following areas- Context (goals) , Input (plan), Process (Actions) and Product (Outcomes).   I believe that the evaluation team focused on the input, process and products extensively.  Specifically the objectives of this review in comparison to the CIPP model were to:

1. Identify and evaluate the LNS initiatives input and process
2. Determine whether and in what ways Ontario’s education system has changed as a result of these initiatives product/ outcomes
3. Determine the extent to which these changes have benefited students and educator -  product/outcomes
4. Draw lessons from these findings, as a guide to the ongoing improvement of Ontario’s educational system – summative outcomes to inform decision-making moving forward. (p. 10)

In order to accomplish these objectives the review team took  on the momentous task of reviewing literature, meeting with focus groups, conducting interviews and surveys, looking at student data, meeting with sample schools and numerous share holders in the education system. At the end of this extensive process the evaluation team concluded that Ontario’s Literacy and Numeracy Secretariat has had a major, and primarily highly positive, impact on Ontario’s education system.  

There are many strengths with this evaluation.  First of all it is very though.  The input of numerous stakeholders (students, parents, teachers, administrators, trustees and the community) were collected, analysed and used to make recommendations. The sources of data are varied and include everything from conversations with focus groups, to student data, to surveys with teachers, and administrators, to the careful studying of products created by the Secretariat.  Secondly, the evaluation was based on sound research in the areas of literacy, and numeracy as well as student learning. Lastly, the evaluations was completed by an evaluation team that was led by five researchers from the Canadian Language and Literacy Research Network based out of four Ontario Universities.  I believe that a major strength of this evaluation is the diverse expertise that the evaluation team brought to the table.

One weakness I see in the report is the seemingly contradictory findings.  The evaluation team recommends accelerating and intensifying the efforts to improve instructional and assessment capacity in numeracy, and continuing the initiatives currently in place.  However,   they also indicated that teachers are feeling overwhelmed with the pace of the information coming out and need more time to internalize the learning and apply it in the classroom.  No indication is given as to how to do both.  As well, the recommendation of increased emphasis on numeracy is not supported with any recommendations as to how to do this.