Assignment 5 Survey Pilot

Fountas & Pinnell Leveled Literacy Intervention Teacher Instruction Survey

Pilot Survey

            My pilot survey was written as part of my final assignment and program evaluation.  My program evaluation plan will be evaluating the reading gains of the Fountas & Pinnell Leveled Literacy Intervention program or LLI.  LLI is an intervention program which has been in use since 2009 and is designed for children who struggle with reading and writing, causing them to fall behind grade level.  This is an intensive intervention program lasting between 18-24 weeks.  Fountas and Pinnell use a scientifically based system of instruction and materials designed specifically for this instruction.  The success of this program is directly linked to its instructional guidelines.

            Therefore, as part of this evaluation I need to assess how closely the Leveled Literacy Intervention teachers are following the instructional guidelines as well as the ease to which teachers are able to use the program.  This survey, therefore, asks teachers for information in six areas:  About You, Supplies, Supports, Instruction, Student Selection, and Final Words.  Follow the link to see the pilot survey.  

 

http://fluidsurveys.com/surveys/pattie-WkV/leveled-literacy-intervention/

 

          After piloting the survey with teachers who are LLI instructors I found that many of my questions were valid.  However, there were still many changes necessary to be made.  Most changes were based on wording.  They were confused by my use of certain phrases such as ‘this past year’.  I was attempting to have them reflect on an entire school year but because they were currently in the middle of a year they were unsure.  By changing the phrasing to be ‘this school year’ the questions were made clearer. 

            Most of the changes made regarding the content of questions were around the use of supplies.  Even though I had read the program guide I misunderstood the use of some of the materials.   When I was asking questions about sharing materials they pointed out that while they had problems at first they were rectified.  This led to changes to two quantitative questions so that more information was acquired.  They also explained to me the importance of the resource CD.  I was unaware of this and that sharing it was very difficult.  This meant additional questions regarding this resource were required. 

            In the area of Supports, again I misunderstood the materials.  I had to combine two questions because I thought I was asking about two different DVDs but they pointed out that this was actually only one DVD.  They also felt compelled to explain why they did not share certain elements with the classroom teacher.  This caused me to add a question asking the teachers to explain their decision not to share this item. 

            In the section on Time, they wanted more clarity about group transitions and instructional time.  This led to a complete rewording of question two.  They also explained to me that entire segments of a lesson could be left out.  Again, I had to

add another question as I thought segments might be cut short but never realized they might be omitted all together.  It was brought to my attention that time was an even bigger issue than I had initially thought.  They felt that when I was asking about scheduling challenges I should ask if the teachers were able to solve them and if so how.  I am a little concerned that this question is now a leading question.  I also had to completely change my variable for question 8, as their answer wasn’t present.

            Finally, it became clear to me that this was not a typical teaching situation.  I asked them how difficult it was for a substitute teacher to step in.  At this, they became very animated and said I had to ask that question because this was a big problem for them and one that needed to be addressed.   We spent an hour going through the survey question by question.  Based on the amount of conversation taking place it would be my estimation that this survey would take about 30 minutes to complete. 

            I was very surprised by the number of changes made to the survey.  I honestly thought there wouldn’t have been so many.  However, I was in agreement with all of the things they saw in need of change.  Without question the revised survey will be easier to answer and give more in-depth and accurate information.  My only regret with the final survey was the difficulty of creating logic paths through branching and looping.  Unfortunately, this became an IT challenge that couldn't be rectified.  Participants can still complete the survey but they will simply have to follow written directions rather than being automatically redirected.  Rather 'old school' but it still works.  By following the link you can see the final survey.

http://fluidsurveys.com/surveys/pattie-WkV/copy-leveled-literacy-intervention/