Well, the team of student researchers have completed their first set of interviews. I have uploaded the video data to a private Google Classroom page (access is invite-only, and requires a private password), and left instructions for the team to try and watch the videos over the half term break. I've also started the ball rolling by writing up my own comments/thoughts about the videos underneath each one. My hope is that the students will watch the videos and then engage with my comments and each other.
The reason I did this was to encourage critical engagement with the research process; I had given each student a notebook to write in, but when I collected these in, only two of the five students had used them. Those that did use the books had only written a few notes on the first page. This was a bit disheartening, as I was hoping to have a second data source from the notebooks. However, I cannot force the students to use them, as I want the experience to be natural for them; I want my input and direction to be minimal, so that they feel a sense of responsibility and ownership.
As I've mentioned, I have already watched the videos once through. It was an interesting experience, because when I piloted the project I conducted the interviews myself. Watching other people conduct interviews that you've worked together to plan is quite an odd experience; I really have felt a certain loss of control. There were a few moments where I wish I'd been able to delve a bit deeper into what's been said. However, the students have been pretty good at using prompt questions to tease out information.
It's interesting to me to see which students are taking a leadership role in the interview process. One male student, who is academically the weakest in the group, is actually engaging really well with the process. He recognised, on more than one occasion, where the responses had been vague, and pushed the students to explain themselves in more detail. The higher ability girls were especially keen to get the interview process started, though I think they were more likely to follow the interview schedule exactly, whereas the boys tended to be able to think 'on the spot' a bit better.
I am eager to start the Year 8 interviews after half term, so that I can see if any of these patterns crop up again (in terms of how the student research team works).
As for what was said in the interviews, the following are points of interest:
- most students recognised that feedback was used to address both strengths and weaknesses
- English and Maths were mentioned as examples of good feedback by all students - Science was suggested being less effective (might be an interesting thing to explore in follow-up interviews)
- negative comments are viewed as unhelpful, but these are rarely given
- a 'bad marker' is a teacher who uses ticks and crosses
- generic feedback is not helpful - it helps when it feels personal
- constructive feedback, which links clearly to a mark scheme, is most helpful
- self-assessment isn't viewed as helpful, because students feel they 'lack a clear perspective' on their own work
- most students recognise that when work is set, it will usually be marked
- however, if students begin to realise that their teacher doesn't regularly mark their work, they lose motivation to do it - this suggests that small, regular bits of work and feedback promote a better work ethic within the students
- most students feel that they can recognise when teachers are marking 'for Ofsted' and when teachers are marking because they genuinely want a student to improve
Obviously more information is needed; I think some follow-up interviews might be in order, to help us clarify some of the points that were made. That, or a few more individuals could be interviewed, to see if any of these ideas appear again.
Friday 27 March 2015
Thursday 19 March 2015
Reflecting on the pilot interview
On Tuesday, three of the student researchers were able to conduct a pilot interview of two Year 9 students. Part of the agreement was that the students would not appear on film, so the students pointed the video-camera at the floor while conducting the interview. They asked the ten questions which we had devised as a group, in roughly the same order that they appeared on the sheet.
Today we met to watch the footage back and discuss what we've learnt from the experience. I also showed them some footage from my own pilot, and we went through each one critically, looking at sound, angle, etc. To start, the students mentioned that the sound quality wasn't very good. You could clearly hear them (or me) asking the questions, but it was sometimes hard to hear the respondents. When I asked them how we could fix this, they suggested that we move the camera closer to the interview subjects. Eventually, we came to the decision that if students don't want to appear on camera we should place the recorder face up on the table. This way, we can clearly hear what they are saying. If students agree to be video-recorded, then we discussed seating the respondents side-by-side and setting the camera clearly in front of them on a tripod. The interviewers themselves don't have to appear on camera, and can be sat to the side of the camera itself.
Another issue that the group brought up was that one of the two interview subjects really dominated the conversation. I am glad this happened in the pilot, as it is a common problem with group interviews. They described using some of the prompting techniques I had taught them, but found that the dominant person still managed to speak for the other student. Again, I asked the students how they could fix this problem. One of the girls suggested knowing the names of the students they were interviewing ahead of time, so that they could personally address questions towards them. We agreed that this was a good technique to use, so I assured students that they would know the names of the students they would be interviewing; I can also use the school behaviour module to show them a picture of the students before-hand, so that they know exactly who is who.
A third issue was that they discovered some of the questions required 'yes/no' responses (which I knew was going to be the case; however, because I wanted the students to devise the questions, I didn't want to influence their choices too much by pointing this out), which didn't always invite much discussion. However, they immediately described using some of the prompt questions to try and dig for more detailed responses.
Similarly, they felt that through the conversation, some of the later questions became repetitive or redundant; because this was their first experience interviewing other people, they stuck to the questions and asked them all, despite this. In our discussion, I told them that they should use their professional judgement in these types of situations; if they think that they've already addressed a question once, they don't need to ask it again. They seemed pleased with this - I think perhaps they were afraid of missing things out if they left questions off, but once I explained that they are essentially in control, that they're the experts once they're in the room and they can use their own discretion, they felt more confident.
Finally, they described how the students seemed to focus on feedback in English lesson. I asked them why they thought this might be. One student suggested that it was because the students knew that I was involved, and that I was an English teacher; their responses may have been focused on English because that's what they thought I wanted to hear. Another suggested that it was because the interviews themselves took place in an English classroom (not my own, but the Head of Departments). Again, this could be an influencing factor, and is one that I hadn't previously considered. The third student suggested that perhaps English was a subject where they received the 'best' feedback, or one where they clearly recognized feedback for what it was. Again, this could be the case, and it will be interesting to see what comes up in the real interviews.
To address these issues, we decided to add a statement at the beginning of the interview informing students that the questions are meant to be about feedback in ALL of their lessons, across the school. Hopefully this will take away any desire to focus on English to please me or due to associations with the setting.
Overall, it was a really good learning experience; if we had more time, I would have them conduct another pilot interview. However, as their time at the school is running out, our first set of interviews will take place next week! How exciting!
Today we met to watch the footage back and discuss what we've learnt from the experience. I also showed them some footage from my own pilot, and we went through each one critically, looking at sound, angle, etc. To start, the students mentioned that the sound quality wasn't very good. You could clearly hear them (or me) asking the questions, but it was sometimes hard to hear the respondents. When I asked them how we could fix this, they suggested that we move the camera closer to the interview subjects. Eventually, we came to the decision that if students don't want to appear on camera we should place the recorder face up on the table. This way, we can clearly hear what they are saying. If students agree to be video-recorded, then we discussed seating the respondents side-by-side and setting the camera clearly in front of them on a tripod. The interviewers themselves don't have to appear on camera, and can be sat to the side of the camera itself.
Another issue that the group brought up was that one of the two interview subjects really dominated the conversation. I am glad this happened in the pilot, as it is a common problem with group interviews. They described using some of the prompting techniques I had taught them, but found that the dominant person still managed to speak for the other student. Again, I asked the students how they could fix this problem. One of the girls suggested knowing the names of the students they were interviewing ahead of time, so that they could personally address questions towards them. We agreed that this was a good technique to use, so I assured students that they would know the names of the students they would be interviewing; I can also use the school behaviour module to show them a picture of the students before-hand, so that they know exactly who is who.
A third issue was that they discovered some of the questions required 'yes/no' responses (which I knew was going to be the case; however, because I wanted the students to devise the questions, I didn't want to influence their choices too much by pointing this out), which didn't always invite much discussion. However, they immediately described using some of the prompt questions to try and dig for more detailed responses.
Similarly, they felt that through the conversation, some of the later questions became repetitive or redundant; because this was their first experience interviewing other people, they stuck to the questions and asked them all, despite this. In our discussion, I told them that they should use their professional judgement in these types of situations; if they think that they've already addressed a question once, they don't need to ask it again. They seemed pleased with this - I think perhaps they were afraid of missing things out if they left questions off, but once I explained that they are essentially in control, that they're the experts once they're in the room and they can use their own discretion, they felt more confident.
Finally, they described how the students seemed to focus on feedback in English lesson. I asked them why they thought this might be. One student suggested that it was because the students knew that I was involved, and that I was an English teacher; their responses may have been focused on English because that's what they thought I wanted to hear. Another suggested that it was because the interviews themselves took place in an English classroom (not my own, but the Head of Departments). Again, this could be an influencing factor, and is one that I hadn't previously considered. The third student suggested that perhaps English was a subject where they received the 'best' feedback, or one where they clearly recognized feedback for what it was. Again, this could be the case, and it will be interesting to see what comes up in the real interviews.
To address these issues, we decided to add a statement at the beginning of the interview informing students that the questions are meant to be about feedback in ALL of their lessons, across the school. Hopefully this will take away any desire to focus on English to please me or due to associations with the setting.
Overall, it was a really good learning experience; if we had more time, I would have them conduct another pilot interview. However, as their time at the school is running out, our first set of interviews will take place next week! How exciting!
Thursday 12 March 2015
Why should teachers engage in educational research?
As part of my MEd session, I was put into a group with a few other MEd students and asked to work as a team to produce a user-friendly guide which helped explain why teachers should engage in educational research. The information that follows is part of our group discussion on the topic.
Guiding Questions
1) What are ethics and why are they important?
Most people think that ethics deal with issues of right and wrong; in research, ethics are used to promote the aims of the research and to ensure that it is conducted in a way that is safe, fair and balanced.
2) How will ethics inform the way I conduct my research?
No matter what subject you choose to research, you must make sure that you are conducting an ethically sound study. This will ensure that people do not question the way you went about collecting, analysing and reporting your data. Being explicit in your ethical considerations means that you will be transparent (open) in the way you conduct your research.
Key Points
1) Make sure you get informed permission from all parties, including parents if students are being consulted.
2) Researchers must be open and honest at all times.
3) All participants have the right to withdraw their participation at any time, without fear of negative consequences.
4) Children should be consulted on matters that involve them.
5) Don't give incentives that could affect participant responses (i.e. chocolate or a day trip out).
6) Participants should remain anonymous wherever possible.
7) If any harmful effects are likely to result from the research process, the participants must be duly informed (this usually only effects medical research).
8) Don't falsify or distort your findings!
9) Don't take credit for anyone else's work.
10) Make sure that you communicate your findings in an accessible way.
Methods and Practicalities of Ethics in Research
1) Use consent forms and letters to parents.
2) Ensure open disclosure of the purposes of the research. This means that you should explain why you are undertaking your research, for what purpose(s) and for whom (audience).
3) Make it clear to participants that they can withdraw their support at any time. This should form part of the consent form. This may mean that you will lose participants throughout your project, and this is a problem when researching your own classroom. A good way around this is to have a backup plan or multiple options. You should remain flexible throughout your project.
4) Don't underestimate the power of student voice. Keep the students informed and answer their questions openly and honestly.
5) Change the name of participants to pseudonyms or numbers/letters. You should not be able to identify any participants by reading the finished report, so avoid any specific identifying features.
6) Keep a record of all your data. Research logs, diaries and journals are a good idea, as are keeping recordings and video data.
7) Always reference the work you are using. This means including the author(s) names and the date in which the source was published. For example, you would use (Carson et. al, 2015) if quoting our article.
8) Write in a way that makes your topic sound relatable and clear to others. Stick to the facts.
Key readings
The British Educational Research Association, (2011) 'Ethical Guidelines for Educational Research', London. Available online at https://www.bera.ac.uk/wp-content/uploads/2014/02/BERA-Ethical-Guidelines-2011.pdf
Hammersley, M. and Traianou, A. (2007) Ethics and Educational Research. London: TLRP. Available online at http://www.tlrp.org/capacity/rm/wt/traianou/
The Essence of Ethics
By K Carson, J Knight and N Taylor
Guiding Questions1) What are ethics and why are they important?
Most people think that ethics deal with issues of right and wrong; in research, ethics are used to promote the aims of the research and to ensure that it is conducted in a way that is safe, fair and balanced.
2) How will ethics inform the way I conduct my research?
No matter what subject you choose to research, you must make sure that you are conducting an ethically sound study. This will ensure that people do not question the way you went about collecting, analysing and reporting your data. Being explicit in your ethical considerations means that you will be transparent (open) in the way you conduct your research.
Key Points
1) Make sure you get informed permission from all parties, including parents if students are being consulted.
2) Researchers must be open and honest at all times.
3) All participants have the right to withdraw their participation at any time, without fear of negative consequences.
4) Children should be consulted on matters that involve them.
5) Don't give incentives that could affect participant responses (i.e. chocolate or a day trip out).
6) Participants should remain anonymous wherever possible.
7) If any harmful effects are likely to result from the research process, the participants must be duly informed (this usually only effects medical research).
8) Don't falsify or distort your findings!
9) Don't take credit for anyone else's work.
10) Make sure that you communicate your findings in an accessible way.
Methods and Practicalities of Ethics in Research
1) Use consent forms and letters to parents.
2) Ensure open disclosure of the purposes of the research. This means that you should explain why you are undertaking your research, for what purpose(s) and for whom (audience).
3) Make it clear to participants that they can withdraw their support at any time. This should form part of the consent form. This may mean that you will lose participants throughout your project, and this is a problem when researching your own classroom. A good way around this is to have a backup plan or multiple options. You should remain flexible throughout your project.
4) Don't underestimate the power of student voice. Keep the students informed and answer their questions openly and honestly.
5) Change the name of participants to pseudonyms or numbers/letters. You should not be able to identify any participants by reading the finished report, so avoid any specific identifying features.
6) Keep a record of all your data. Research logs, diaries and journals are a good idea, as are keeping recordings and video data.
7) Always reference the work you are using. This means including the author(s) names and the date in which the source was published. For example, you would use (Carson et. al, 2015) if quoting our article.
8) Write in a way that makes your topic sound relatable and clear to others. Stick to the facts.
Key readings
The British Educational Research Association, (2011) 'Ethical Guidelines for Educational Research', London. Available online at https://www.bera.ac.uk/wp-content/uploads/2014/02/BERA-Ethical-Guidelines-2011.pdf
Hammersley, M. and Traianou, A. (2007) Ethics and Educational Research. London: TLRP. Available online at http://www.tlrp.org/capacity/rm/wt/traianou/
Devising Interview Questions
This is a bit delayed, of which I am a bit sorry to admit; usually I like to type up my thoughts immediately after a sessions. However, last week was a little hectic, so I haven't been able to reflect on my most recent meeting with the student researchers. In our last meeting, we looked at the questions I used during my pilot (adapted from Gamlem & Smith, 2013) and adapted them to suit our own purpose. I was quite struck by a number of the students suggestions, and will briefly outline my thoughts below:
Question 1: When was the last time you got feedback? What was the feedback? Can you give examples? How useful was it?
The SaRs felt that this question was far too long, and suggested that we trim it down and simplify it. They suggested that we leave the first part of the question, but simply ask students to describe the feedback, if they could recall a specific example.
Question 2: What do you find useful when getting feedback from your teacher? Why?
Again, the students suggested that we change this question slightly. They felt that by including the phrase 'from the teacher' was too leading; their argument was that feedback doesn't always come from the teacher, and that by leaving it as simply 'when getting feedback' it left the options of peer and self-assessment there, while not necessarily bringing it up (which would be left specifically for later).
Question 3: What don't you like when getting feedback from your teacher? Why?
We removed the 'from the teacher' part again, for the same reasons as above.
Question 4: What do you usually do with the teacher feedback? How often do you get the opportunity to work with your targets?
This question was changed dramatically. Again, the students felt that the question was too 'leading'. First of all, it implied again that the only type of feedback was teacher feedback. Secondly, they felt that it implied that students DID get the opportunity to work with their targets, which might influence their answers. Instead, we changed the question to 'How do you respond to feedback?'
Question 5: What usually happens when you get feedback? How is feedback followed up?
This question was completely deleted; students felt that it was too similar to the previous question AND suggest that it was (again!) too leading.
Question 6: How do you do peer assessment?
Once again, they identified this was a 'leading' question; it implied that they DID do peer assessment. Instead, we changed the question to be 'Do you ever use peer assessment? If so, how?'
Question 7: Do you self-assess your own work in lessons? How do you do this?
We left this one the same.
Question 8: Are there times when you don't get feedback but you'd like to?
We left this one the same.
Question 9: How are you told about moving from one level to the next?
We changed this one as well, eliminating the 'how' at the beginning and changing 'one level to the next' to simply 'Are you told how to improve your work? If so, how?' The students felt that this left the question more open ended.
Question 10: Do you know what your teacher looks for when assessing your work?
In this question, 'assessing' was changed to 'marking' in order to be less formal.
Finally, we decided to add a question at the very beginning and start with 'What is the main purpose of feedback?'
Question 1: When was the last time you got feedback? What was the feedback? Can you give examples? How useful was it?
The SaRs felt that this question was far too long, and suggested that we trim it down and simplify it. They suggested that we leave the first part of the question, but simply ask students to describe the feedback, if they could recall a specific example.
Question 2: What do you find useful when getting feedback from your teacher? Why?
Again, the students suggested that we change this question slightly. They felt that by including the phrase 'from the teacher' was too leading; their argument was that feedback doesn't always come from the teacher, and that by leaving it as simply 'when getting feedback' it left the options of peer and self-assessment there, while not necessarily bringing it up (which would be left specifically for later).
Question 3: What don't you like when getting feedback from your teacher? Why?
We removed the 'from the teacher' part again, for the same reasons as above.
Question 4: What do you usually do with the teacher feedback? How often do you get the opportunity to work with your targets?
This question was changed dramatically. Again, the students felt that the question was too 'leading'. First of all, it implied again that the only type of feedback was teacher feedback. Secondly, they felt that it implied that students DID get the opportunity to work with their targets, which might influence their answers. Instead, we changed the question to 'How do you respond to feedback?'
Question 5: What usually happens when you get feedback? How is feedback followed up?
This question was completely deleted; students felt that it was too similar to the previous question AND suggest that it was (again!) too leading.
Question 6: How do you do peer assessment?
Once again, they identified this was a 'leading' question; it implied that they DID do peer assessment. Instead, we changed the question to be 'Do you ever use peer assessment? If so, how?'
Question 7: Do you self-assess your own work in lessons? How do you do this?
We left this one the same.
Question 8: Are there times when you don't get feedback but you'd like to?
We left this one the same.
Question 9: How are you told about moving from one level to the next?
We changed this one as well, eliminating the 'how' at the beginning and changing 'one level to the next' to simply 'Are you told how to improve your work? If so, how?' The students felt that this left the question more open ended.
Question 10: Do you know what your teacher looks for when assessing your work?
In this question, 'assessing' was changed to 'marking' in order to be less formal.
Finally, we decided to add a question at the very beginning and start with 'What is the main purpose of feedback?'
Subscribe to:
Posts (Atom)