Announcement

Collapse
No announcement yet.

STEM Judges feedback for teams

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • STEM Judges feedback for teams

    I have been thinking about how I would like to do feedback for teams. I agree that the rubrics should not be passed back for the sake of candid judges notes. How about this one? I think it would be really easy for the judges and would give the kids an idea of what they should focus on.
    Attached Files

  • #2
    It's not too bad compared to some other ideas but I still think the following scenarios happen with this:

    1. Teams (in particular adults/mentors/parents) look at each others feedback and get confused why they didn't win if they had more + then a team that got a - (remember rubric is only part of winning a judged award.) Teams/parents/mentors demand to see the sheet from the team that won, etc. etc.

    2. This will still add some amount of time to the judging process - 30-60 seconds per team is significant at anything larger then a small event. I've seen judges take deliberation all the way to the final cutoff just because scores/impressions were so close more then once.

    I feel like teams should be able to get a pretty good idea from having parents/volunteers/mentors/teachers/adults they don't know go through the process with them during PRACTICE. This is something we encourage mentors to make happen BEFORE events.

    Comment


    • #3
      Originally posted by sankeydd View Post
      I have been thinking about how I would like to do feedback for teams. I agree that the rubrics should not be passed back for the sake of candid judges notes. How about this one? I think it would be really easy for the judges and would give the kids an idea of what they should focus on.
      I think it's a great start. Would love to see it at events.

      Comment


      • #4
        I strongly disagree with any notion of providing teams direct feedback.

        The best way I've found to help teams improve is by directing them to ask the team that did earn the award to share their work - most teams are happy to point out technical details, show engineering notebooks, and the like. By not showing them checkboxes or a rubric, you are giving them an opportunity to observe and reflect - a skill that is very useful as they go on in other endeavors, including their next competition.

        Teams will tend to take any rubric, no matter how rudimentary, and start comparing notes. What if a group of teams from the same school gave very similar STEM Presentations, but got awarded different marks? What if 2 teams as an experiment hand in identical notebooks to compare how close the judging is? What if they find discrepancies between judges? What if they got a minus mark on something they paid particular attention to do? What if a judge felt bad about giving too many minus marks and put in a few positive checks to balance them out, even if those weren’t really plus worthy?

        None of these things are probably what you want, which I presume is to give teams feedback so they may meaningfully invest their time and energy to improving. Many people here come from the world of education, where we give students rubrics so they may improve for the next assignment or project. In a classroom setting this makes sense, and speaking as a teacher, a lot of this comes from that I am the only person looking at their present and future work for the course. A part of my job is building the relationships and rapport with students so that the results of my rubric, shared openly, can offer a student insight on what they should do to do better next time.

        At a competition, judges do not have that same relationship, so feedback generally becomes a point of fixation, frustration, or rationalization. The rubric you have provided minimizes the direct commentary, but it does not minimize the risk of teams misinterpreting what you give them, and it furthermore invites more questions - questions that I candidly tell teams at events I have a hand in organizing they are not permitted to answer. Giving this kind of feedback isn't fair to the judges who now how their thought process exposed to those children whose work they needed to judge, and it isn’t really productive or fair to those students either – ultimately they are far better served by looking at the publicly available rubrics, looking at the award winning example, and by self-evaluating. This lets students learn and improve at their own pace and continues to let judges deliberate, score, and judge openly without having the individual judgments leading up to their ultimate decision exposed to be canvassed after each event.

        Comment


        • #5
          Originally posted by Ben Mitchell View Post
          The best way I've found to help teams improve is by directing them to ask the team that did earn the award to share their work - most teams are happy to point out technical details, show engineering notebooks, and the like.
          I agree with this. But after an event it's just not going to happen for the vast majority of teams. I'm trying to bring a great STEM experience to a vast number of students.

          I posted a new version on a different thread, limiting the +'s and -'s to 2 each. Is that any better? Perhaps one each? I teach in a middle school classroom and for most of my students it would be great for them to have something to fixate, frustrate and rationalize over. Getting no feedback gives them absolutely nothing to do. I go over their materials at least twice a month but something from the outside helps a lot.

          And how do you show your students the winning STEM presentation? I have thought about doing recordings, and playing the winning one at the end, but am I going to make all those parents and coaches sit there for extra time after the whole thing is over? Would the kids even be able to pay attention at that point? Should I post in on youtube? Would the winning team ever consent to either?

          Comment


          • #6
            Originally posted by sankeydd View Post

            I agree with this. But after an event it's just not going to happen for the vast majority of teams. I'm trying to bring a great STEM experience to a vast number of students.

            I posted a new version on a different thread, limiting the +'s and -'s to 2 each. Is that any better? Perhaps one each? I teach in a middle school classroom and for most of my students it would be great for them to have something to fixate, frustrate and rationalize over. Getting no feedback gives them absolutely nothing to do. I go over their materials at least twice a month but something from the outside helps a lot.

            And how do you show your students the winning STEM presentation? I have thought about doing recordings, and playing the winning one at the end, but am I going to make all those parents and coaches sit there for extra time after the whole thing is over? Would the kids even be able to pay attention at that point? Should I post in on youtube? Would the winning team ever consent to either?

            Bringing a STEM experience and offering direct feedback on judging are 2 entirely different things. If students are interested, they will ask each other questions and seek each other out.

            The number of pluses and minuses don't matter - feedback so overbroad invites comment and followup questioning in order to become useful, which is the problem.

            Perhaps at the end of a season, the teams with winning STEM awards could be convinced to post a youtube video of their presentation as an example for the next year. Perhaps someone can organize a post-season workshop for those who are interested.

            Ultimately teams need to be self-reflective and do the best they can - and self-improvement should be their goal, not winning the award itself. Teams can be encouraged to video themselves giving their presentation at various stages of the season to see how they grow and improve.

            I know that as a judge and judge advisor, I would flat out refuse to volunteer if notes from the judging room, even notes as broad as what you suggest, were to be made public.


            Comment


            • #7
              Ben Mitchell, I'm sorry you feel that way. As a coach, and as an event partner, I know that feedback is beyond crucial, it is necessary. Yet, here we lock the kids in a room, don't allow them to see their competition, and then don't tell them how they did. Huh? How does that make any sense. Everyone knows that in order to get better, you have to have feedback. Even the RECF knows that: https://www.facebook.com/RECFoundati...155039/?type=3. But, they don't want the kids to receive feedback from the most important person, the judge?

              Comment


              • #8
                Originally posted by daddycrusader View Post
                Ben Mitchell, I'm sorry you feel that way. As a coach, and as an event partner, I know that feedback is beyond crucial, it is necessary. Yet, here we lock the kids in a room, don't allow them to see their competition, and then don't tell them how they did. Huh? How does that make any sense. Everyone knows that in order to get better, you have to have feedback. Even the RECF knows that: https://www.facebook.com/RECFoundati...155039/?type=3. But, they don't want the kids to receive feedback from the most important person, the judge?
                Where is it stated that we cannot let them see their competition? There is nothing prohibiting teams from displaying their notebooks, robots, or STEM presentations on whatever medium is appropriate - be it youtube, a web site, or in person. That's something teams can always elect to do, and many do so.

                I think I described the problems with giving students judging feedback rather extensively. Wouldn't it be better, perhaps, for you to organized a workshop where you can talk about awards and what makes a good submission, without comparing teams to each other? Perhaps the winning teams from the past season would be willing to show their work, explain their process, and give examples? This way the feedback is general, and again, students and coaches can take that general advice and reflect on their own work.

                The link to the REC Foundation suggests having teams present to adults or mentors for practice and for feedback. This makes sense, as I mentioned previously in my first post on the subject, because those adults and mentors would presumably have the personal connection that would make the feedback constructive. Having judges reveal their judging process is entirely different, since that connection does not exist. Teams will be getting feedback that is impersonal, one-sided (You can't expect judges to engage every team in a conversation about every submission) and because it is happening immediately after the award ceremony, many teams I fear would fixate on outcomes rather than improvement. "What can we do to improve?" - the core question of constructive feedback, is replaced by "Why didn't we win?" When every team is getting a rubric back, it invites this controversy.

                I understand that many coaches would want direct feedback, but again, what works in a classroom does not work at a competition, and it isn't right to put judges in a position where their feedback may inadvertently give offense or cause hurt, where perceived slights or oversights can be given a forum to be aired and perpetuated, and it isn't fair to students, since feedback this general is likely of very limited use anyhow, and all it takes it one person to cry foul for the entire process to be needlessly called into question, damaging the experience for many participants.

                Lastly, the judges guide mentions that all notes and rubrics should not be returned to teams, so doing this would be a violation of the guidelines set forth in that document.

                If you want feedback, by all means get some adults who are good with kids to review the rubrics and guides, and see your students' presentations or notebooks or whatever and give their feedback in the form of a conversation. Asking competition judges for that feedback is neither appropriate nor truly helpful, for the reasons discussed above.

                Comment


                • #9
                  Ben, I agree with you from the point of view from the judges. I don't like the idea of handing back the rubric as I don't want the judges to be distracted while they are taking notes. I don't want them to be worried about offending anybody and I want their remarks to be candid. I also know that any feedback is extra work for the judges, and the "easier" you make it for the judges the less meaningful the feedback is.

                  I also agree that judges are just human, and five different sets of judges could have selected five different winners after watching the exact same presentations or looking at the exact same notebooks... Their feedback isn't terribly useful in that sense either as different things.

                  I was a STEM judge for our FL Elementary IQ State competition last year. It was a long stressful day and you better believe I did not want to return those rubrics! It was my first experience on that side of the coin. There were a couple of groups that did so poorly that I was a little upset with their coach for not just telling them to not do it. One of them was actually a little offensive. There was no feedback I could have given that team that would have been helpful.

                  Originally posted by Ben Mitchell View Post
                  Where is it stated that we cannot let them see their competition? There is nothing prohibiting teams from displaying their notebooks, robots, or STEM presentations on whatever medium is appropriate - be it youtube, a web site, or in person. That's something teams can always elect to do, and many do so.
                  I do disagree with this model. In my experience it is just not practical. My kids are all middle school students in my classes. I can't just take them to some other school or team to see notebooks and projects. You see a few videos for stem presentations but not many. And a lot of the published notebooks are from teams that win design at worlds, that's not exactly the type of standard I want to throw at my kids. And after events coaches, parents and kids want to go home, and don't have time to share what they have with anyone.

                  I really don't have much of an answer for you. I will try to do... something. Maybe I could create another volunteer position for "feedback," someone who is not a judge but watches the presentations only for the sake of providing feedback to the groups. If only we had this many people!

                  I would like to continue this discussion and I am trying to approach it from all sides. I want to help point these kids in the right direction.

                  Comment


                  • #10
                    Perhaps on a local level, teams that have winning submissions can be asked to do their STEM Presentation for youtube, or scan some selected pages of their notebook to be posted online or put into an email for other teams to view when convenient for them?

                    This would allow those teams that want to improve a model to compare their performance against, and doesn't take up time at the end of the day at a competition. It also shows a model while keeping direct feedback out of the equation. I think the majority of teams would be proud to show off their projects.




                    Comment

                    Working...
                    X