How to give good feedback

From TED Translators Wiki
Revision as of 12:39, 20 March 2017 by Symbolt (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Feedback could be defined as comments about someone’s work designed to improve future performance. By sharing constructive feedback on the subtitles they work on, reviewers and language coordinators act as mentors, helping less experienced volunteers develop the skills necessary to raise the quality of their subtitles and have an easier time working in the TED Translators program.

This article contains tips and resources to help you structure feedback on other volunteers’ subtitles.

The goal of sharing feedback

Your feedback should allow the other volunteer to gain the skills necessary to make their future subtitle work better, on their own. Feedback should never be about simply lavishing the volunteer with praise that doesn’t suggest what to stick to in the future. Feedback must also never be about making the other volunteer feel bad about the quality of their subtitles. By sharing constructive positive and negative feedback, and always remaining open to discuss one’s edits, both beginners and experienced subtitlers in the TED Translators Program can continue to learn how to hone their craft and become better at accurately representing the speakers’ ideas in their subtitles.

Constructive feedback vs positive/negative feedback

It is important to distinguish between constructive and non-constructive feedback, as well as positive and negative feedback.

Constructive feedback consists of information that allows something new to be “constructed”: comments that show the user how to change the way they subtitle and use better strategies. Positive feedback is praise, while negative feedback defines areas where quality was lower than expected. All feedback should be constructive. Both positive and negative feedback can be used in constructive or non-constructive ways, for example:

You did great! In my experience, beginners’ work is rarely as amazing as yours, I was really blown away. Keep up the good work!

Positive, non-constructive feedback: the volunteer gets praise, but doesn’t know what exactly are the things they did well and which they should continue doing.

Some of the subtitles had reading speeds over 21 characters / second. Subtitles with technical issues like that have a red “exclamation point” in the editor. You can learn how to deal with reading speed-issues by watching this short tutorial.

Negative, constructive feedback: the volunteer learns about what they did wrong, but is not judged for it, and instead is given information and tools that will allow them to avoid making the same mistakes in the future.

Below, you can find many more examples of constructive and non-constructive positive and negative feedback.

How to use positive feedback

Many of us have to make an effort to share positive feedback, since we are mainly used to getting negative corrective feedback at school and at work. However, positive feedback is vitally important, because it motivates the user to keep doing a good job, and shows them what they are doing well and should keep on doing.

Train yourself to point out the positive

In order for feedback to be constructive, you should try to maintain a balance of positive and negative feedback, and make double-sure that you do not fail to provide enough constructive, positive comments. Look at the subtitles for examples of things that work well, and point them out, explaining why the strategy used is correct. Even if the spelling was bad, perhaps there were no technical problems? And if you found huge technical issues in the subtitles, perhaps the translator had a few amazing translation ideas, did good research to find the right equivalents of terms and proper names, or (in case of transcripts) had very good timing?

Even if everything needs to be corrected, you can compliment the translator on investing their time to try to help share this particularly inspiring or challenging talk with their language community.

Remember that positive feedback should be constructive and accurate. Never say anything that isn’t really true about the subtitles, just to make the other volunteer feel better. Instead, challenge yourself to try to find real, positive things in their work to praise and talk about.

Examples of good and bad positive feedback

Great job! Very few edits. You should give the reader more details about what was good about their work.

Great job! No technical mistakes, good style. Punctuation OK too. This is better, but if a lot of negative feedback follows after this, you should consider making this opening bit of positive feedback more detailed, like this:

Great job! There were no issues with reading speeds, line lengths, or line breaks (which were correct in terms of line-length balance, but also didn’t split up linguistic wholes). I especially liked the way you improved the distribution of text between subtitles compared to the original, in cases where the original transcript merged the end of one clause and the beginning of another (e.g. at 3:09, the original was "y eso era lo que quería. Y descubrí | otro ejemplo en un estudio de 1999." and you fixed it to say “and that was what I wanted. | And I found another example/in a study from 1999.”). The speaker is sometimes somber, and sometimes funny, and you recreated that balance really well. It’s good to show the speaker’s style in your translation, and especially good to make the audience laugh at the translation where the original audience laughed, too! Your punctuation was also perfect (you used spaces before dots – love that!). So, awesome job! You’ll find some more detailed comments below.

This is a very detailed bit of introductory positive feedback, which nicely sets up the tone for your remaining comments. Even if there are multiple constructive negative comments below, the reader will see that you recognized their skills, and the negative feedback is designed to help them get better. This bit of feedback always merges a bit of praise with some more details that help the subtitler who reads this zone in on what they should keep doing the way they did in these subtitles. If possible, such an introductory bit of positive feedback should be joined by some bits of positive feedback about some specific subtitles below, among negative feedback. Here’s an example:

0:03.69 - I love the way you translated “colocar” here – you made it as funny as the original! You should try to find such examples of good subtitles and comment on them. This bit of feedback also explains why this translation is good: it was as funny as the original, which tells the translator that trying to make the translation as funny as the original is a good strategy. Even if such things seem obvious to you, they may not be intuitive to a beginner, who may be trying different strategies pretty randomly, and even this small bit of guidance will be helpful to them.

Note that this comment includes the start time code of the subtitle, which may make it easier to find. If you are not sure that just giving the time code will be enough for the volunteer (e.g. they’re a total beginner and may not understand about time codes), consider quoting the whole subtitle.

Above, I shared comments about the most important edits. You can see all of my edits easily by using Amara’s revision comparison feature; to learn how to use this feature, watch this short tutorial: http://tinyurl.com/Amara-diffing Let me know if you have any questions or whether you’re unsure about any of my edits (we can always work together on coming up with a different version of a subtitle, if you’re not happy with the current one). Once again, I want to congratulate you on how well you did, especially with the technical aspects of the subtitles, and the humor in the talk. Keep up the good work! Your Spanish-English translation skills are exceptional. If you want to help translate more Spanish talks into English, you can use this link from a tool created by an OTP volunteer, which displays Spanish talks without an English translation: http://amaratools.ted-ja.com/tasklist/?query=&role=Translate&project=all&language=en&source_language=es&assign=unassigned&sort_order=new_task&page=1

This bit of closing feedback instructs the reader about how they can check all of the edits in detail and emphasizes that you are open to discussing your changes. It also reiterates some positive feedback with detailed comments, and leaves the subtitler with an action item, which is always a good strategy. Examples of such “action items” or links to other resources include: a link to find more subtitling tasks of a specific type (e.g. shorter videos, if you feel that the given volunteer should practice on shorter material), links additional resources that they can study if they wish to learn more (e.g. the full transcribing guide), or the OTP Facebook group for the given language.

How to use negative feedback

Negative feedback should point out errors, and at the same time, explain how to avoid them in the future, to “emancipate” the reader and allow them to learn better strategies. Don’t forget to explain what exactly makes the given item incorrect, instead of just quoting the incorrect word or phrase (explain the type of mistake: spelling, grammar, style, etc.).

Remember that the purpose of negative feedback is never to make the other volunteer understand “how bad they are” at subtitling. Instead, your negative feedback must always be about identifying where the quality of the given volunteer’s subtitles should be improved, and doing your best to teach them how to improve it.

Emancipate the other volunteer

Always try to “emancipate” the receiver of the feedback: give them the tools to understand what they have been doing incorrectly, and to learn how to avoid making similar mistakes in the future. As much as possible, don’t simply state that something is incorrect, but include links to outside resources (dictionaries, grammar / style guides) that support your statements. This way, the other volunteer will be exposed to tools that they can use in the future to check how to do things before they make a totally different mistake. Instead of feeling like the person who reviewed their work has a lot of knowledge and skills that they themselves lack, they will think “Hey, when I’m not sure, I can look for information about commas in grammar blogs like this one.”

Instead of viewing negative feedback as a list of items they are ignorant about, empowered with such tools and strategies, the other volunteer will be able to start developing confidence in subtitling, and to learn the skills that may eventually allow them to start reviewing other people’s work as well.

Be objective, never personal

Always make sure that the feedback is about the work (the subtitles), and do not make any personal comments. Avoid adding any “judgment” to your negative feedback, like “terrible,” “surprisingly bad” or “objectively wrong.” Instead, make it objective, to the point, and instructive. A lot of people will feel embarrassed or concerned just because you point out mistakes in their work, so be extra careful not to add to the often unpleasant feelings of reading negative feedback by making it about the person.

Even if you’re feeling frustrated with the amount of mistakes in the subtitles, you will not gain anything by making the receiver of the feedback feel bad about themselves. Focus on how these mistakes can be avoided in the future, and this way, you’ll emancipate the volunteer by helping them learn how to get better and better on their own, and thus, you will limit your future frustration by fostering an increased subtitle quality in the languages you work in. Always keep in mind that even though online collaboration makes the other volunteer seem remote, you are talking to another volunteer like you, who did not make these mistakes on purpose.

Limit your comments to the most important issues

Overwhelming the reader with too many comments about their work will most likely make your feedback less effective. Focus on the most important issues – those that deeply impact the meaning or accessibility of the subtitles, and those where the other volunteer can grow by reading your feedback.

Always try to prioritize. For example, if you found 30 different grammatical mistakes in the subtitles you are reviewing, you may want to comment in detail only on 5-10, and then indicate that you made other grammatical improvements, and that the volunteer can review them in detail by using the revision comparison feature, and ask you for more details if necessary. By doing this, you allow them to realistically study your detailed guidelines, remember and learn them. If you commented on all 30 mistakes, and then added feedback about technical issues, style, accuracy and the title and description format, the reader would likely get lost in your comments and retain none of your suggestions.

Distinguish between errors and mistakes

It’s useful to remember the distinction between errors and mistakes. Errors are stable, recurring incorrect strategies or patterns. An example of an error would be a situation where a translator very often makes mistakes when using commas with subordinate clauses. Mistakes, on the other hand, are not stable, recurring patterns, but are more haphazard or accidental. For example, a translator uses commas with subordinate clauses correctly almost all the time, but did not use a comma correctly with two out of 20 subordinate clauses in the talk.

When deciding what to talk about in your feedback, always go with errors over mistakes. It is more useful to point out a recurring, incorrect approach and explain how to correct it than to draw attention to a few slip-ups that may have been completely accidental. Focusing on errors in favor of mistakes makes it easier to avoid overwhelming the other volunteer with too much negative feedback.

Avoid exaggerating frequency

When talking about mistakes, avoid talking about their frequency. Most volunteers will see how frequent your corrections were by comparing revisions, and emphasizing how many there were usually only serves to make the other volunteer feel bad.

If you do wish to mention error frequency, do not use vague expressions like “many errors” or “a lot of mistakes,” because such non-specific descriptions of frequency only express one’s subjective opinion, and simply serve to exaggerate the amount of issues in the subtitles to make the translator or transcriber realize “how bad they are.” However, the purpose of sharing feedback is not to make the other volunteer doubt their skills, but to give them new skills that will allow them to raise the quality of their subtitles. If you do refer to error frequency, always use objective language in favor of subjective judgments, and don’t exaggerate (“I fixed this in multiple subtitles throughout the talk” --> “I fixed this in a few more subtitles / there were about 3-4 more subtitles where I added a comma: not bad, for an 18-minute talk!”).

Examples of good and bad negative feedback

Consider these examples of negative feedback. They all represent various ways of commenting on the following edit:

Original subtitle: The tree, it is -- deciduous, and it’s leaves are pretty

After edits: The tree is deciduous, and its leaves are pretty

“The tree -- it is deciduous, and it’s leaves are pretty” – a lot to fix here (spelling, grammar / style)

This bit of feedback is not very useful, since even though it identifies the types of mistakes in the subtitle, it doesn’t specify where they occur (what was misspelled? where exactly are the stylistic mistakes?) and how to avoid them in the future. It also fails to prioritize the most important mistake in the subtitle, and unnecessarily mentions and exaggerates error frequency (“a lot to fix”).

“The tree -- it is deciduous, and it’s leaves are pretty” – spelling

This comment correctly identifies the type of mistake (spelling), and focuses on the most important issue in the subtitle, but it doesn’t explain where exactly the mistakes are (which words are misspelled?), and it doesn’t provide any information about how to avoid making the same mistake in the future.

You misspelled “its.” A little better, because it’s clear which item is the most important, but this feedback still doesn’t explain how to avoid making this mistake again.

You should use the possessive adjective here (“its”), not the contracted form of the copula.

Even though the grammatical terms used in this sentence are correct, including them here without any additional explanation of what they mean is not useful, since these terms don’t communicate anything to the reader and make the feedback sound unnecessarily complicated. It is of course perfectly all right to use sophisticated terminology, but only coupled with immediate, practically useful explanation of what the terms mean. If you find that your feedback can be equally useful without using such terminology, consider removing it altogether.

“It’s” should be “its.” This is a very basic mistake and it’s important to avoid it, since it would be bad if people saw things like that in a published translation.

This bit of feedback clearly identifies what was incorrect and how it was changed, but doesn’t provide the user with an “emancipating” explanation of how to avoid similar mistakes in the future. It is also unnecessarily personal. Nothing practical is gained by using judgment / opinion terms like “a very basic mistake,” or admonishing the reader by talking about the unpleasant consequences of exposing viewers to the low quality of their work.

I changed “it’s” to “its.” It’s easy to confuse the two, but it helps if you focus on the difference in meaning between these: “it’s” is the contraction of “it is” (similar to “he is” --> “he’s” or “she is” --> “she’s”), while “its” is a possessive form, similar to “his” or “her.” To learn more, see this article: http://grammarist.com/spelling/its-its/

This is an example of good negative feedback: it explains what exactly is the mistake being addressed, shows what the correct version should be, is focused on the work, not on the person, and provides strategies that should allow the reader to learn how to avoid making this mistake in the future. Even though there were other edits made in the subtitle, this bit of feedback is focused on the most important element (the most serious issue). It also has the additional perk of sharing a link to a type of external resource which can help the reader with other grammatical questions.

Make it enjoyable

Always make your feedback clear and easy to follow, but don’t be overly dry and unnecessarily formal. Stay objective and to the point, but don’t be afraid of sharing an interesting and related link or commenting on something you learned when working on the subtitles. These “personal touches” remind the other volunteer that they are talking to a real person who is trying to be friendly, which may make it easier for them to follow up on your advice. Even though it is entirely up to you whether to occasionally share such comments or to strictly focus on business, keep in mind that hundreds of volunteer TED Translators and Transcribers forged friendships that started from a discussion about a review, so perhaps you should go ahead and share that link to that funny Doctor Who episode that the talk immediately made you think of!

Start and end with the positive

It is a good idea to start with some positive feedback, then continue with negative feedback, interspersing it with positive comments, and then finish on a positive note again. Starting with the positive makes it emotionally easier to learn about one’s mistakes, and ending on a positive note helps to motivate the other volunteer to make use of your advice, by reminding them that even though they may have a lot to learn, they already do possess some skills that they can lean on.

Always be open to discussing your feedback

Encourage the other volunteer to reach out to you if they are not sure about some of your edits and / or comments. Let them know that you are open to discussing your changes, and possibly collaborating to create an even better version of some more challenging subtitles. A big part of the magic and joy of volunteering in the TED Translators Program is the fact that both beginners and seasoned subtitlers can keep learning from one another. Make the discussion fun and civil. If a dispute should arise, do not allow conflict to escalate, and if necessary, do not hesitate to reach out to your Language Coordinator for advice or, if necessary, direct mediation.

Other tips, tools and resources

You can make creating feedback easier by using some of the following hints and resources.

Watch the video tutorials

Don’t forget to watch our tutorial on giving feedback, and do this TED-Ed lesson on reviewing, which also includes suggestions of good and bad feedback.

Watch and read other feedback tutorials

In addition to this article and the OTP Learning Series tutorials, you can find many useful guides to giving feedback online. Here are some suggestions:

An interactive tutorial with many examples of good and bad feedback A short and well-organized paper on giving feedback on writing A very interesting article on cultural differences in feedback strategies

Save templates of the most common feedback items

Feedback must be tailored to the particular person and subtitles, but there are some comments that often recur for anyone. You can make giving feedback more convenient if you assemble these comments as templates, and then only adapt them in each case. See a list of examples here.

Explain how people can check your revisions

Because you don’t want to comment on every single edit, it’s a good idea to always point people to the tutorial on using Amara’s revision comparison feature, which will allow them to see all of your changes in detail.

Use the timecode link to comment on a specific subtitle

In Amara’s interface, hover over a subtitle, and then select the clock icon from the little menu that shows up. This will insert the timecode for the subtitle into the note field, making it easier for the volunteer reading your comment to figure out which subtitle you are talking about.

Consider quoting the before-and-after versions of a subtitle

Subtitlers can always use Amara’s revision-comparison feature to see what edits have been made in a given subtitle, but to make it easier for the volunteer you’re working with, consider quoting the original subtitle and your edited version when giving feedback, especially when the changes you made are really important or tough to see easily, e.g.:

Original: I went to the park to play with my dog Porthos.

Edited: I went to the park to play with my dog Porthos.

The translation was completely fine, but I changed the way the line was broken because of the 50% length rule: one line should not be more than 50% shorter than the other. You can find more useful, small tech rules like this one in the TED Translators Cheat-sheet (hint: it’s printable!): http://translations.ted.org/wiki/TED_Translators_Cheat-sheet

Point beginners to short tasks

If you are giving feedback to a beginner, and you can tell they need more practice, it may be good to suggest that they start with shorter videos, since this way, they will get more frequent feedback. To help them find short videos, you can send them a link to Amaratools, a third-party app written by Japanese Language Coordinator Yasushi Aoki. This tool makes it possible to sort tasks by video length. For example, this link shows the shortest TEDTalks without a Somali translation. By changing the filter settings, you can create different lists to share in your feedback, like a list of the shortest videos in need of a transcription.

Link to tutorials in the playlist

When you link to one of the video tutorials, it’s a good idea to use a link that shows the tutorial as part of the whole OTP Learning Series playlist. This way, the volunteer you are working with will be more likely to watch other tutorials too, if they haven’t done so before. Here are shortened links to the Learning Series tutorials as part of the whole playlist:

Use the Main TED Translator Resources guide to find links

If you want to share a guide or article, but can’t remember where it was, browse the index of the Main guide to TED Translators resources to find the topic related to this resource, and then click to find the link in that section.