Difference between revisions of "Evaluation report"

From TQAuditor Wiki
Jump to: navigation, search
(→‎Units display)
 
(190 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
''Evaluation report page: [http://cloud.tqauditor.com/evaluation/index?id=XXXX/ http://cloud.tqauditor.com/evaluation/index?id=XXXX] (replace XXXX with a valid number)''
+
=='''General information'''==
−
----
 
  
 +
After the evaluator uploaded files, they can start the evaluation.
  
−
After evaluator uploaded files, he can start evaluation:
+
The evaluator can start the evaluation whether with automatic word count or enter it manually when starting the process:
  
−
[[File:Evaluation report-Start evaluation.png|Start evaluation]]
+
[[File:Evaluations.png|border|200px]]
−
+
 
−
You may configure evaluation process:
+
For more info on both methods, please check the relevant sections below.
 +
 
 +
=='''Automatic vs. manual word count'''==
 +
 
 +
'''Automatic:'''
 +
 
 +
1. Used for fully reviewed files.
 +
 
 +
2. The "Evaluation sample word count limit" is used to adjust how many segments for evaluation will be displayed.
 +
 
 +
3. The system will display only corrected segments (selected randomly) with the total word count specified as "Evaluation sample word count limit".
 +
 
 +
''For example'', if 1000 was specified as "Evaluation sample word count limit", the system will display around 100 segments with around 1000 words in total.
 +
 
 +
::''Please note that the number of segments varies depending on the size of segments''.
 +
 
 +
::<span style="color:orange">'''Note'''</span>: If the evaluator specifies 1000 as "Evaluation sample word count limit" while there are only 500 words in all corrected segments (let's say, there are 900 words in the file), the system will still display corrected segments with around 500 words in total. It means that ''1000 can be safely used as "Evaluation sample word count limit" even if the real total word count is lower''.
 +
 
 +
4. When calculating the score, the "Total source words" from the "Evaluation details" section (not the Total source words of a file) is used:
 +
 
 +
[[file:TSW.png|border|300 px]]
 +
 
 +
''For example'', if the evaluation report includes corrected segments with around 1000 words and the total source words is 1757, 1757 will be used in the formula.
 +
 
 +
'''Manual:'''
 +
 
 +
1. Used for partially reviewed files (in order not to split the file into parts and import only the reviewed part).
 +
 
 +
2. The "Evaluated source words" should reflect the total number of words in the reviewed part of the file.
 +
 
 +
''For example'', a reviewer reviewed only 1500 words in a 5000-word file. Then they should specify 1500 as "Evaluated source words" and the system will not take the remaining 3500 words into account.
  
−
[[File:Evaluation report-configure evaluation process.png|Configure evaluation process]]
+
3. The system will display all the corrected segments. So, if the reviewed part of the file is large, the evaluator will have to evaluate way more segments.
−
 
−
'''Skip segments with match less than''' — fuzzy match percentage. Program will hide segments with matches less than you specified.
 
  
−
'''Skip segments with match greater than''' — fuzzy match percentage. Program will hide segments with matches greater than you specified.
+
4. When calculating the score, the "Total source words" is used. In this case, "Evaluated source words" = "Total source words".
  
−
'''Skip locked segments''' — hide "freezed" segments (for example client wants some parts, extremely important for him, stayed unchanged
+
=='''Start evaluation (automatic word count)'''==
  
−
and block them, and extra segments slow down editor’s work).
+
If you select this option, the system will display randomly selected segments containing only corrected units for evaluation:
  
−
'''Evaluation sample word count''' — quantity of words in edited segments, chosen for evaluation.
+
[[File:Start evaluation automatic.png|border|200px]]
  
−
Press '''Start evaluation'''.
+
Then you may configure the evaluation process:
  
−
=='''Mistakes'''==
+
*"Skip repetitions" — the system will hide repeated segments (only one of them will be displayed)
  
−
You can '''Add mistake''':
+
*"Skip locked units" — the "frozen" units will not be displayed (for example, this setting is used if a client wants some important parts of the translated text to stay unchanged).
  
−
[[File:Evaluation report-Add mistake.png|Add mistake]]
+
*"Skip units with match >=" — units with matches greater than or equal to a specified number will not be displayed.
−
 
−
And describe it:
 
  
−
[[File:Evaluation report-Describe mistake.png|Describe mistake]]
+
*"Evaluation sample word count limit" — this value is used to adjust how many segments for evaluation will be displayed.
−
 
−
You may also edit, delete mistake/comment:
 
  
−
[[File:Evaluation report-Delete mistake (comment).png|Delete mistake (comment)]]
+
[[File:Start automatic evaluation settings final.png|border|500px]]
−
 
−
Or add another mistake — '''Add mistake''':
 
  
−
[[File:Evaluation report-add another mistake.png|Add another mistake]]
+
Adjust the settings and click "Start evaluation".
−
 
−
'''View in comparison''' — this link redirects you on the page with <u>[[Comparison report|Comparison report]]</u>.
 
  
−
When the mistakes classification is done, the project evaluator has to press '''Complete evaluation => Complete''':
+
=='''Start evaluation (manual word count)'''==
  
−
[[File:Evaluation report-Complete evaluation.png|Complete evaluation]]
+
If the file was reviewed partially, you can use the evaluation with manual word count.
−
 
−
At the left side displayed different buttons and filters:
 
  
−
'''Complete evaluation''' — the button, that finishes evaluation process.
+
To do this, click '''"Start evaluation (manual word count)"''':
  
−
'''Evaluation report''' — evaluation report view.
+
[[File:Start manual evaluation.png|border|200px]]
  
−
'''Comparison report''' — comparison report view.
+
Enter the number of evaluated source words (total number of words in the reviewed part of the file):
  
−
'''Project details''' — basic information about the project.
+
[[File:Start manual evaluation settings 3.png|border|450px]]
  
−
'''Project files''' — original translation and amended translation.
+
Then click "Start evaluation" and the system will display all corrected segments of the document.
  
−
Option '''Markup display''' defines tags display:
+
=='''Mistakes'''==
  
−
• '''Full''' — tags have original length, so you can see the data within:
+
Click the "Add mistake" button within a needed segment to add a mistake:
  
−
[[File:Quick compare-2.png|Markup display: Full]]  
+
[[File:1. 91.png|border|700px]]
 
   
 
   
−
• '''Short''' — tags compressed and you see only their position in the text:  
+
Specify mistake type and severity, leave a comment if needed, and click "Submit":
  
−
[[File:Quick compare-3.png|Markup display: Short]]  
+
[[File:2. mistake.png|border|290px]]
 
   
 
   
−
• '''None''' — tags are totally hidden, so they will not distract you:
+
You can edit, delete mistakes or comments, and add mistakes by clicking the corresponding buttons:
  
−
[[File:Quick compare-4.png|Markup display: None]]  
+
[[File:3. mistal.png|border|700px]]
 
   
 
   
−
'''Units display''' shows you all text segments — '''All units''' or nothing but amended — '''Corrected units only''':
+
*'''View in comparison''' — this link redirects you to the page with the '''<u>[[Comparison report|comparison report]]</u>''':
 +
 
 +
[[File:View in comparison.jpg|border|700px]]
 +
 
 +
When all the mistakes are added and classified, click "Complete evaluation", write an evaluation summary, and click the "Complete" button. The translator will get a notification.
 +
 
 +
[[File:1 complete evaluation.png|border|530px]]
 +
 
 +
::<span style="color:orange"> '''Note:'''</span> If you press '''"Complete"''' and no mistakes are added to the report, the system will warn you:
 +
 
 +
[[file:Evaluation no mistake are added.jpg|border|400px]]
  
−
[[File:Quick compare-4.png|"All units" shows you all text segments]]
+
==='''Markup display'''===
  
−
[[File:Quick compare-5.png|"Corrected units only" shows you only amended segments]]
+
Markup display settings allow you to choose how tags will be displayed:  
−
 
−
Also, you may find here '''Evaluation details''' and '''Comparison details''', such as:
 
  
−
[[File:Evaluation report-Evaluation details and Comparison details.png|Evaluation details and Comparison details]]
+
*"Full" — tags have original length, so you can see the data within:
  
−
'''Units''' —the quantity of segments.
+
[[File:1 full.png|border|140px]] [[File:1.png|border|650px]]
  
−
'''Source words''' — the quantity of words in the source.
+
*"Short" — the contents of the tags are not displayed and you see only their position in the text:
  
−
'''Total mistakes''' — total quantity of errors.
+
[[File:2 short.png|border|140px]] [[File:2.png|border|350px]]
 +
 +
*"None" — tags are not displayed:
  
−
'''Quality score''' — the complex index of performing translation, that depends on total quantity of words, specialization, severity of mistakes, etc.
+
[[File:3 none.png|border|140px]] [[File:3.png|border|270px]]
  
−
'''Quality mark''' — the evaluation based on quality score.
+
==='''Units display'''===
 +
 +
*"All" - units with and without mistakes are displayed:
  
−
'''Total units''' — total quantity of segments.
+
*"With mistakes" - only units with mistakes are displayed:
  
−
'''Corrected units''' — the quantity of segments with amendments.
+
*"Last commented by evaluator" — only units with the last comment left by the evaluator are displayed.
  
−
'''Total source words''' — full quantity of words in the source.
+
*"Last commented by translator" — only units with the last comment left by the translater are displayed.
  
−
'''Source words in corrected units''' — the quantity of source words in amended segments.
+
*"Last commented by arbiter" — only units with the last comment left by the arbiter are displayed.
  
−
When mistakes classification is done, the project evaluator has to press '''Complete evaluation => Complete''',
+
[[File:Units display.png|border|170px]]
  
−
and the system will send the quality assessment report to the translator.
+
=='''Reevaluation and arbitration requests'''==
  
−
When translator received this report and looked through classification of each mistake, he may '''Complete project'''
+
When the evaluation is done, the translator can complete the project or request the reevaluation if they disagree with mistake severities:
  
−
(if agree with translator; the project will be completed) or '''Request reevaluation''' (if disagree).
+
[[File:Request reevaluation.png|border|200px]]
  
−
{|align="right"
+
If the translator requests the reevaluation, the evaluator will have to reply to all the translator's comments.
−
|-valign="bottom"
 
−
|[[File:Evaluation report-Request arbitration.png|Request arbitration]]
 
−
|[[File:Evaluation report-Request reevaluation.png|Request reevaluation]]
 
−
|}
 
  
−
The project will be sent to the evaluator, who will review translator’s comments.  
+
::<span style="color:orange">'''Note:'''</span> Unless the number of [https://wiki.tqauditor.com/wiki/Evaluation_settings maximum evaluation attempts] has been adjusted, the translator can request the reevaluation for 2 times.
  
−
If they are convincing, the evaluator may change mistake severity. Translator will receive the reevaluated project.
+
If there is no agreement between the translator and evaluator, the translator can request the arbitration:
  
−
The translator can send this project for reevaluation one more time.  
+
[[File:Arb.png|border|200px]]
  
−
If an agreement between the translator and evaluator wasn’t reached, translator can send the project to the arbiter
+
The arbiter provides a final score that cannot be disputed and completes the project. Once the arbitration is completed, all the project participants will receive an email notification.
  
−
by pressing '''Request arbitration''' (it appears instead of '''Request reevaluation''').
+
[[File:Redirect.jpg|40px|link=Quality evaluation]] Back to the table of contents.

Latest revision as of 17:52, 18 February 2022

General information

After the evaluator uploaded files, they can start the evaluation.

The evaluator can start the evaluation whether with automatic word count or enter it manually when starting the process:

Evaluations.png

For more info on both methods, please check the relevant sections below.

Automatic vs. manual word count

Automatic:

1. Used for fully reviewed files.

2. The "Evaluation sample word count limit" is used to adjust how many segments for evaluation will be displayed.

3. The system will display only corrected segments (selected randomly) with the total word count specified as "Evaluation sample word count limit".

For example, if 1000 was specified as "Evaluation sample word count limit", the system will display around 100 segments with around 1000 words in total.

Please note that the number of segments varies depending on the size of segments.
Note: If the evaluator specifies 1000 as "Evaluation sample word count limit" while there are only 500 words in all corrected segments (let's say, there are 900 words in the file), the system will still display corrected segments with around 500 words in total. It means that 1000 can be safely used as "Evaluation sample word count limit" even if the real total word count is lower.

4. When calculating the score, the "Total source words" from the "Evaluation details" section (not the Total source words of a file) is used:

TSW.png

For example, if the evaluation report includes corrected segments with around 1000 words and the total source words is 1757, 1757 will be used in the formula.

Manual:

1. Used for partially reviewed files (in order not to split the file into parts and import only the reviewed part).

2. The "Evaluated source words" should reflect the total number of words in the reviewed part of the file.

For example, a reviewer reviewed only 1500 words in a 5000-word file. Then they should specify 1500 as "Evaluated source words" and the system will not take the remaining 3500 words into account.

3. The system will display all the corrected segments. So, if the reviewed part of the file is large, the evaluator will have to evaluate way more segments.

4. When calculating the score, the "Total source words" is used. In this case, "Evaluated source words" = "Total source words".

Start evaluation (automatic word count)

If you select this option, the system will display randomly selected segments containing only corrected units for evaluation:

Start evaluation automatic.png

Then you may configure the evaluation process:

  • "Skip repetitions" — the system will hide repeated segments (only one of them will be displayed)
  • "Skip locked units" — the "frozen" units will not be displayed (for example, this setting is used if a client wants some important parts of the translated text to stay unchanged).
  • "Skip units with match >=" — units with matches greater than or equal to a specified number will not be displayed.
  • "Evaluation sample word count limit" — this value is used to adjust how many segments for evaluation will be displayed.

Start automatic evaluation settings final.png

Adjust the settings and click "Start evaluation".

Start evaluation (manual word count)

If the file was reviewed partially, you can use the evaluation with manual word count.

To do this, click "Start evaluation (manual word count)":

Start manual evaluation.png

Enter the number of evaluated source words (total number of words in the reviewed part of the file):

Start manual evaluation settings 3.png

Then click "Start evaluation" and the system will display all corrected segments of the document.

Mistakes

Click the "Add mistake" button within a needed segment to add a mistake:

1. 91.png

Specify mistake type and severity, leave a comment if needed, and click "Submit":

2. mistake.png

You can edit, delete mistakes or comments, and add mistakes by clicking the corresponding buttons:

3. mistal.png

  • View in comparison — this link redirects you to the page with the comparison report:

View in comparison.jpg

When all the mistakes are added and classified, click "Complete evaluation", write an evaluation summary, and click the "Complete" button. The translator will get a notification.

1 complete evaluation.png

Note: If you press "Complete" and no mistakes are added to the report, the system will warn you:

Evaluation no mistake are added.jpg

Markup display

Markup display settings allow you to choose how tags will be displayed:

  • "Full" — tags have original length, so you can see the data within:

1 full.png 1.png

  • "Short" — the contents of the tags are not displayed and you see only their position in the text:

2 short.png 2.png

  • "None" — tags are not displayed:

3 none.png 3.png

Units display

  • "All" - units with and without mistakes are displayed:
  • "With mistakes" - only units with mistakes are displayed:
  • "Last commented by evaluator" — only units with the last comment left by the evaluator are displayed.
  • "Last commented by translator" — only units with the last comment left by the translater are displayed.
  • "Last commented by arbiter" — only units with the last comment left by the arbiter are displayed.

Units display.png

Reevaluation and arbitration requests

When the evaluation is done, the translator can complete the project or request the reevaluation if they disagree with mistake severities:

Request reevaluation.png

If the translator requests the reevaluation, the evaluator will have to reply to all the translator's comments.

Note: Unless the number of maximum evaluation attempts has been adjusted, the translator can request the reevaluation for 2 times.

If there is no agreement between the translator and evaluator, the translator can request the arbitration:

Arb.png

The arbiter provides a final score that cannot be disputed and completes the project. Once the arbitration is completed, all the project participants will receive an email notification.

Redirect.jpg Back to the table of contents.

Retrieved from "https://wiki.tqauditor.com/index.php?title=Evaluation_report&oldid=4049"