validators.reward.test_task_validator#

Module Contents#

Classes#

TaskValidatorTestCase

This class contains unit tests for the TaskValidator class.
class validators.reward.test_task_validator.TaskValidatorTestCase(methodName='runTest')#

Bases: unittest.TestCase

This class contains unit tests for the TaskValidator class.

The tests cover different scenarios for the reward method of the TaskValidator class. The reward method is expected to return a reward based on the task name and the completion text.

setUp()#

Hook method for setting up the test fixture before exercising it.

test_augment_with_answer_keyword()#

Test if the reward method returns 0 when the task “name” starts with ‘augment’ (summarization) and the completion contains the ‘Answer:’ keyword.

test_followup_with_answer_keyword()#

Test if the reward method returns 0 when the task “name” starts with ‘followup’ (question generation) and the completion contains the ‘Answer:’ keyword.

test_augment_with_question_keyword()#

Test if the reward method returns 0 when the task “name” starts with ‘augment’ (summarization) and the completion contains the ‘Question:’ keyword.

test_answer_with_question_keyword()#

Test if the reward method returns 0 when the task “name” is ‘answer’ (answer generation) and the completion contains the ‘Question:’ keyword.

test_followup_and_answer_with_summary_keyword()#

Test if the reward method returns 0 when the task “name” is different from “augment” (summarization) and the completion contains the ‘Summary:’ keyword.

test_reward_valid_followup()#

Test if the reward method returns 1 when the task “name” starts with ‘followup’ (question generation) and the completion contains a question

test_reward_valid_answer()#

Test if the reward method returns 1 when the task “name” is ‘answer’ (answer generation) and the completion contains an answer

test_reward_valid_augment()#

Test if the reward method returns 1 when the task “name” is ‘augment’ (summarization) and the completion contains the a summary.

test_reward_valid_other()#

Test if the reward method returns 1 when the task “name” is different from “augment”, “followup”, and “answer” and the completion does not contain the ‘Summary:’, ‘Answer:’, and ‘Question:’ keywords.