Result Validator

The result validator decides whether each individual result returned by a client is valid, decides whether a client has passed a spot-check, and selects a canonical result from the set of all results returned for a particular work unit. The canonical result is the result that is accepted as the final result for a work unit; all other results for that work unit are deleted. Implementing the result validator is optional because the framework provides a default validator. The default validator marks all results as valid, marks all spot-checks as passed, and selects the first result received as the canonical result. If a custom result validator is implemented, the project developers can also choose to use the default behavior for one or more functions of the validator. For example, if a project only needs the spot-checking functionality of the result validator, the validator can be made to always mark normal results valid and to select the first result returned as the canonical result. The use of a default behavior for a particular function will be referred to as stubbing that function. If a custom result validator is implemented, all three functions must be implemented, but any number of them may be stubbed. The figure below shows an overview of the result validator's control flow:

Result Validator Control Flow

The first step taken by the result validator is to execute the server.getResultForValidation RPC on the project server. This RPC does not take any parameters. This RPC will return an exception if there are no results that need to be validated. If this is the case, the validator should sleep for some amount of time and then execute the RPC again. If there is a result to validate, the server returns a vector in one of three formats. The first element (index 0) of the vector will always be an integer, which indicates the type validation that the validator needs to perform. A value of 0 indicates that the vector contains a single result to be validated. In this case, the vector will contain two other elements. The element at index 1 will be a string representing the ID of the result to be validated. The element at index 2 will be the data from that result. The result validator should decide whether the result returned by the project server is valid or not. If the result is valid, or this function is a stub, the result validator should execute the server.markResultValid RPC on the project server. Otherwise, the validator should execute the server.markResultInvalid RPC. Both of these RPCs take a single parameter, the ID of the result to mark valid or invalid. If the result was successfully marked valid or invalid, the return value of the RPC will be true. If the return value is false, it indicates that the given result ID could not be found.

If the first element of the vector is 1, it signifies that the validator needs to select a canonical result. In this case, the element at index 1 indicates the number of results that are contained in the vector. Each result is comprised of two separate elements in the vector. The first element is the result ID, and the second is the data for that result. For example, if the element at index 1 is the integer 2, it means that there are four more elements in the vector. The element at index 2 is the result ID of the first result, and the element at index 3 is the data for the first result. The element at index 4 is the result ID of the second result, and the element at index 5 is the data for the second result. The composition of the vector is presented as a table in the XML-RPC Interface Specification; the table format might be easier to understand. After all of the results have been examined, the validator should select one to be the canonical result. It should then call the server.selectCanonicalResult RPC on the project server. This RPC takes one parameter, the ID of the canonical result selected by the validator. It returns a boolean indicating whether the RPC was successful. A value of true indicates that the canonical result was successfully selected, and a value of false means that the given result ID could not be found.

If the first element of the vector is 2, the result that is being validated is a spot-check. There will be three other elements in the vector. The element at index 1 will be the ID of the result. The element at index 2 will be the spot-check result data returned by the client. The element at index 3 will be the accepted spot-check result data that was computed by the server. The validator should compare these two results to determine whether the result submitted by the client is close enough to the accepted result. If the validator decides that the client passed the spot-check, it should execute the the server.markResultValid RPC, passing the result ID as the parameter. If the client did not pass the spot-check, the validator should instead execute the server.markResultInvalid RPC.

After each result validation, the validator should sleep for some amount of time, and then execute the server.getResultForValidation RPC again.