C++ Templates

Throughout this section of the documentation we will refer to the c++ directory inside the templates directory as $CTBASE. The source code for the C++ templates is located in $CTBASE/src. There should be five subdirectories in $CTBASE/src: common, lib, sci_app, validator, and work_gen. The lib directory contains the source code for the necessary XML-RPC libraries, which will be built by the build script.

To create a work unit generator component using the templates it is necessary to edit the WorkUnitGenerator class, contained in $CTBASE/src/work_gen/generator.cpp and $CTBASE/src/work_gen/generator.hpp. The parts of the template files that will or might need to be changed are denoted by a TODO comment. There are three methods that need to be modified in the WorkUnitGenerator class: recoverState, generateWorkUnit, and generateAndSend. If the work unit generator were ever restarted, it would need some way to reinitialize its state information so that it could resume generating work units from the appropriate place in the science data. The recoverState method retrieves the last work unit that was generated from the server and uses that work unit to determine what work unit should be generated next. This method retrieves the last work unit using the provided getLastWorkUnit method of the WorkGeneratorClient class. If the project server throws an exception while retrieving the last work unit, it means that no work unit was ever generated, so the work unit generator should begin generating the first work unit.

The generateWorkUnit method in the WorkUnitGenerator class generates the next work unit and returns a byte vector representing the new work unit. It should also update some state information in the WorkUnitGenerator class to indicate the next work unit that should be generated.

The last WorkUnitGenerator method that needs to be edited is generateAndSend. This method generates a given number of work units by calling generateWorkUnit and then sends each work unit to the server by calling one of the overloaded WorkGeneratorClient::sendWorkUnit methods, which are declared in $CTBASE/src/work_gen/work_gen_client.hpp. All of the sendWorkUnit methods require the work unit byte vector as a parameter, but some allow extra parameters to be passed such as the work unit ID, the priority of the work unit, the point value of the work unit, and the amount of time before the work unit should expire. All of these variants are documented in both the WorkUnitGenerator::generateAndSend method and the WorkGeneratorClient class.

In addition to the three previously mentioned methods, project developers may wish to edit the work unit generator's main function, located in $CTBASE/src/work_gen/work_gen.cpp. The main periodically queries the project server to determine how many ingress work units it has. If the server has less than a certain number of work units, which we will refer to as the low-water mark, the main will invoke the WorkUnitGenerator::generateAndSend method to send more work units to the server. The main can be edited to change how often the server is queried, change the low-water mark, or the number of work units that are generated and send to the server when the low-water mark is reached.

The next component that should be modified is the science application, located in ScienceDataProcessor, located in $CTBASE/src/sci_app/sci_data_proc.cpp and $CTBASE/src/sci_app/sci_data_proc.hpp. There are two methods that need to be modified: run and scienceAlgorithm. The run method takes a byte vector representing the work unit to compute as its parameter. If the science application uses check-pointing, the first action that should be taken by the run method is to check for previously saved check-points by calling the getCheckpoint method of the ScienceApplicationClient class. This method will return a byte vector representing the check-point. The template source code demonstrates how to do this. If the returned check-point vector is of zero length, it means that no check-point was found. If a check-point was found, the run method should set some private member variables to indicate where in the work unit to resume processing. The next action taken by the run method is to initialize and start a low-priority thread, called the compute thread, in which the scienceAlgorithm method will execute. This is also demonstrated in the science application template. Any data needed by the science algorithm will have to be passed as a void* because that is the way the pthreads library handles parameter passing to threads. After starting the compute thread, the run method should join with the compute thread, causing the run method to block until the compute thread has terminated. The compute thread will return a void* representing the result for the work unit, which will then have to be cast to the appropriate type and converted to a byte vector.

The scienceAlgorithm method uses information passed by the run method to compute the result for a work unit. If the science application uses check-pointing, the scienceAlgorithm method should periodically save a check-point by calling the saveCheckpoint method of the scienceAlgorithm method must return the result as a void* to the run method, which will convert it to the appropriate type.

Implementing the result validator component is optional, but the template for this component can be found in $CTBASE/src/validator. The class that needs to be modified is ResultValidator, located in $CTBASE/src/validator/result_validator.cpp and $CTBASE/src/validator/result_validator.hpp. There are three methods in the ResultValidator class that must be implemented: validateSingleResult, selectCanonicalResult, and validateSpotCheck. The validateSingleResult method examines the data from a result and decides whether or not it is valid. This method should return true if the result is valid or false if the result is invalid. If necessary, it is possible to examine the work unit from which the result was computed using the ResultValidatorClient class; this is demonstrated in the template source. The selectCanonicalResult method examines all of the valid results for a work unit and determines which one of those results should be chosen to be the canonical result. This method should return the result ID of the result that was chosen. It is possible to examine the work unit from which the results were computed in the same way as was done when validating a single result. The validateSpotCheck method compares the spot-check result computed by a client to the accepted spot-check result that was computed by the project server to determine whether the client passed the spot-check. This method should return true if the client passed the spot-check or false if the client failed the spot-check. Again, it is possible to examine the spot-check work unit from which the spot-check result was computed.

Another optional template component is the assimilator. Although it is optional, most projects will find it necessary or convenient to use this component. The assimilator extracts valid results from the project database. The assimilator template is in $CTBASE/src/assimilator. There is only one method that needs to be modified: bool Assimilator::assimilateNextWorkUnitResultPair(Result& r) in the file $CTBASE/src/assimilator/assimilator.cpp. This method should assimilate the given result, whatever that means for your specific project. For example, you may wish to insert the result data into another database for later analysis or recombination with other results, or you may wish to send a message to another component based on the contents of the result. The work unit from which the result was computed is also available for assimilation by accessing the struct member r.workUnit. If your assimilateNextWorkUnitResultPair method returns true, the work unit that was just assimilated will be deleted from the project database. If your method returns false, the result will not be deleted from the project database. However, if your method returns false, your assimilator will keep receiving the same result every time it asks for the next result to be assimilated until that result is deleted from the database. Therefore, your method should only return false if an error occurred and the result needs to be re-assimilated.