Fork me on GitHub

Proctor - A/B Testing Framework by Indeed

Test Rules

A test may contain two types of rules: eligibility rules and allocation rules.

Rule Format

All rules need to be written in Unified Expression Language, and return a boolean value. Test rules may be set to null or literal values. Test rules that are set to null, an empty rule "${}", "${true}", or "${TRUE}" will evaluate to true. Test rules that are set to "${false}" or "${FALSE}"will evaluate to false.

Variables

Test rules can use any variables in the specification's providedContext, test constants, or special constants. Special constants set in a test defintion can modify a test rule in the test-matrix. For example, if __COUNTRIES is set in specialConstants, the test-matrix will add proctor:contains(__COUNTRIES, country) to the rule for that test. The following two JSON files show the change made from the test definition to the test-matrix.

Example Test Definition

Example Test Matrix With Modified Rule

Rules Using Variables

The following example shows a rule that indicates a test should only be applied when the language is english, the country is US, and the test constant variable id is greater than 1000.

Next is an example of a rule using special constants.

Namespace Function Libraries

Test rules have two namespace function libraries available to them by default:

Namespace Implementation Class Notes
fn org.apache.taglibs.standard.functions.Functions JSP EL functions from the standard tag library
proctor com.indeed.proctor.common.ProctorRuleFunctions Proctor-specific functions for rules

Rules Containing Namespace Functions

Test rules may take advantage of the two namespace function libraries available to them. In the following example, the eligibility rule uses the "contains" function from the "proctor" namespace.

Extending Default Namespace Function Libraries

In order to add to the default namespace function libraries, Proctor provides users with a static function that generates a LibraryFunctionMapperBuilder that contains the default fn and proctor namespaces. Simply add the namespace and class to this builder and build. Building will return a FunctionMapper containing the default namespaces as well as any that were added. See the following sample code as an example.

final JsonProctorLoaderFactory factory = new JsonProctorLoaderFactory();
// Loads the specification from the classpath resource
factory.setSpecificationResource("classpath:/org/your/company/app/ExampleGroups.json");
// Loads the test matrix from a file
factory.setFilePath("/var/local/proctor/test-matrix.json");

// A custom FunctionMapper without the default namespaces fn and proctor
final FunctionMapper myFunctionMapper = new LibraryFunctionMapperBuilder()
                                            .add("namespace1", Class1.class)
                                            .add("namespace2", Class2.class)
                                        .build();

// A custom FunctionMapper WITH the default namespaces fn and proctor
final FunctionMapper extendedFunctionMapper = RuleEvaluator.defaultFunctionMapperBuilder()
                                              .add("namespace1", Class1.class)
                                              .add("Namespace2", Class2.class)
                                          .build();

factory.setFunctionMapper(myFunctionMapper); // or pass extendedFunctionMapper

final AbstractJsonProctorLoader loader = factory.getLoader();