Package org.variantsync.boosting.eval.experiments

Class Evaluator

java.lang.Object
org.variantsync.boosting.eval.experiments.Evaluator
public class Evaluator extends Object
Class for evaluating the result of a feature trace run.

  • Constructor Summary

    Constructors
    Constructor
    Description
    Evaluator(org.variantsync.boosting.TraceBoosting traceBoosting)
    Constructor for the Evaluator class.

  • Method Summary

    Modifier and Type
    Method
    Description
    double[]
    compare(org.variantsync.boosting.datastructure.MainTree maintree, Map<String,org.variantsync.vevos.simulation.variability.pc.groundtruth.GroundTruth> productPC, Set<String> relevantFeatures, int strip)
    Compares the given MainTree with the ground truth and evaluates the boosted traces.
    double[]
    compareMappings(String groundTruth, String mapping)
    Compares two mappings and calculates the true positive, true negative, false positive, and false negative rates.
    static boolean
    evalExpression(String expression)
    Evaluates a logical expression containing only the 'AND' operator.
    boolean
    evalTruthLine(String mapping, int lineNumber)
    Checks for the mapping in a line of the truth table whehter it evaluates to true Truth table looks like this, for instance, line and binary represenation 0 00000 1 00001 2 00010 3 00011 4 ...
    boolean[]
    evalTruthTable(String mapping)
    Evaluates the truth table for a given mapping and returns the corresponding boolean array.
    static double
    f1score(double prec, double rec)
    Calculates the F1 score based on precision and recall values.
    static double
    precision(double tp, double fp)
    Calculates the precision of a model based on the true positives (tp) and false positives (fp).
    static double
    recall(double tp, double fn)
    Calculates the recall score for a binary classification model.
    static double[]
    scoresFunc(double tp, double fp, double fn)
    Calculates precision, recall, and F1 score based on true positives, false positives, and false negatives.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • Evaluator

      public Evaluator(org.variantsync.boosting.TraceBoosting traceBoosting)
      Constructor for the Evaluator class. Initializes the Evaluator with the given TraceBoosting object. Calculates the number of features in the TraceBoosting object and sets the table size accordingly. Creates a binary map with keys representing binary numbers up to the table size.
      Parameters:
      traceBoosting - The TraceBoosting object to initialize the Evaluator with.

  • Method Details

    • compare

      public double[] compare(org.variantsync.boosting.datastructure.MainTree maintree, Map<String,org.variantsync.vevos.simulation.variability.pc.groundtruth.GroundTruth> productPC, Set<String> relevantFeatures, int strip)
      Compares the given MainTree with the ground truth and evaluates the boosted traces.
      Parameters:
      maintree - The MainTree to compare with the ground truth
      productPC - A map containing product names as keys and corresponding GroundTruth objects as values
      relevantFeatures - A set of relevant features to consider during comparison
      strip - the number of path elements to cut off at the start of paths
      Returns:
      An array of doubles representing the evaluation results for the boosted traces
      Throws:
      IllegalArgumentException - if maintree is null or productPC is empty

    • compareMappings

      public double[] compareMappings(String groundTruth, String mapping)
      Compares two mappings and calculates the true positive, true negative, false positive, and false negative rates.
      Parameters:
      groundTruth - The ground truth mapping as a string
      mapping - The mapping to be compared against the ground truth
      Returns:
      An array of doubles containing the true positive rate, true negative rate, false positive rate, and false negative rate
      Throws:
      IllegalArgumentException - if the groundTruth and mapping strings are not of the same length

    • evalTruthTable

      public boolean[] evalTruthTable(String mapping)
      Evaluates the truth table for a given mapping and returns the corresponding boolean array. If the truth table for the given mapping has already been computed, it is retrieved from a cache. If not, the truth table is computed and stored in the cache for future use.
      Parameters:
      mapping - the mapping for which the truth table needs to be evaluated
      Returns:
      the boolean array representing the truth table for the given mapping

    • evalTruthLine

      public boolean evalTruthLine(String mapping, int lineNumber)
      Checks for the mapping in a line of the truth table whehter it evaluates to true Truth table looks like this, for instance, line and binary represenation 0 00000 1 00001 2 00010 3 00011 4 ... And the mapping may be string like this "!Root and FeatureA"

    • evalExpression

      public static boolean evalExpression(String expression)
      Evaluates a logical expression containing only the 'AND' operator.
      Parameters:
      expression - a String representing the logical expression to be evaluated
      Returns:
      true if all sub-expressions separated by the 'AND' operator evaluate to true, false otherwise
      Throws:
      IllegalArgumentException - if the input expression is null or empty

    • precision

      public static double precision(double tp, double fp)
      Calculates the precision of a model based on the true positives (tp) and false positives (fp). Precision is defined as the ratio of true positives to the sum of true positives and false positives.
      Parameters:
      tp - The number of true positive predictions.
      fp - The number of false positive predictions.
      Returns:
      The precision of the model as a double value.
      Throws:
      IllegalArgumentException - if tp or fp is negative.

    • recall

      public static double recall(double tp, double fn)
      Calculates the recall score for a binary classification model. Recall, also known as sensitivity, is the ratio of true positives to the sum of true positives and false negatives.
      Parameters:
      tp - The number of true positive predictions made by the model.
      fn - The number of false negative predictions made by the model.
      Returns:
      The recall score, a value between 0 and 1.
      Throws:
      IllegalArgumentException - if tp or fn is negative.

    • scoresFunc

      public static double[] scoresFunc(double tp, double fp, double fn)
      Calculates precision, recall, and F1 score based on true positives, false positives, and false negatives.
      Parameters:
      tp - The number of true positives.
      fp - The number of false positives.
      fn - The number of false negatives.
      Returns:
      An array of doubles containing precision, recall, and F1 score in that order.
      Throws:
      IllegalArgumentException - if any of the input values are negative.

    • f1score

      public static double f1score(double prec, double rec)
      Calculates the F1 score based on precision and recall values.
      Parameters:
      prec - The precision value, a double between 0 and 1.
      rec - The recall value, a double between 0 and 1.
      Returns:
      The F1 score, a double between 0 and 1.
      Throws:
      IllegalArgumentException - if prec or rec is not between 0 and 1.