Loss functions

Binary classification losses

class radbm.losses.binary_classification.BCELoss(log2_lambda=-1.0)

Similar to torch.nn.BCELoss, but where the weighting is class-based instead of data point-based. E.g. If y in {0,1}^n are classes and p in [0, 1]^n are the model estimated probabilities then torch.nn.BCELoss(weight)(p, y) = BCELoss(log2_lambda)(log(p)[y], log(1-p)[~y]) where weight = (~y)*w0 + y*w1, with w1 = n*lamda/n1 and w0 = (n-n1*w1)/n0. n = n1 + n0 and n1, n0 are the number of positive and negative samples respectively. This formulation disentangle the concepts of oversampling and class weighting. i.e., when trying with different oversampling probability, we don’t need to update log2_lambda to compensate the bias.

Parameters:log2_lambda (float (in [-inf, 0]) or tuple (of size 4)) – It controls the weighting of true positive. over the true negatives.
__call__(tp_log_probs, tn_log_probs, step=0)
Parameters:
  • tp_log_probs (torch.tensor (dtype: float)) – Estimators of the log probability of a true positive. E.g. \(log(f_{\theta}(x))\) where \(f_{\theta}\) is the model and x is a positive data point.
  • tn_log_probs (torch.tensor (dtype: float)) – Estimators of the log probability of a true negative. E.g. \(log(1-f_{\theta}(x))\) where \(f_{\theta}\) is the model and x is a negative data point.
class radbm.losses.binary_classification.FbetaLoss(log2_beta, prob_y1, naive=False, estimator_sharing=True)

The F-beta score loss and its variants. This loss is an estimator of the log F-beta score of the model.

Parameters:
  • log2_beta (float or tuple (positive)) – The parameter that control the importance of the recall over the precision. If a tuple of four float (n0, n1, log_beta0, log_beta1), __call__ takes an additional parameter n which determines the log_beta linearly between log_beta0 and log_beta1.
  • prob_y1 (float (in [0, 1])) – The marginal probability that the class is one
  • naive (bool (optional)) – If True, the log(recall) of the model is estimated with log(mean(recall_hats)). If False mean(log(recall_hats)) is used. (default: False)
  • estimator_sharing (bool (optional)) – If True, the log probability of true positive estimators are used twice to compute the loss. If False, they are split in two and each halve is used only once. (default: True)
__call__(tp_log_probs, fp_log_probs, step=0)
Parameters:
  • tp_log_probs (torch.tensor (dtype: float)) – Estimators of the log probability of a true positive. E.g. \(log(f_{\theta}(x))\) where \(f_{\theta}\) is the model and x is a positive data point.
  • fp_log_probs (torch.tensor (dtype: float)) – Estimators of the log probability of a false positive. E.g. \(log(f_{\theta}(x))\) where \(f_{\theta}\) is the model and x is a negative data point.
  • n (float (positive) (optional)) – Controls the ramping of the log_beta. (default 0)

Binary vectors matching losses

class radbm.losses.matching.FbetaMultiBernoulliMatchingLoss(log_match, log2_beta, prob_y1, naive=False, estimator_sharing=True, reg=<radbm.utils.torch.regularization.HuberLoss object>, reg_alpha=0.0)

Callable, see MultiBernoulliMatchingLoss for documentation of how to call this.

Parameters:
  • log_match (function) – A matching function which takes logits and output log probabilities. E.g. HammingMatch(dist=0).
  • log2_beta (float) – See FbetaLoss.
  • prob_y1 (float) – See FbetaLoss.
  • naive (bool) – See FbetaLoss
  • estimator_sharing (bool) – See FbetaLoss
  • reg (function (optional)) – The regularization over the logits to use. (default HuberLoss(1, 9))
  • reg_alpha (float (optional)) – The factor to multiply the regularization with before adding it to the loss. (default 0., i.e. deactivated regularization)
__call__(queries_logits, documents_logits, r, step=None)

Call self as a function.

class radbm.losses.matching.BCEMultiBernoulliMatchingLoss(log_match, log2_lambda=-1.0, reg=<radbm.utils.torch.regularization.HuberLoss object>, reg_alpha=0.0)

Callable, see MultiBernoulliMatchingLoss for documentation of how to call this.

Parameters:
  • log_match (function) – A matching function which takes logits and output log probabilities. E.g. HammingMatch(dist=0).
  • log2_lambda (float or tuple (optional)) – See radbm.losses.BCELoss. (default: -1., i.e. usual BCE)
  • reg (function (optional)) – The regularization over the logits to use. (default HuberLoss(1, 9))
  • reg_alpha (float (optional)) – The factor to multiply the regularization with before adding it to the loss. (default 0., i.e. deactivated regularization)
__call__(queries_logits, documents_logits, r, step=None)

Call self as a function.

class radbm.losses.matching.MIHashMatchingLoss(nbits, match_prob)

MIHashMatchingLoss as in MIHash: Online Hashing with Mutual Information.

Parameters:
  • nbits (int) – The number of bits in the codes.
  • match_prob (float (in [0,1])) – The probability that there is a match given a random query and a random document.
forward(queries_logits, documents_logits, r, step=0)
Parameters:
  • q (torch.Tensor) – A batch of queries.
  • d (torch.Tensor) – A batch of documents.
  • r (torch.Tensor (dtype: torch.bool, ndim: 1 or 2)) – A matrix (block mode) (2D tensor) with r[i,j] indicating if q[i] match with d[j] or a vector (1D tensor) with r[i] indicating if q[i] match with d[i]
Returns:

loss – The loss (negative mutual information) of the current batch.
Return type:

torch.Tensor (size 1)
class radbm.losses.matching.HashNetMatchingLoss(log2_lambda=-1, alpha=1, beta0=1, stage_length=10000, reg=<radbm.utils.torch.regularization.HuberLoss object>, reg_alpha=0)

As in HashNet: Deep Learning to Hash by Continuation.

Parameters:
  • log2_lambda (float (in ]-infty,0])) – Used for class balancing.
  • alpha (float (optional)) – The HashNet’s alpha used in the adaptative sigmoid, there is no recommendation in the article. Maybe 6/nbits is a good place to start. (default: 1)
  • beta0 (float (optional)) – The initial HashNet’s beta. Beta increase linearly with the stage, i.e. \(\mathrc{tanh}(\mathrc{stage} \beta_0 x)\). (default: 1)
  • stage_length (int (optional)) – The number of steps before increasing the stage. (default: 10000)
  • reg (function (optional)) – The regularization over the logits to use. (default HuberLoss(1, 9))
  • reg_alpha (float (optional)) – The factor to multiply the regularization with before adding it to the loss. (default 0., i.e. deactivated regularization)
__call__(queries_logits, documents_logits, r, step=0)
Parameters:
  • queries_logits (torch.Tensor (ndim: 2)) – A batch of queries.
  • documents_logits (torch.Tensor (ndim: 2)) – A batch of documents.
  • r (torch.Tensor (dtype: torch.bool, ndim: 1 or 2)) – A matrix (block mode) (2D tensor) with r[i,j] indicating if q[i] match with d[j] or a vector (1D tensor) with r[i] indicating if q[i] match with d[i]
  • stage (float (optional)) – The stage for computing the tanh(stage*beta*x). (default=1)
Returns:

loss – The loss of HashNet.
Return type:

torch.Tensor (size 1)