About the loss function problem

[Pefect96]

In the function of

ProbVLM/src/losses.py

Line 81 in cb69f28

l1 = self.L_l1(mean, target)

, nn.L1Loss is inconsistent with Eq.3 in the paper, and Eq. 3 does not include the nn.L1Loss.

Waiting for your help! @udion @sgk98

[bravePinocchio]

Why isn't this line of code being used instead of the twenty-ninth line of code? Beta is power in the paper.

ProbVLM/src/losses.py

Lines 28 to 29 in cb69f28

	# resi = torch.pow(resi*one_over_alpha1, beta1).clamp(min=self.resi_min, max=self.resi_max)
	resi = (resi*one_over_alpha1*beta1).clamp(min=self.resi_min, max=self.resi_max)

[udion]

For stability during training, beta in power leads to unstable training hitting nans, so we used Taylor series approximation, explained in Appendix of the paper here: https://arxiv.org/pdf/2307.00398 (Eq. 16 in appendix A)