Mahajan et al. and Yalniz et al. demonstrate the benefit of pre-training on large unlabeled image datasets using user tags as labels for weak-supervision. We take this approach further, considering whether the use of user tags in large labeled image datasets is beneficial.