We studied the role of attention in aversive risky choices where all outcomes were unpleasant. We used two eye-tracking experiments in which participants made binary choices between two lotteries in two conditions: (a) a gain condition where outcomes for lotteries were weakly positive, and (b) a loss condition where outcomes were weakly negative. Contrary to the predictions of the standard aDDM, we found that attentional choice biases in the loss condition were identical to those found in the gain condition, suggesting that attention nudges choices towards the attended option even in losses. To explain these results, we propose a variation of the Attentional Drift-Diffusion-Model (called the Hybrid aDDM) that incorporates (a) both a value-dependent and a value-independent effect of attention on the choice process and (b) reference-dependent value signals. We show that the observed attentional choice biases and other behavioral signatures in the loss condition can only be explained by the Hybrid aDDM with a reference-point rule that sets the reference-point at or below the minimum possible outcome in a given context.