There seems to be evidence that young children transmit the disease less, so I think it's not clear you'd have been the biggest risk factor. Nonetheless, not going seems like the right call.