If you're truly deleting the bounce from your contacts, they won't be on lists or your account. Not unsubscribing them means that if you end up importing the contact again, they'll be re-added as an active contact and will likely continue to bounce.
