You should be able to delete a contact from the contact file, and not have to go back to the mail list of bounces, check their name and then hit remove. It's too slow going back and forth between screens and we don't have time to waste.
Thanks for posting this feedback! One thing to note, you can unsubscribe a contact from within the contact details by editing the contacts permission status. Using the remove emails button on the bounce page takes a similar action but just removes the email address, leaving the contact in the account.
This would save you from returning to the bounce page to remove the contact.