Sorry for the limitation--there isn't currently a way to see who declined an event through the API. You can see all of the possible values for an event contact on this page. As you can see, it is all currently centered around who registered (or cancelled, abondoned, etc their registration). Getting a list of declines is something that I would love to see our API do, and I have forwarded your feedback onto the event team.
API Support Specialist