check_rcptto: a plugin for qpsmtpd
Jan 4 update: Seems there is/was already a plugin for qpsmtpd called "check_rcptto" but from what I've been able to gather, its purpose was to reject addresses deemed invalid (no system account, no entry under ~alias, etc.), and somebody has since extended/rewritten that plugin to come up with a new one called "check_delivery".
This plugin is capable of doing that but it will take some extra work on your part. I have an incomplete example in the plugin itself that only checks for valid system users with a warning not to actually use it; the "tails you lose" configuration example might actually be more useful for no-database-required pseudo-greylisting. But don't use that one either.