Sorry for the slow response. You're spot on with all those observations, and many of them we were planning to do anyway. The truth is that there are still some higher priority issues that our small team needs to address first.
A couple notes:
Preview images are stored in the DB mostly because of the Windows legacy. They certainly can be moved to just regular files if deemed necessary for stability and playing nice with Time Machine.
The tmp directory is inside the Library because currently you can shut down NeatReceipts will it process a bunch of receipts, copy the library to another machine with NR installed, open the library, and it will pick up where it left off. If the temp files lived in ~/Library/Caches this wouldn't be possible. However, I'm not sure if this feature is actually that useful in practice so the tmp files could change their location in the future.
Also, the temp files are currently cleaned on a timer, which by default fires every 5 mins and cleans up to 10 files. We can tweak these params and perhaps also add a purge on startup.
And finally, in response to your "true dirty secret" :)
The behavior you observed is actually a side effect of one of Apple's built-in save operation methods. In truth, we did not discover what it was actually doing until it was too late. We tracked down the issue after reports of libraries being "slow to close" it is one of our top priority bugs. It will be fixed next update.
Except for the last issue, I can't say when the rest will be addressed. However, they are all important, valid issues that are present in our product backlog. Thanks for all your input!