This is an alternative of #31877 to fix #31876 caused by #28808.

This issue was caused by a combination of several loose implementation.

* finding automatic inverse association of polymorphic without context (caused by #28808)
* returning `klass` even if `polymorphic?` (exists before #28808)
* loose verification by `valid_inverse_reflection?` (exists before #28808)

This makes `klass` raise if `polymorphic?` not to be misused.
This issue will not happen unless polymorphic `klass` is misused.

Fixes #31876.
Closes #31877.