Let $\cal P$ denote the equivalence relation on an epigroup in which any of its classes consists precisely of those elements owning the same pseudo-inverse. The purpose of the paper is to characterize epigroups on which any congruence either contains $\cal P$ or its join with $\mathcal P$ is the identity relation on epigroups. As a special subclass of epigroups, completely 0-simple semigroups having the same property are also described.