Don't use fast timeouts for fully established connections.

During the fast retry period, we want to have a fast ping timeout until we have
a fully working connection. However, the code still used fast timeouts during
the fast retry window even if the connection was fully established.