In default SSH config, it’s most likely that if you connect an unknown SSH server, the server is able to enumerate all your public keys in your agent by keeping rejecting your public keys tried. This seems how the SSH protocol work. It doesn’t pose a security risk, but it does have some Privacy concerns - those public keys can identify who you are on the web. The fix? It’s simple: I think either of the following will work (I personally tried method #2):

  1. Append to the end of /etc/ssh/ssh_config or put at the beginning of ~/.ssh/config (as suggested by chrisfosterelli on HN1) the following:
# Ignore SSH keys unless specified in Host subsection
IdentitiesOnly yes
  1. Append to the end of /etc/ssh/ssh_config or ~/.ssh/config the following:
Host *
  IdentityFile /nosuchkey
  IdentitiesOnly yes

Reference: