One key royal 'can never' forgive Prince Harry