Your question is a little vague, but if you mean excluding robots from your html website, then exclude them in the robots.txt file. You need to know the user-agent and put it in the file with the directory subtree you want the robot to ignore. This can be / for everything. Malicious bots ignore the robots.txt, and sometimes the only way to stop them is to block their IP address or something like that. The proxied ones can be extremely hard to stop without some custom code.