phpbb forum profile

Hi all, Have a list of phpBB forums I have harvested and thought I would pass them onto you all :). Only thing is that I can't find something that will clean all the urls eg distinguish www.forum.com and www.forum.com/forum as duplicate urls, as about 10% of the forums in the list are like...