How do you know it's a competitor? Maybe average CL Users hate your ads. There's a lot of bullshit when it comes to CL, meaning that stupid people believe things, and they repeat it often enough so that eventually people that aren't stupid accept it as reality. Which ultimately is stupid, if you think about it. After all, who is stupider? The stupid person that decides that something that's wrong is right, or the person that believes the stupid person?
So, I'd take a few steps backwards and re-think the whole equation. If it really IS a competitor, that means they either monitor the section constantly, or they have auto-flagging software running.
I have a method that may or may not work for protecting an ad vs. flagging, but I will not share it for obvious reasons. However, the best defense is a good offense, and so if you were to find a method to flag the competitor's ads, you might teach them that an eye for an eye means that everyone is blind. But you have to take their eyes in order to make this point. Bad people never respect diplomatic words or threats. All they understand is aggressive action, so taking a piece of rebar and violently fucking them in the ass with it 1st is a necessary stage in the peace process. Otherwise they have no reason to change what they are doing.
After all, it's working, duh.
Another thing to do is set up a timed macro to call their business line every "x" minutes, timed such that it leaves a voice mail of "y" minutes, with the intention of annoying the fuck out of them by the phone calls, and filling up their voice mail so they have to delete all the messages, and have no room for legit phone calls.
Also, negative reviews on the business, optimized by business name AND phone number. On craigslist, because some people search the number to see if there are any negative reports connected to it, but also the other local platforms such as Yelp, G+, Merchant Circle, Kudzu, etc...
The real question is where is the most efficient use of your time, and the 2nd question is, Why are you dependent on a half-assed advertising medium like craigslist, as the CL customer is a cheap, stingy and stupid tightwad that will wants as much as they can get and for nothing, and there's a good chance that the time & energy you spend screwing around with them & chasing their meager dollars would be better spent developing a website and Local SEO.