You don't know how you got it?
If you outsource your backlinking, your provider is careless
If you're backlinking yourself, well then you didn't filter out the list you either bought/leeched/scraped.
You might see negative results because of 'bad' backlinks, but in the long run it shouldn't hurt you and many tests have been conducted with intentionally linking to 'bad' sites that all end up in an increase in rank.
The increase, I believe, is an algorithm check to prevent a competitor from spamming your link to unscrupulous sites with the hope of hurting your ranking through sabotage. This might change eventually, but right now, I'd say build good backlinks and relax.