The simple answer (in simpler words) to why you shouldn't have the backlinks from pharma sites is you
should not have those backlinks from pharma sites. Another point is, pharma backlinks are considered 'sketchy', 'bad neighborhood', 'spam', etc. Just like p0rn, PDLs, or g@mbling sites are to be avoided, the pharma backlinks too should be avoided.
And if your site is not in those niches mentioned above, those kind of backlinks would create not only unrelevant backlinks concentration, but also a huge spammy and low-trust backlinks concentration on your backlink profile. Of course, I don't know what the result would be to a site that's related to those niches AND also has those spamy backlinks! I never thought of that angle... lol
You'd be doing something to your site which is called pharma hack. You would be purposefully doing negative SEO to your own site!!! Google, the SE that matters the most, will never 'like' your site any more! lol
Very risky and dangerous and negative, if you ask me.
