Hey Goel,
From the sound of it, I would assume that most of those links could be from low-quality and/or irrelevant domains. So, to your question, it is possible and something you can see somewhat frequently.
Other issues could be regarding content, on-page and/or penalties.