since both you, and the #1 competitors got booted out of page 1 it can't be anything technical IMO (server issue, for example).
It's most likely something that you and your #1 competitors are lacking that the guys who outranked you don't. So, check your page, and your #1 competitor's page against the pages that took your spots, and see what they have that you don't, or what they're doing better than you are.
Unfortunately, without knowing your niche, site, and target keyword we can't pinpoint the problem exactly, so my blind guess is that it's one of the following:
- recent google update (just finished rolling out last week I believe, so it's possible you've been affected by that thing)
- poor content, or poorly optimized content (not enough / the right LSI and / or SEO entities)
- poor / no internal linking
- also regarding content it could be outdated content (that's why you need to compare your content vs the content of those who outranked you)
- not enough trust and / or authority for the page that got affected, or for the entire site (again, compare these things against the same things of the guys who outranked you and see if you can find something).
If I were to blindly-guess even harder I'd say it's either outdated content, or lack of trust / authority (either at page, or site level), it's almost always one of these 2, so I'd start with these...