I can't say I recall polygamy as a hill that was particularly fought over, either as one where the conservative side claimed that we would eventually slide there and this would be the nightmare outcome, or one where the progressive side claimed that it would never happen and worries that we were on a slippery slope towards it are insane. (Single anecdotes of people saying this on either side in hindsight don't really count for much.) For starters, what is the argument against legal recognition of polygamy? It has been practically feasible in most Western countries for a long time now for all purposes but the "tax advantages and hospital visitation rights" one, and hasn't at all gone mainstream (presumably at least in part because a lot of people actually prefer monogamy).
Instead, the main claim that I've heard from the conservatives since Day 1, and that I do think the described reaction from the progressive side occurred to, is that we would eventually wind up legalised bestiality and/or pederasty.
"It will never happen, and when it does, you will have deserved it".
This is yet another thing I have noticed; the vaguely progressive types who maintain doggedly "Nobody ever said that", "Okay somebody said it but it was only one person somewhere obscure", "Okay yeah but it's only a few kids on college campuses", "Fine, it's the national media but these are just opinion columns and some guy with a Youtube channel", "Why shouldn't it be legal?", and finally "Why are you objecting to this perfectly normal, uncontroversial thing, you bigot?"
we would eventually wind up legalised bestiality and/or pederasty. I still think the former is unlikely and the latter is extremely unlikely
Because it's not like there have been activists trying to get ages of consent lowered as much as possible, most infamously the NAMBLA types piggy-backing off gay rights activists back in the 70s and early 80s, some of which activists were happy to recount tales of when they or a friend of theirs was nine and having sex with an adult man and it did them no harm at all.
It's always "extremely unlikely" until it starts happening. Myself, I often was embarrassed by the types who claimed "legalise gay marriage and what's next? incest, bestiality?" because I thought that was stupid and there were better arguments if you were anti-this.
Now I'm not so contemptuous. Yes, at present it's still a dumb argument. Zoophilia is also a kink, however, and a lot of people are very vocal about how kink should be protected and not swept away with respectability politics. The more people anthropomorphise animals, the more there is talk (in different contexts) of "non-human animals" and pushes to give them rights on a par with humans, the nearer we get to "why shouldn't it be considered okay if I love my non-human companion like you love your human companion?"