If they aren't popular with anyone, how would they be able to continuously being growing in cause and nature? By the very being of "unpopularity" there would be more resistance. If it's a position where people don't agree, they should easily be able to state it so we can get it resolved, or stop it before it comes worse. Instead, you have one group of people shouting it down vigorously, and another group passively dismissing, not wanting to get involved, or defending it.
I don't feel it's a major issue in comparison to say, voting on healthcare or government assistance plans. Others would take priority. But I do feel it's important enough to not let it go unchecked to avoid escalation or amplification of our current course. People less involved in political discourse would hold it in higher regard, dictating their decision making though.