I hate it when someone criticizes a boxer, and someone else replies with, "You've never boxed, so you have no right to criticize!". The idea that you need to have boxed to know whether someone is a good fighter or not is, quite frankly, stupid. You don't need to be a gourmet chef to know whether someone's cooking is good or bad.
Well, if they respond with those words exactly, then yeah. There are plenty of valid opinions from people who have never boxed. You see this a lot when fans call a fighter a "bum," but a boxer is way more reluctant to use the word (unless deliberately trash talking to get under skin). To get at the level where you are on T.V. you aren't a bum. It's a mean word used by loser fans.
With that in mind, fans can be awful ignorant and downright stupid sounding. People who haven't boxed giving their advice or "should haves" are commenting on something they have an incomplete opinion about. It's like someone saying, "oh, I wouldn't like having kids, I wouldn't enjoy it," but they never had them so how do they know? Likely, they see the frustrated parent wringing his or her fists in idle fury as their kid throws a bitch fit in an aisle at a department store and think, "that sucks." And, there is a truth to it. It sucks. But it's a limited truth with much ignorance. That observer also doesn't get to watch their kid learn something, like untie a knot, or finish a puzzle, and then feel this sense of fullness and love, or get it in a more direct way, when the kid walks up to them when they are sad, gives them a hug and says, "I love you." They don't really get it b3cause they don't have it. Nieces, nephews, friends with kids, none of that is the same, it's like Plato's forms. There is very nearly no way to replicate that feeling of being a parent except by living it. Those people who comment on not wanting kids might just be right, to a point, but honest truth: they don't know what they are missing out on so how much do they really know? Limited, at best.
To bring it back to the sport, it's like saying, "that boxer should retire," but if you've never experienced boxing, you don't know their "why," with regards to their refusal to quit. Maybe they should, but it's like someone who was an addict understanding the plight of recovery much better than someone who hasn't. You've got some ignorance tucked away in your soul. To experience anything is to understand it in a deeper way. Process manifests in the brain automatically.
Likely, people who haven't boxed will try to defend their position, and they certainly can have valid points. Coaches who have never boxed are obviously very close to the sport, like Angelo Dundee, and people like him have an incredible boxing mind. But he sees the sport in a deeper and different way from fans because he, too, lives the mentor part of it (by extension, not all boxers make great coaches just because they have boxed).
So, as informed as fans can sometimes be, they have an ignorance that people who have boxed won't have. They need to accept this and not be insecure about it.