I'm pretty sure Mill would not have extended his belief to cover those who - from the point of view of an educated 19th century person - would have seemed insane. The mentally ill, 200 years ago, were ignored or imprisoned at best. That's the context in which Mill wrote about the value of opinion.

The kind of anti-vax nutjobs we have today would have surely been seen as mentally ill 200 years ago. (Granted, this is an assumption I'm making, but I contend it is a reasonable one.)

It is evident to me that he was referring to the opinions of only those who were sane enough to have robust and well-informed opinions.