I have no issue with leaders, the existence of leaders is a good and proper circumstance within a natural human society. It is rulers to whom I object and will struggle against. A leader does not need to rule to lead but a ruler can only lead by rule. Ron Paul, amongst others, helped lead me to a place where I could see that my innate desire for freedom was legitimate, conversely 'the state' rules that desire illegitimate and, in doing so, leads the masses to believe the legitimacy of that rule.
We cannot all be authorities on all things, there are those who are legitimate authorities on matters and by being so they are leaders in that field. The ruler's authority is only supposed authority because they rule it so and that does not make them leaders in that field but rather just rulers of the field instead.