As usual, you'll have to refer to your governing docs and state laws...
...but typically the membership can either vote directors out as their terms expire at the annual election OR you hold a special meeting for the purpose of removing and replacing one or more directors. I agree with the others that it would be very unusual for the board to remove some of their members. They can remove a director from his officer position, but the director would remain on the board.
Your quorums may be different, but in my community the quorum for the annual election is the same as that for a special meeting (20%).
There will almost certainly be a number of requirements for holding a valid special meeting besides the quorum. These requirements generally include notice (how soon before the meeting, written vs electronic) and a published agenda in which the only item is the vote(s) to recall and replace. You may not conduct any other business beside what appears on the notice/agenda.
Before you decide to replace directors, make sure you have willing volunteers who will do a better job. This is where the remove-and-replace effort can fail. You don't want to just get rid of directors and find yourselves without a functioning board. There are worse things than untreated ponds ("Ask Me About Receivership!").
Usually the underlying problem with dysfunctional boards is a disengaged community. Nobody wants to volunteer, and few pay attention to what's going on. There are
exceptions to this such as when a group of bad actors take control of the board and makes it hard for homeowners to get rid of them. It's a lot of work to get a community back on track, and you need allies who see things your way. You don't want people who say "we've got a problem", you want people who will say "we have a problem and I'm willing to do the work to fix it."