Outsiders use of HOA's amenities

I'd like some input or advice on this subject. This matter is something that other members of our HOA (this is TX) are concerned about but it's a big battle between the board, members, swim team, (outsiders have stayed out of the fight because they know they have no say), etc. But homeowners (payers of dues) don't have much say either. Our homeowners association has a board of 9 members (I was on it for five years). The HOA has never sponsored, but has allowed the swim team (ST) to use the association's pool for practices (when homeowners are not using it) and for meets (limited) on Monday afternoons when the pool would be closed after having been cleaned/shocked the night before anyway, therefore no homeowners are using it. However, the ST, thanks to a group of militants has become a large group now, but ONLY because they've recruited and encouraged from outside of the subdivision in order to stay large enough to stay in a certain division in order to keep the practices and meets closer to the area. The problem in our sub is that children are getting older and older and the homeowners are getting older and older and there isn't that much interest in the ST except by a few who are just out in left field--in fact, they DO NOT request permission from the board for use of the pool anymore, do not abide by the board's request for an insurance certificate, do not give the board a list of the swimmers, per the board's request, and if a homeowner has requested use of the facilities (paid for by that homeowner as part of their membership), they are refused use of the pool/clubhouse/park or whatever because a ST member or outsider on the ST has priority. It is an awful mess and has caused much contention in the association. In fact, a year or so ago, a homeowner requested the facilities for the first time since they had lived in the HOA (over 10 years) to have a once-in-a-lifetime celebration, and because it was at the time of a ST practice, they were refused the use of the facilities. The number from outside (of swimmers) has increased to over 30% being from outside and paying nothing for use of our facilites, yet, if we, as members have not paid our dues, we cannot vote nor use the facilities--this is outrageous. A lot of the problems started when ST parents got on the board and turned it into a social club instead of an HOA Board and they do what they want, when they want with the other member's monies. It has become dangerous for people who do not agree(and I will not go into that detail here or yet). I have searched everywhere for any similar situations, but think we are unique (no one has written about it anyway).

These are the questions--is there a law about this or some law that can be used by those of us who are trying to have controls put on it, can be use to at least get the rules the board set forth enforced? (The board set up a policy about the rules and the present and now almost former president--has moved and still is pres.--never had the policy signed or envoked because of nepotism)?
Isn't this like some are giving a gift of our facilities to outsiders and using our monies unscrupulously?
What can be done? Would a class action suit help with this matter? The board has now spent the majority of the reserves on totally redoing the pool and facilities, but it is not for the members of the HOA, it's for the ST and outsiders on it. It has made a lot of people on fixed incomes and others (about three fourths of the members) very angry and I do not blame them. But hands are tied and these people committed fraud (lied) about the state of the association in order to get on the Board and do these things. What can be done?
EdR

Check with your insurance company and also your CC&R's. If your insurance company states you're liable for any injuries, bar them from using your pool...and fast! Also, you might have backing in your CC&R's. Ours specifically states that our pool is for the residents only and up to two guests.

Did you ever get this issue solved? If so, how? I have a similar, although much less severe situation in my neighborhood.

We have a "pool attendant" that basically babysits the people in the pool, she is NOT a lifeguard and signs are posted saying so. Our HOA issues pool passes every year, for $10 per family of 4, to those that want to use it and the pool attendant keeps it in an index card box at the pool. If she doesn't have our pass in the box, we can't go in to swim. Also if we bring a guest, they must pay a $1 to swim. I have suggested to our board to change the gate/lock to the pool so that homeowners can have their own unduplicatable key, raise the pool pass to $20, and eliminate the pool attendant (save money). They have denied my suggestion due to them giving up "control". Sorry I went off on a tangent.