I expect that you placed the sign on your property and this is not a sign from a highway department placed on an easement.
I expect that you didn't seek and receive approval from the Association prior to installing the sign, you were in violation for not getting prior approval.
I expect that the Association just noticed or, it was just brought to their attention. They are now acting on it.
I think you would have a stronger argument if the Association did regular inspections looking for violations.
I know you would also have to have proof that the sign was installed 15 years ago (hence, pleading guilty to breach of contract).
Doing some research on the internet, you might have a defense, see
Statute of Limitations for a Texas Breach of Contract Claim, however, if it goes to court you will likely pay more to test your theory on the statute of limitations.
How much is that sign worth to you (monetary wise, energy spent on a legal battle, etc.) if the Association wants to take the issue to court?
My suggestion, when called before the board, have your evidence ready proving the sign has been there for more than 4 years (the farther back the better).
Provide a copy of the statute of limitations.
Wait to see what will happen.
Hopefully, the Association will check with their attorney.
You may also want to check with your attorney prior to the hearing before the board.