The "building line" may be an imaginary box in which the house can be built. It is different from the property line.
When we had our house build the developer had a 'to scale' overlay of our house asked us where we wanted in placed on the lot, BUT it had to be within the box. I'm not sure if that box represented the county setbacks for home construction. ie. setback from the street, sideyards.
If you build over a property line you are royally screwed, but if you build outside the 'box' that represents the setbacks/building line then you may be able to get a variance from the county. If the HOA has its own rules that establish where the imaginary box (building lines) are acceptable, and you build outside it, then really the HOA should have already had in place a process to address ...since it isn't that easy to move a whole house!
