Jackson residents pushed back against a proposal for what could be the biggest building in the town’s history.