Road safety fears have been raised over a bid to level a derelict school for scores of new homes as part of an assisted living scheme.