I think you're absolutely right in the sense that the area has shitty urban design. Two 'freeways', a casino, few street retail options and low pedestrian traffic.It happens in other cities. I used to live around the corner from a shelter of similar size to Alpha House in Toronto (+100 beds). It had very little impact on the neighbourhood. I think the big difference is that there was sufficient density and street life in the neighborhood that the dozens of people who used the shelter just blended in with the hundreds of people walking around the neighborhood. Additionally there was a level of "eyes on the street" that kept people feeling comfortable.
Alpha House wouldn't be an issue if this section of the Beltline wasn't so bleak. Blame the terrible street level treatment of all the new architecture in the area. Blame the car sewer that is 1st SE. It's hard to see how this area becomes significantly better even if Alpha House vanished tomorrow.
But let's be honest. The clientele the Alpha House attracts doesn't do any favours for the area. No two homeless shelters are alike.