The maximum number of people will take transit into the downtown core if the whole trip to the place they want to go to is convenient, and the vast majority of places people want to go to are not east of City Hall. Nobody working at Eighth Avenue Place or Centennial Place will say "Hooray! I'm in the downtown core!" getting off their train east of City Hall; they'll say "Now how the heck do I get to my office?"
If you work somewhere in the middle of the downtown, say Banker's Hall and the train stops east of City Hall, you have the following options:
1). Walk all the way to work, 1 km -- about 15 minutes.
2). Take escalator or stairs to street level - 1 minute. Wait for the light at 7th Ave - 1-2 minutes. Wait for the next train - 1-2 minutes (in rush hour). Ride it one stop - 2-3 minutes. Walk 300m - about 5 minutes. Total: 10-13 minutes.
Adding in 10-15 minutes to someone's commute will reduce how attractive the transit alternative is. And yes, there's absolutely an argument that the cost savings is worth adding time to people's commutes (and reducing ridership).
But guess what? It doesn't matter where you add the 10-15 minutes to people's commute. The SETWAY busway plan that preceded the Green Line LRT idea envisioned a dedicated transitway to the southeast along the same route as the Green Line. The travel times for a trip all the way from Seton to downtown was
43 minutes by transitway, and 35 minutes by LRT. 8 minutes difference.
If the argument is that adding 10 minutes to the commute is worth the cost savings of the previous Gray proposal, there's an even stronger argument to not build LRT to the southeast but instead build the SETWAY, which adds 8 minutes to the commute, and would be a fraction of the cost of the Gray proposal.