from what i've seen, the prices, at least in and around Virginia, are roughly the same for the various 9, 40 and 357sig glocs of the three sizes. looks like the g23 you're looking at may have an aftermarket barrel resulting in a higher price?
if you're not going to carry, i'd get a larger gun, so if you want .40, either the g22 or even the g35. i believe the grip circumferences are identical between all their .40 offerings, and at least for me, the grip on the g23 is long enough so that both my hands are fully engaged. but the longer sight radius of the g22 or 35 along with a little extra weight should make them a little easier to shoot for you.