The 'sweet spot' where everything feels good, there's no printing, nothing shifts varies from person to person. Please don't get hung up on "Everyone else carries at 3:00, why can't I."
For me, carry position also varies based on the cut of my pants, placement of belt loops, position of pockets, even fabric. There's also the holster to consider as well - thickness, leather, kydex, combination. Oh, and that thar thang you're carrying, its size comes into play, as well.
You say, "...I would need my pants to be about 3" too big in the waist!" Do I read from that you're trying to carry IWB with the same size pants as usual? That will be uncomfortable.
You will need a larger waist size. That may be anywhere from 1" to 3". The odd waist sizes, 31", 35", etc., are normally found on higher priced or custom made slacks. 2" extra works well for most. If you're still having problems, buy a larger pant and take it to a tailor. If you carry concealed 12-18 hours each day, that money spent on getting the waist size just right, tailoring the fit to match your body will be well worth it. If you go the tailor route, think about having the waistband reinforced where you'll carry the holster.
Cant is another important factor. How do you know what is right for you? Easy... Unload your carry piece. Remove the ammo from the area where you are. Unload your carry piece again, checking the chamber this time. Now that you are certain the firearm is empty, check it one more time. Okay, you will need a piece of flexible cardboard measuring approximately 8 1/2" x 11", a pencil, your carry gun, and a friend.
Wearing the pants you'd normally carry in, using the gun belt (You do use a gun belt, right?) you normally wear, find the sweet spot on your waist by inserting the pistol inside your waistband Mexican style. Once you find it, slip the cardboard between the pistol and your body. Make certain the top edge of the cardboard is parallel with the top edge of your waistband. Having a helper makes this much easier.
Now adjust the cant of the pistol until the grip angle offers you the best purchase on the pistol. Once you find that perfect angle, have your friend draw a line parallel to that angle on the cardboard. Remove the cardboard. Remove the pistol. Do not reload the pistol yet, you might want to try other pants.
Measure the angle. Most holster makers build cants in 5 degree increments, 0-5-10-15-20 degrees. Choose the one closest to what's right for you.
Okay, now that you know where is most comfortable and the cant that offers the best draw, you can look for the right holster and belt.
That belt is extremely important. We've had many GTers come on saying they can't find the right combination for comfortable carry. Turns out they're using belts that cannot support the weight of the holster and gun. Don't cheap-out on the belt. There are belts designed to support today's rigs that look like dress belts, casual belts and work belts. Some are built using Chicago screws so you can change buckles.