as long as you're healthy, and plan on practicing into your 60's, i say go for it.
we have a few attendings in their 70's still going strong--that would give you at least 15 years practice. if you have major debt right now, i'd probably still say go for it, because you'll make more in ortho than you will in primary care, even if you consider you'll be going fairly lean during the 5 years you'll be in residency (money-wise)--when you get out, you'll quickly recover what you would have earned if you were still in primary care.
if your back is at all questionable, or you're not in the best health, then i'd say hold off.
hell, when i was in medical school, we had a guy who started med school at 48 years old, and he went into gen surg, and is doing a fellowship next year (vascular) from what i heard--that puts him at 58 or 59, and just getting ready to start his first practice