Let's NOT focus on USMLE scores, try something else

I don't know much about the actual matching process (as I have yet to tackle it), but what is really compelling to me about the emphasis on USMLE scores in ortho is how little they have to do with orthopaedic surgery. Albeit they are a reasonable measure of general medical aptitude and how hard a person studied for two years, but in ortho it seems much more important that you have excellent visual/spatial reasoning, manual dexterity, "aesthetic" sense, and physical/emotional stamina. This is, of course, part of what you prove on rotations...

I agree with the above posts, but wanted to add a little about USMLEs.

Recent article showed a moderate correlation between USMLE Step 2 (but not Step 1) and OITE: J Bone Joint Surg Am. 2006 Mar;88 (3) : 671-6

Another article found a trend with higher OITE scores in those with Step 1 above 220 and those with AOA: South Med J. 2005 May;98 (5) : 528-32

---

Now, at the risk of starting a flaming war...

Another point that may be interesting for discussion: Step 2 scores correlate significantly with Verbal SAT and MCAT reading scores: Acad Med. 1996 Feb;71 (2) : 176-80

Furthermore, SAT scores correlate with IQ - Psychol Sci. 2004 Jun;15(6):373-8, which begs the question - is all this just a way to assess the IQ of applicants? IQ testing for the purposes of employment is illegal, but as well-described in a somewhat controversial book "The Bell Curve", IQ scores are by far the best way to predict on-the-job performance for many professions.

Just a thought.

Hey Bob, you make some interesting points. Just to add some logs to your fire. At one interview, I was asked my SAT scores. I was like how does that matter because it was such a long time ago. They said that the SAT cannot really be studied for and therefore is a higher perdictor of raw intelligience......

This info was used for an informal survey and not to rank the candidates.

in JBJS and was just going to post about it. Very interesting. Maybee programs should require early step II.

I find this very interesting. I wonder if that correlation exists because the step II is more clinical or because it is a test that people don't worry about as much, don't study for as much and consequently is a better assessor of fundamental/retained knowledge and problem solving. If programs started requiring step II, than people would start freaking out about it like they do for step I and scores would probably rise (not because people are smarter, but because they studied specifically for it).

The more appropriate question to ask is: how should USMLE board exam scores be used? As I heard on the interview trail from an attending who sits on the USMLE board and helps design the surgery portion of the exams: Step I is intended to determine who has successfully integrated information from the basic science years to such a level that they can now begin the clinical portion of their training. Similarly, Step II is designed to determine who has successfully integrated their clinical training to such a level that they can now safely be considered for post-graduate medical training (residency).

Neither exams are "residency aptitude tests." Studies have been done to correlate board exam scores and numerous future markers (such as OITE scores), but SHOULD they be used for this purpose? As the JBJS article from 3/2006 states: "Although we identified a significant correlation between Step-2 USMLE scores and resident in-training examination scores, we also believe that it would be inappropriate to overemphasize USMLE Step-2 scores in resident selection. As noted previously, the USMLE is a licensing examination and is not designed to predict future performance on other examinations nor is it designed to predict resident performance or, ultimately, the quality of the practice of medicine. Test results can be biased by medical school characteristics and curricula that are patterned on a USMLE format, resulting in substantial advantages to those students taking the examination."

Just some more food for thought....

This is exactly the problem. As Dr. Simon states in JBJS A 83:1416-1423 (2001), despite picking the brightest candidates there's still a substantial number of residents who are "substandard" or "marginal", and up to 10% fail written and oral boards. So there's no good way of predicting who will be a successful orthopaedic resident based on any test devised to date. Actual on-the-job performance is probably better, but we can't rotate at every program we're applying. Maybe there was good reason for old-school pyramidal programs in surgery (not that I would ever advocate bringing that back).

This is total absurd
Many people think it is possible to run some tests and find out every thing about person (disease) and if tests are negative or scores are high - it guarantee something . I think it is very primitive approach.

USMLE exams were not created as an IQ or whatever ?measurer? ...it was invented to confirm (Yes /No) MD qualification..
.therefore, interesting ?..why it is not in the pass/fail format?
I got 99/91 on steps and can say. These score correlates mainly with your effort, ?condition? at the time.. Everyone could score very high if he/she studies a lot.
What happens after people get in residency? After ?med students? achieve their main goal/motivation ? getting into the residency - they relax and do not kill themselves anymore...or they get too busy in county hospitals. This is main reason for poor in training scores...

...About articles and quotes. I always wondered why somebody would conduct large study just to prove very logical and "common sense" conclusions.

IMHO... selection process might have following pattern
- Screening - most (but absolutely not all) applicants with score below 200 considered ?not good enough for interview selection?
- "right personality? and "hand skills" - are main criteria ????they are even more important then low scores. I know many MD?s with excellent grades but who are poor physicians/surgeons and opposite.. Especially it is true for Ortho. You have to be ?Mechanic? first and only after that ?Good Scorer?. But in our times, when it is very prestigious to be a bonedoctor??? people often become Orthopaedic Surgeons only because they feel it is highest ?prize? for their hard work in the med school and not because they really like it or they ?were born for it?

While I think away rotations are the reason many people match where they do, there are probably as many that match at seemingly random places. My aways didn't have the least to do with where I matched, unless unknown to me one of my LOR from one of my aways was noticed? And no, I was not a jerk, lazy, etc. on my aways. Everyone seemed quite impressed and happy to have me there, but apparently these places ended up ranking some other great appliants ahead of me. The bottomline, I don't think anything about the match is predictable, nor do I think there is a great predictor of OITE scores until you start taking them as a resident. However, it's interesting to read you arguing about this, so more power to those of you that have time for it- I know I won't in 3 months. Congrats again to all that matched!

It would probably make sense to test applicants on visuo-spatial skills, since that is one important skill an orthopaedic surgeon needs say, for instance, to be able to imagine a fracture pattern in 3D from 2D views. However, I disagree on a more fundamental level. I think that "mechanic first" is a completely wrong view of this specialty. A "mechanic first" is the kind of hot-shot surgeon that quickly ends up doing the wrong procedure for the wrong indication, but he does it damn well. Not somebody I would want as my surgeon.

So yes, there are many untestable aspects that are vital to being a competent orthopaedic surgeon, but I think intelligence and being able to learn/integrate information is more important than technical skill. Besides, technical skills are learned, in my opinion, more easily than decision making. All it takes is practice - tying knots, suturing, drilling, etc. But if you don't have the ethic / effort to read and learn the science of medicine, then you're gonna be a shitty doctor, no matter what your specialty.

again whole idea about tests is very slippery you have to spend time and work with that person to see real "situation"
"Talent might be "hiding" from these tests " (c)
Rendering Error in layout BBCode/Image: Layout 'BBCode/Image:default' Not Found. Please enable debug mode for more information.
(point of view from our position)

when I was saying "mechanic" that was big compliment and not under any circumstances disrespect of this specialty

It is the ?BASE?, ?kind of mind and hands? on which in your intelligence and education will be build for that profession.
Being mechanic is different way of thinking and acting/working.
It is about:

- Seeing whole system in interaction
- Strong common sense
- Spatial sense and good eye-hand coordination
- ENJOYMENT of doing hand work
- Understanding of ?light touch? concept (if you get it you are almost Master)
- ?Inventor? and creative personality

Is it possible to be an Orth Surg without it? YES. It is possible to be a Regular Orthopod.
But to be The Surgeon who is object of admire or opposite jealousy of his colleges you will need that ?mechanic core?. Trust me. I had opportunity to observe many ortho residents and attendings in 11 years - 5 years in my residency back in Moscow and 6 years working as an ortho tech in USC Orthopaedic Department. You will see it yourself.
And in the end. High USMLE score might be actually very useful for ortho applicants screening. If I was the Program Director I would run from people with scores 270 and above ?.they probably never had space in brain for ?mechanic? personality?

I am actually curious about this - how competitive is orthopaedics in Russia? And what is the selection process like?

right now
it is not so competitive as in US
- because orthopods make less money then say bus driver
Rendering Error in layout BBCode/Image: Layout 'BBCode/Image:default' Not Found. Please enable debug mode for more information.


- because medicine belongs to the goverment and after collapse of Soviet Union
famouse Russian medicine, sport, science and education were left to the mercy of fate

but still even long time ago (1993) when I was appliyng to ortho residency it used to be very hard to get in but not that ridiculous difficult like now in USA. People apply to Ortho in USA mostly because it?s ?very secure future?

OK. Now about Russia

Firs of all: In USSR Ortho it always was mainly some kind of emergency/charity ?not for profit? medicine. They even call orthopods - Traumatologists. Most of the people had no luxury to treat chronic conditions. Most of the patients were ?broken? old grandmothers and alcoholic men. Than is why Ortho is not very prestigious profession in Russia and most of ortho residents were very average students in the class. Best ones usually wanted to be a Scientists (like Immunologists) , Internists, Neurologists. I have a friend in US. His father in Moscow was very depressed when his son after graduation with excellent grades from very expensive American medical school decided to go in Ortho Residency in US 

Second: In spite Russia being biggest country in the world (USSR was even bigger), Russians do not like to move from city to city. It happens VERY VERY RARE. So if you were born in Moscow probably 99% you will go to med school and stay all your live same city , in Moscow. Therefore even ?big? (programs like 5-7 spots) were mostly dealing with only 10-20 local applicants. Usually they interviewed mostly students who graduated from same medical school or medical school in the same city. Also we had something like ?affirmative action? when applicants from other USSR republics (like Armenia, Georgia etc) were guaranteed certain amount of position in Moscow universities or post graduation programs.

Residency Selection process was always very subjective in Russia. Usually decision was made during personal interview with head of department or vice chair. An of course some kind of ?connections? was required. But I think American system has same problems too (subjectivity, connections)


Shortly: after all if you become an Orthopedic surgeon in Russia it means you are ?idiot? not good enoufh for other programs or you like your profession a lot ('')