The problem is that doctors aren't really there to challenge what patients tell them about their conditions and how they are affected by them. Now in many cases those things may be obvious and undeniable but in others they may not be so the consultant would be put in a difficult situation if he/she was at odds with what the patient was claiming and I'm not sure they would appreciate that extra responsibility and any adverse effects on their doctor/patient relationships.
Doctors do need to be involved in the assessment process where there's a medical issue but I'm not sure they should be the ones who're treating the people in question even though they clearly would need to have access to any relevant reports etc from the claimant's doctor(s). It's a really difficult problem.