TE14 Policy for the In-Training Assessment

AUSTRALIAN AND NEW ZEALAND COLLEGE OF ANAESTHETISTS

                                                                                                        Download PDF (**PILOT**)

                                                                           Download Background Paper PDF (**PILOT**)

 1. OVERVIEW

1.1 The In-Training Assessment (ITA) Process is an integral part of the assessments for the ANZCA Training Program. It complements other assessments, such as the Primary and Final examinations and the Formal Project, by assessing Trainee performance in the workplace.

1.2 The ITA Process is a staged, recurring, joint means of goal setting, monitoring, feedback and assessment involving Trainees and Supervisors of Training (also referred to in this document as ‘Supervisors’). The process requires active participation by both Trainees and their Supervisors.

1.3 The ITA Process is a summative assessment with a formative component included (i.e. the formal feedback interview). The feedback or formative component is included to assist all Trainees to improve their performance and to provide guidance to help each Trainee plan for ongoing improvement in the Training Program. This formative assessment is personal and aims to be supportive of Trainees. It is expected that a majority of Trainees will progress through Training performing at or above the expected level. Summative assessment determines progression of Trainees from one stage of Training to another. Trainees who do not achieve an acceptable level of performance in the ITA Process can be referred to the processes outlined in College Professional Document TE18 Policy for Assisting Trainees in Difficulty (1). If this does not resolve the issue(s), then further assessment can be undertaken via the Trainee Performance Review process (2).

1.4 It is essential that the ITA Process is conducted in accordance with sound educational principles (3, 4) and that the principles of natural justice are observed (5).

2. GOALS

The goals of the ITA Process are to:
2.1 Discuss and set appropriate clinical and educational goals for the training Term.

2.2 Ensure that training department, Supervisor and Trainee expectations are understood and negotiated to ensure that a realistic plan for the Term is generated.

2.3 Assess Trainees’ progress towards obtaining the agreed clinical and educational goals.

2.4 Provide Trainees with regular, constructive feedback.

2.5 Develop any remedial activities that may be required to ensure that Trainees are performing at or above the level expected for their stage of training.

3. DOCUMENTATION

3.1 The ITA Process is supported by the ITA-Form and the ITA-Short Form (ITA-SF) (6). These forms were developed using the ANZCA Curriculum Outcomes Framework (7) which aims to explicitly define all aspects of contemporary practice of Fellows of the Australian and New Zealand College of Anaesthetists.

3.2 The ITA-Form includes a record of:

3.2.1 Approved Training and Training Details, including hospital name, Term dates, full- or part-time status, rotational training status, type of clinical experience (e.g. anaesthesia, intensive care medicine), and amount of leave taken during the Term.

3.2.2 Formal meetings between Trainees and their Supervisors: the Initial interview, Mid-Term Interview(s) (if required) and the End-of-Term Interview. The Form includes prompts for Supervisors undertaking the Initial and End-of-Term Interviews. There is space to document discussion about Trainee Performance, including concerns and any recommendations made or plans agreed.

3.2.3 The End-of-Term Assessment, which includes domains based on the ANZCA Curriculum Outcomes Framework (7). Each domain is rated as ‘rarely meets’, ‘inconsistently meets’, ‘meets’, ‘sometimes exceeds’ or ‘consistently exceeds’ expectations, with a rating of at least ‘meets’ expected. A global assessment is used to indicate whether Trainees are assessed as performing at the level expected for their stage of training or not (‘yes’, ‘borderline’ or ‘no’).

3.2.4 The signature of both Supervisor and Trainee, and Trainee comments about the End-of-Term Assessment. Note: references to signatures in this document include both hard-copy and on-line signatures.

3.3 The ITA-Short Form (ITA-SF) is a shortened version of the End-of-Term Assessment section of the ITA-Form. It is used by senior staff to provide individual or combined assessments of Trainee performance in the workplace.

3.4 The Learning Portfolio
Throughout training, Trainees should maintain a Learning Portfolio which includes copies of the ITA-Form for each Term. See College Professional Document TE8 Guidelines for the Learning Portfolio for Trainees in Anaesthesia (8) for further information about the Learning Portfolio.

4. MEETINGS BETWEEN SUPERVISORS OF TRAINING AND TRAINEES

4.1 The following formal (i.e. documented) meetings must occur between Trainees and their Supervisors during each six-month period (or earlier if the term is less than six months). The meetings are often initiated by Supervisors. If any of these meetings have not been arranged within the expected timeframe, Trainees should contact their Supervisor.

4.1.1  The Initial Interview should be held within two weeks of the start of each Term. At this initial interview, Supervisors will assist Trainees in setting appropriate clinical and educational goals for the Training Term. This will involve consideration of Trainee performance in the Training Program to date, where relevant, via discussion with the Trainee, review of the Trainee Profile, the Learning Portfolio and previous ITA-Forms. Supervisors of Training may contact other Supervisors and the Regional/National Education Officer (REO/NEO), if necessary, to assist in this process.

4.1.2  The End-of-Term Interview should be held within two weeks of the end of each six-month Term (or earlier if the Term is shorter than six months). At this Interview, Trainees and their Supervisors review and discuss Trainee performance
and the End-of-Term Assessment for the completed Term.

4.1.3 If Trainees are continuing at the same institution for the following Term, then the End-of-Term Interview may be combined with the Initial Interview for the
next Term.

4.2 Mid-Term and Other Interim Meetings

4.2.1 Additional meetings between Trainees and their Supervisors should occur as required. This may be the case, for example, where Trainees are in difficulty (1).

4.2.2 Such meetings may be initiated by either Supervisors or Trainees.

4.2.3 Trainees experiencing difficulty, at any stage, should bring this to the attention of their Supervisor as early as possible and request a formal meeting with their Supervisor.

4.2.4 If senior or other staff have any concerns about Trainee performance, including concerns that Trainees may be in difficulty, this should be brought to the attention of the Supervisor who should arrange a formal meeting with the Trainee.

4.2.5 All formal meetings between Supervisors and Trainees should be documented.

5. THE END-OF-TERM ASSESSMENT 

5.1 At the End-of-Term Interview, the formal assessment of Trainee performance should be based upon the following Compulsory Assessments:

5.1.1 An ITA-SF completed by each of at least three senior staff who are best placed to provide that assessment

 AND/OR

5.1.2 A meeting of the senior staff of the Department to complete an ITA-SF.

5.2 The End-of-Term Assessment may also be based upon the following Optional Assessments.

5.2.1 The College supports the use of Workplace Based Assessment (WBA) tools to inform assessment of Trainee performance. In addition to the compulsory assessments above, Supervisors and Departments may use additional WBA tools, for example the Mini-Clinical Evaluation Exercise (Mini-CEX) (9, 10), or forms developed by Departments for this purpose.

5.3 The above assessments should be used by Supervisors to facilitate completion of the ITA-Form. This information is then used at the End-of-Term Interview to discuss performance during the completed Term and to set goals for the following Term. Once Trainees have had an opportunity to add comments about the assessment, the ITA-Form is signed by both the Trainee and the Supervisor. The Trainee’s signature confirms receipt of the ITA-Form, but does not necessarily indicate acceptance of all its contents.

5.4 Signed ITA-Forms should be printed out by Supervisors and filed by Trainees in their Learning Portfolios.

6. SUBMISSION OF FORMS TO THE COLLEGE AND MONITORING PROCESSES

6.1 Within two weeks of the End-of-Term Interview, the signed ITA-Form should be
submitted by Supervisors to the Training & Assessments Unit at the College in
Melbourne.

6.2 The ITA-Forms will form part of Trainees’ records of training.

6.3 The Training & Assessments Unit at the College in Melbourne will monitor the results of the global assessment of Trainee performance. In the case of global performance that is below the level expected for the stage of training (‘no’ or repeated ‘borderline’ ratings on the global assessment), Supervisors and the REO/NEO will be contacted to ensure that Trainee performance is being managed according to College processes (1, 2).

7. PERFORMANCE BELOW THAT EXPECTED FOR THE STAGE OF TRAINING

7.1 The College processes for dealing with Trainees with performance issues are:

 7.1.1 The TE18 Policy for Assisting Trainees in Difficulty (1).

 7.1.2 The Trainee Performance Review Process (2), which may be undertaken if the processes outlined in College Professional Document TE18 Policy for Assisting Trainees in Difficulty (1) are unable to resolve the issue(s).

7.2 In the case of Trainee illness or disability, reference should be made to College Professional Document TE19 Policy on Trainee Illness or Disability (11).

7.3 Any serious errors or incidents affecting or potentially affecting the safety of patients, Trainees or other staff should be managed as outlined in College Professional Document TE18 Policy for Assisting Trainees in Difficulty (1).

7.4 When Trainees perform at a level which is considered borderline for a developing specialist anaesthetist (i.e. ‘borderline’ for the level expected for their stage of training on the ITA-Form global assessment), in the first instance, Supervisors and Trainees should together negotiate a plan for the subsequent Term that includes the remedial activities that are required to assist Trainees perform at or above the level expected for their stage of training.

7.5 The provisions outlined in College Professional Document TE18 Policy for Assisting Trainees in Difficulty (1) should be implemented in the following circumstances. This requires that processes outside In-Training Assessment are undertaken.

7.5.1 When Trainees consistently perform at a level which is considered to be below that acceptable for a developing specialist anaesthetist, that is below the level expected for their stage of training (‘no’ on the ITA-Form global assessment).

7.5.2 When Trainees perform at a level which is considered borderline for a developing specialist anaesthetist (i.e. ‘borderline’ for the level expected for their stage of training on the ITA-Form global assessment), on two successive End-of-Term Assessments.

7.6 The above not withstanding, the following may assist Supervisors and senior staff in situations where an aspect (or aspects) of Trainee performance is/are not at the level expected for the stage of training.

7.6.1 If Trainee performance is assessed as less than ‘meets expectations’ in any of the domains listed on the ITA-Form, then this matter must be discussed with the Trainee (and meetings documented) to establish remedial strategies to improve that area of performance. A single domain assessed as less than ‘meets expectations’ does not necessarily, of itself, constitute an unsatisfactory assessment.

7.6.2 If Trainee performance is assessed as less than ‘meets expectations’ in any of the domains listed on the ITA-Form on more than one End-of-Term Assessment or there are multiple unsatisfactory domains on the one occasion, serious consideration should be given to whether the global assessment should reflect performance that is below the level expected for the stage of training. The ratings must be discussed with the Trainee (and meetings documented) with a view to establishing remedial strategies to improve that area (or those areas) of performance.

7.7 Any plan for remediation of Trainee performance should include: agreed goals which are specific, outcomes that can be measured, and a clear timeframe for improvement. It is likely that additional formal meetings between the Supervisor and Trainee will be required to provide feedback and encouragement in progress towards agreed performance measures. Additional information is in College Professional Document TE18 Policy for Assisting Trainees in Difficulty (1).

7.8 Advice regarding any aspect of Trainee performance or assessment may be obtained through the REO/NEO and/or the Training & Assessment Unit at the College in Melbourne.

8.  PRIVACY

8.1 Information collected in relation to Trainees will be held, used and distributed as provided in the College’s Privacy Statement (12), the Training Agreement (13) and as permitted by law.

8.2 Ordinarily each ITA-Form will only be considered by the College, and used for the purposes of the Training Program. An ITA-Form is not intended to be used for employment purposes, but rather is intended solely for use in the Training Program in accordance with the goals outlined in section 2. It will not ordinarily be provided to the hospital/employer, unless it is appropriate to do so in the interests of patient safety or as required by law.

8.3 To assist Trainee supervision, assessment and, where required, remediation of performance, ITA-Forms will be shared with Supervisors in subsequent rotations. Supervisors are able to view all on-line Trainee Profiles of Trainees under their supervision, as well as ITA-Forms with end dates within the prior 365 days. Access to earlier ITA-Forms may be requested by contacting the Training & Assessments Unit at the College in Melbourne.

REFERENCES
1. Professional Document TE18 Policy for Assisting Trainees in Difficulty
2. Trainee Performance Review: www.anzca.edu.au/resources/regulations/regulation-33.html
3. ANZCA Guidelines on Assessment: www.anzca.edu.au/resources/guidelines/anzca-guidelines-on-assessment/
4. Kilminster S, Cottrell D, Grant J, Jolly B. AMEE Guide No. 27: Effective educational and clinical supervision. Med Teacher 2007; 29: 2-19.
5. Principles of natural justice: www.ag.gov.au/www/agd/agd.nsf/Page/Securityvetting_Whataretheprinciplesofnaturaljustice and www.justice.govt.nz/publications/global-publications/t/the-guidelines-on-the-new-zealand-bill-of-rights-act-1990-a-guide-to-the-rights-and-freedoms-in-the-bill-of-rights-act-for-the-public-sector/publication (sections 27.1 – 27.3)
6. For copies of Training Forms click here
7. ANZCA Curriculum Outcomes Framework
8. Professional Document TE8 Guidelines for the Learning Portfolio for Trainees in Anaesthesia
9. Weller JM, Jolly B, Misur MP et al. Mini-Clinical Evaluation Exercise in anaesthesia training. Brit J Anaesthesia 2009; 102(5): 633-41.
10. Weller J, Jones A, Merry A et al. Investigation of trainee and specialist reactions to the Mini-CEX in anaesthesia: implications for implementation. Brit J Anaesthesia 2009; 103(4): 524-30.
11. Professional Document TE19 Policy on Trainee Illness or Disability
12. ANZCA Privacy Policy
13. ANZCA Training Agreement: www.anzca.edu.au/trainees/trainee-forms/training-agreement

ANZCA professional documents

College Professional Documents are progressively being coded as follows:

TE Training and Educational

EX Examinations

PS Professional Standards

T Technical

POLICY–defined as ‘a course of action adopted and pursued by the College’.These are matters coming within the authority and control of the College.

RECOMMENDATIONS–defined as ‘advisable courses of action’.

GUIDELINES–defined as ‘a document offering advice’.These may be clinical (in which case they will eventually be evidence-based), or non-clinical.

STATEMENTS–defined as ‘a communication setting out information’.

This document has been prepared having regard to general circumstances, and it is the responsibility of the practitioner to have express regard to the particular circumstances of each case, and the application of this document in each case.

Professional documents are reviewed from time to time, and it is the responsibility of the practitioner to ensure that the practitioner has obtained the current version. Professional documents have been prepared having regard to the information available at the time of their preparation, and the practitioner should therefore have regard to any information, research or material which may have been published or become available subsequently.

Whilst the College endeavours to ensure that professional documents are as current as possible at the time of their preparation, it takes no responsibility for matters arising from changed circumstances or information or material which may have become available subsequently.

Promulgated as E14: 1994
Reviewed:  2001, 2007, 2010
Date of current document:  June 2010*

*This Professional Document is being piloted for 12 months and will be reviewed in June 2011.

©  This document is copyright and cannot be reproduced in whole or in part without prior permission.

ANZCA Website: http://www.anzca.edu.au/