Educator Portfolio Analysis Tool

MedEd Portal logo peer-reviewed

Download file in MS Word

Faculty Evaluated:
Evaluator:
Date:

Note to Evaluator: This tool has been peer reviewed and approved by MedEdPORTAL, 2008. The tool is designed for use with the APA Developmental Educator Portfolio Template [click here], approved by MedEdPORTAL, 2007], but it is adaptable for use with EPs in other formats. An accompanying instruction sheet offers an itemized explanation for each of the sections below.  Each item in this analysis tool is either qualitative or quantitative:

  • For quantitative items (e.g. item 6-8), put scores in the third column.
  • For qualitative items (e.g. items 1-3), put scores in the fourth column, using the scale shown at the top of that column. Add comments to explain each qualitative score.

Missing data: If data for a whole section are missing from an EP, check box to indicate this at the beginning of that section. If data for individual items are missing, enter NA (not available) in the scoring cell.


INTRODUCTORY SECTION OF THE EP

Evaluation Item Specifiers for Rating or Score
Note: For qualitative ratings, Level 2 =   INTERMEDIATE is described 		Quantitative
Score		 Qualitative Ratings and Comments
1=Novice
2=Intermediate
3=Expert
EDUCATIONAL PHILOSOPHY Check if no entries in this section
  • Self-reflection, self-appraisal
 Learns from one’s educational experiences about oneself as a teacher and about learning in general. Articulates lessons learned   Rating:
Comments:
  • Philosophy both rooted in theory or principle, and  applied to experience
 Uses a principled approach, based on educational theory or models, illustrated by specific examples from experience   Rating:
Comments:
  • Evidence of philosophy applied throughout EP
 One’s instructional and evaluation strategies are consistent with one’s stated educational philosophy   Rating:
Comments:
FIVE-YEAR GOALS Check if no entries in this section
  • Goals set bar appropriately high
 Stated goals reflect commitment to personal growth as an educator   Rating:
Comments:
  • Focused and realistic plan
Goals are specific and feasible, and clearly linked to an appropriate learning plan   Rating:
Comments:

FIVE DOMAINS OF EDUCATIONAL ACTIVITY


Evaluation Item Specifiers for Rating or Score
Note: For qualitative ratings, Level 2 =   INTERMEDIATE is described 		Quantitative
Score		 Qualitative Ratings and Comments
1=Novice
2=Intermediate
3=Expert
I. TEACHING  Check if no entries in this section
  • Total Learner Number/yr
Documented directly in the EP
[See instructions if other formats are used]		 Total Learner Number/yr =  
  • Total Teaching Hours/yr
 Documented directly in the EP
[See instructions if other formats are used]		 Total  Teaching
Hours/yr= 		 
  • Total Teaching Impact Score
 		 Each activity gets a geographic impact score:
Department= 1
Institutional= 3
Regional= 5
National=10 International= 10		 Total Teaching Impact
Score=		  
Total Teaching Impact Score is the sum of geographic impact scores for all teaching activities
  • Variety of teaching strategies
 Teaching incorporates at least 3 strategies that go beyond passive transfer of knowledge (e.g., , interactive lectures, small group sessions, workshops, clinical precepting)   Rating:
Comments:
  • Teaching evaluation score
Comparison with local peers, if available:
Below average = 0
Average = 1
Above average= 3
Superior= 5		 Teaching evaluation score =  
  • Multiple sources and types of  evaluations
 Uses 2 or more types of evaluation of teaching from 2 or more different sources (e.g., learners, peers, etc.)   Rating:
Comments:
  • Teaching Awards Score
 		 Each award gets a geographic level score:
Department= 1
Institutional= 3
Regional= 5
National=10
International= 10 		Teaching Awards Score=  
  Teaching awards score is the sum of  geographic level scores for all awards    
       
II. ASSESSMENT OF LEARNERS Check if no entries in this section
  • Total Learner Assessment Score
 		 Implementation=  1
Analysis/Synthesis= 2
Development=3
If EP author plays more than one role in a particular assessment, sum the scores for each role. 		Total Learner Assessment Score =  
Total score = Sum of all role subscores
  • Learner assessment strategies
 Uses multiple learner assessment  strategies suitable to teaching context and desired learner outcomes   Rating:
Comments:
  • Balance of methods that include upper level “Miller’s Triangle” strategies
Uses highest level feasible in one or more teaching contexts   Rating:
Comments:
       
III. CURRICULUM DEVELOPMENT Indicate curriculum rated in Items 18-21 _____________________ Check if no entries in this section  
  • Curriculum Impact Index
 		 # of learners taught:
< 50=     1
50-150=  2
> 150=   3		 Curriculum Impact Index=  
Implementation score:
Curriculum not yet implemented=1
Curriculum implemented = 2
Geographic Impact Score:
Department= 1
Institutional= 3
Regional= 5
National= 10
International= 10
Individual Curriculum Index=
Implementation score x learner # score x geographic impact score
Total Curriculum Impact Index=sum of Individual Curriculum Indices
  • Curriculum Role Score
 		 Each curriculum is scored for degree of responsibility in development
Leader= # x  5
Contributor = # x  1		 Curriculum Role Score=  
Curriculum Role score is the sum of individual subscores
  • Quality of Goals/Objectives
 Goals are appropriate in scope; objectives are specific and measurable/evaluable   Rating:
Comments:
  • Quality of Needs Assessment
Curricular design uses learner needs assessment to choose and refine goals/objectives and methods; use learner assessment data to refine needs assessment   Rating:
Comments:
  • Quality of Methods
 Curriculum design includes variety of methods that address educational goals, and meet needs of diverse learners in specific educational settings   Rating:
Comments:
  • Quality of Evaluation
 Curricular design includes sound learner assessment methods (valid, reliable, feasible); curriculum modified periodically  using results of learner and program evaluations   Rating:
Comments:
IV. MENTORING AND ADVISING Check if no entries in this section
  • Mentee Number
Number of mentees/advisees Mentee Number =  
  • Mentee Productivity Score
 		 Score reflects mentees’ grants and publications.
Use 1-5 scale:
1= poor
2= marginal
3= satisfactory
4=good
5= excellent		 Mentee Productivity Score=  
  • Mentee Professional Advancement Score
 		 Score reflects mentees’ promotions, leadership roles, and/or teaching awards. Use 1-5 scale:
1= poor
2= marginal
3= satisfactory
4=good
5= excellent		 Mentee Professional Advancement Score=  
  • Quality of Mentoring
 Mentoring philosophy reflects careful thought about experience; mentees describe significant impact of mentor on career    Rating:
Comments:
V. EDUCATIONAL LEADERSHIP & ADMINISTRATION Check if no entries in this section
  • Program Leadership Index
 		 For each leadership activity in courses or programs, multiply its geographic impact score and its duration score to create a leadership sub score Program Leadership Index =  
Geographic impact:
Department=  1
Institution= 2
Regional= 3
National= 4
International= 5
Duration of Leadership:
1-2 years= 1
3-5 years= 2
6-10 yrs= 3
>10 yrs= 4
Program Leadership Index is a sum of leadership subscores for all leadership activities in this category
  • Committee Leadership Index
 		 For each committee leadership activity, multiply its geographic impact score and its duration score to create a committee leadership subscore Committee Leadership Index =  
Geographic impact:
Department=  1
Institution= 2
Regional= 3
National= 4
International= 5
Duration of Leadership:
1-2 years= 1
3-5 years= 2
6-10 yrs= 3
>10 yrs= 4
Committee Leadership Index is a sum of all the committee leadership subscores
28. Total Committee Membership Score Duration of membership:
1-2 years=# x1
3-5 years=# x2
6-10 yrs=# x3
>10 yrs=# x4		 Total Committee Membership Score =  
Committee membership score is a sum of all the subscores
29. Quality of Leadership Strong stakeholder testimonials or accreditation results   Rating:
Comments:
30. Reviewing and Moderating Index Reviewing at national level only:
Of grants: # x 10
Of papers: # x 5
Of Abstracts (present or workshops): # x 2		 Reviewing and
Moderating Index = 		 
Moderating sessions at national meetings:# x 2
Total index= sum of subtotal scores for reviewing and moderating


SCHOLARLY ACTIVITIES

Evaluation Item Specifiers for Rating or Score
Note: For qualitative ratings, Level 2 =   INTERMEDIATE is described 		Quantitative
Score		 Qualitative Ratings and Comments
1=Novice
2=Intermediate
3=Expert
VI. SCHOLARLY APPROACH TO EDUCATION
Note to the Evaluator: To complete this section, review all parts of the Educator Portfolio, especially Educational Philosophy, Goals, and narrative comments. Place ratings (using the scale below) and comments in the third column. 		 
  • Total Educational Professional  Development Score
 		 Scores for types of educational professional development activities:
Degree-granting  program = 20
Professional development = 10
Conferences (>1day)=# x 5
Workshops (<1day) =# x 1 		Total Educational Professional  Development Score =  
Total Educational Professional  Development Score = sum of totals above
  • Evidence of reflective educational practice
Adapts one’s teaching in response to teaching and program evaluations, and outcomes of learner assessments   Rating:
Comments:
NOTE:For Items 33 and 34, give special consideration to a educator’s area(s) of focal educational effort, such as innovative projects or special curricula or tools. List focal area(s) of scholarly effort evaluated for these items:  
  •  Application of an accepted model or structured approach
 Uses Miller, Kirkpatrick, Musick,  or equivalent model of own design   Rating:
Comments:
  • Adherence of Glassick’s criteria of excellence
     Composite rating of all 6 criteria: Comments:
  • Clear goals
 Stated purpose, realistic objectives, important questions  
  • Adequate preparation
 Understanding of lit, appropriate skills, needed resources  
  • Appropriate methods
 Choice of methods that match goals, effective use and flexible application of methods  
  • Significant results
 Goals achieved, results important, field advanced  
  • Effective communications
 Presentation well organized, forums appropriate, message clear and sound  
  • Reflective critique
 Work critically evaluated, supported with good evidence, evaluation used to improve future studies  
VII.  PRODUCTS  OF EDUCATIONAL SCHOLARSHIP
TheProducts of Educational Scholarship items are all quantitative and yield two summary index scores. Data may come from any EP domain. 		 
  • Scholarly Productivity Index
Index = sum of 5 subtotals for items below		 Calculate each subtotal and add the five subtotals to derive this index Scholarly Productivity Index =  
  • Peer reviewed publication (print or electronic)
 		 Journal impact factor score:
Low  <1= 1
Medium  1-2 = 3
High  >2= 10		 For each publication listed, multiply impact factor score by author score. Sum all of these to get Subtotal 1
SUBTOTAL 1= 		 
Author score: First/second or last author=3
Other=1
  • Peer reviewed/Invited presentations and workshops
 		 Presentation mode score:
Poster=1
Didactic presentation or workshop=2		 For each presentation listed, multiple mode score by geographic impact score. Sum all of these to get Subtotal 2
SUBTOTAL 2=		  
Geographic Impact Score:
Department=1
Institutional= 3
Regional= 5
National/International= 10
  • Non-peer-reviewed publication
 Author score:
First/second or last author publication= 1
Other=0		 Subtotal 3 is the number of non-peer-reviewed publications with author score of 1
SUBTOTAL 3= 		 
  • Book Chapter or Book
Book score:
Chapter author=1
Book editor =3
Book author= 5		 For each book related activity, multiply the number by the book score. Sum all of these to get Subtotal 4
SUBTOTAL 4=  		 
  • Educational Product
 Geographic Impact Score :
Department=1
Institutional= 3
Regional= 5
National/International= 10		 For each educational product, multiply the number by the geographic impact score. Sum all of these to get Subtotal 5
SUBTOTAL 5=		  
  • Educational Grants Index
 		 Score only grants for which author is the PI or Co-PI :
For each grant multiply the geographic impact score by the dollar score. Sum all of these to get the Educational Grants Index.		 Educational Grants Index =  
Geographic Impact Score:
Department=1
Institutional= 3
Regional= 5
National/International= 10
Direct costs:
>$500,000= 10
$100,000-500,000=5
$10-100,000=3
$1-10,000=1
 

 

Join APA

Donate

APA Journal

MailingList

ResidentFellow



RSS Feed



Follow Us on Twitter