Like many colleges and universities, the University of Southwestern Louisiana (USL) has employed a means for assessing the productivity of its faculty for several years. In this system, each member of the faculty must complete a three-page "Faculty Self-Evaluation of Performance Form" annually (see Appendix 1). The completed form, together with a condensed second form provided by the department chairperson (see Appendix 2), is then subject to appraisal by successive levels of administration. Faculty are ranked according to the perception of productivity therein conveyed, and the results are used as the basis for merit pay decisions.

Methods designed to assess faculty productivity reflect the priorities of their formulators. This fact, when coupled with the difficulties inherent in rating the quality of work performed by individuals sharing little other than the generic designation "university faculty member," has made the perception of inequitable treatment a criticism leveled at many attempts at faculty evaluation. The USL system is representative. For example, one of the common faculty criticisms of evaluation on this campus is that it perpetuates the infamous teaching/scholarship paradox, namely, that universities hire professors to teach (i.e., for their ability to disseminate knowledge); but retain and reward them almost exclusively on the basis of their scholarship (i.e., for their ability to advance knowledge).

The existence of this paradox is not immediately apparent from an examination of the USL evaluation forms themselves. These documents are designed to rate an individual's yearly production in four major areas: teaching, research, university service, and community service. Although the form to be completed by individual faculty does not stress the relative importance of each category, that perspective is evident on the condensed version submitted by department heads (see Appendix 2). The two categories of university and community service play a relatively minor role in the evaluation process. Together they account for a maximum of only twenty percent of a faculty member's total "score." The remaining eighty percent can either be divided equally (40/40) between the teaching and research categories, or weighted in favor of teaching (60/20) for those viewing the classroom as their primary charge. Thus it would appear that this system places equal or greater emphasis upon teaching vis-a-vis research-related activities.

In fact, the relative abilities of university faculty in areas of disseminating information (teaching) have historically defied effective measurement and hence have proven to be of limited value in merit evaluations of this type. The ubiquitous "student evaluation of instruction," although commonly used in this context, is a controversial tool at best. Indeed, some faculty complain that administrators limit its use to a kind of negative reinforcement. Thus while instructors have speculated that a poor evaluation by the students in their classes might be used to support an administrative decision to refuse tenure or promotion or to award a low merit raise, the belief that a glowing evaluation will carry a correspondingly positive effect is truly rare.

It can be argued that the most accurate method of measuring a teacher's competence is achieved by examining the preparation of his students. However, assessments of this type are a long-term process; the yearly merit evaluation, on the other hand, is clearly a measure of short-term success. That no generally accepted method for determining teaching effectiveness exists must rank as the principal irony of the profession.

If the problems associated with an institutional inability accurately to measure teaching effectiveness are indeed universal, then it is difficult to dispute the thesis that professional status for the university professor generally accrues through an assessment of contributions in the areas of research and publication. These are, after all, visible and measurable contributions to both knowledge and the profession. Ideally then, status can be determined by other professionals who review and evaluate these activities. Thus, it is specifically because faculty research and publication are perceived by administrators as being measurable—and thus that such contributions of individual faculty members can be compared—that it has become the de facto standard for faculty evaluation at USL and many other universities.

Any merit evaluation that stresses research and publication will pose serious problems for that segment of the university community which counts a significant portion of its productivity in terms of "creative activities." Although the "visibility" of such activities is obvious, these faculty members have suffered from the same lack of measurability for their specialties that affect the pure teachers on university faculties. Indeed, because of their teaching function, such individuals often feel doubly faulted. Departments of applied arts, dance, and theatre are affected. Music faculty members, especially those employed in applied teaching and performing, have always felt frustrated at this state of affairs. In addition to the universal problems of a teaching faculty confronted with a system that cannot determine teaching competence, applied instructors also suffer from the perception that they are a high-cost/low-yield commodity. Further, although the "creative activities" of these individuals might well be recognized within the department, such faculty believe that their accomplishments are likely to receive less weight than the more easily quantifiable research and publications of their colleagues in other disciplines.

At USL this concern would seem to be corroborated by the formal organization of the faculty evaluation itself. Significantly, this form (Appendix 1) contains no category suited to the listing of compositions, conducting appearances, or musical performances. Such activities must be listed, somewhat apologetically, in section d of the Research category (2). The inference by applied music faculty that they are dealing with an unfavorable set of administrative priorities might not be unreasonable.

If the USL merit system can be considered typical, then a majority of faculty complaints are incurred when the recommendations of departmental administrators must be altered at a higher level. The USL system operates on a fixed set of percentages. Only a certain percentage of the faculty, university-wide, can receive the higher merit categories. Therefore department recommendations do not always prevail; each step higher in administrative evaluation can result in "adjustments" (category reductions) based upon these fixed percentages. In a real sense, then, faculty from many different departments with disparate job descriptions will find themselves competing against one another. Given this state of affairs, it is not surprising that the creative artists can feel disadvantaged. Naturally, the sense that their efforts have gone largely unappreciated by university administrators possessing little knowledge of their specialties affects the morale of faculty conductors, composers, and performers.

In an effort to rectify this situation, USL is presently experimenting with a system designed to equate "performance/creative activity" in music to the traditional scholarly pursuits. The goal is to provide university administrators with a simple means of assessing such "creative" pursuits and, at the same time, to offer individuals some assurance that their efforts will be evaluated objectively.

The preliminary draft of the new USL system for equating creative performance has been provided in Appendix 3A. Page one of this document provides a graduated comparison of the traditional research categories of publication and verbal presentation with that which is so important to faculty composers and performers. The attempt here is to equate a variety of creative activities to those types of research with which university administrators are much more familiar. Thus numerical values are assigned to activities grouped in six steps within each of the three basic categories. Note that this rating system recognizes no activity under creative performance rated as equal to publishing a book or a lengthy monograph.

It should be stressed that the values assigned to activities in each of the three categories reflect the priorities of the USL music faculty. However, these assigned values are in no sense inviolate. Other departments might well find adjustments within and among categories to be necessary. For example, it can be argued that certain types of composition should qualify in one of the higher categories—certainly departments which count nationally or internationally recognized composers on their faculty might find a revised distribution appropriate.

Two principal considerations were used in the formulation of this experiment: contribution/value to the profession and research/preparation time required to accomplish the project. However, it was apparent from the outset that several additional factors had also to be addressed. Page 2 of the university document identifies five qualifying factors, together with a second, graduated numerical system intended to be used with the chart on the preceding page (see Appendix 3B). The system would function as in the following examples:

A. A faculty violinist presents a full solo recital in the university's recital hall. The material performed is newly prepared (that is, it is new material to this performer). The performance is not remunerated and is not reviewed. This faculty member is assigned load reduction to compensate for his performance activities.

1. Solo recital—basic value			8
2. Local audience (A)*			+ 1
3. (New) Material			+ 1
4. Not reviewed			- 1
5. Reduced load credit			- 3
6. Unpaid			+ 1
	Assigned Value		7	points
*qualifying factor

 

B. A faculty theorist presents a research paper at the annual convention of his professional society. This is a regional convention and the paper involves a condensation of a portion of his dissertation. A screening/selection committee chose his paper; he receives no load reduction and no remuneration for this activity.

1. Presentor/Professional Society			6
2. Regional audience			+ 3
3. Not new research			- 1
4. Screened paper			+ 1
5. No reduced load			+ 1
6. No remuneration			+ 1
	Assigned Credit		11

At least two of the five qualifying factors already have engendered lively debate. Most questions focusing upon the question of new music/research have been resolved through a clarification of the terminology. The intention here is to offer both creative and research-oriented faculty the incentive to expand their professional experience. Hence, although it is possible to argue that any performance is a new contribution, it is also reasonable to assume that a faculty member's performance of works that he has played many times before should not receive the same credit as a performance of compositions that, because they are new to the artist, demand considerably more time to prepare. Of course the same criteria can be applied to a research-oriented member of the faculty.

The question of remuneration has been a particularly difficult one. Professionals expect to be paid for their services and, in fact, have been conditioned to expect such payments to escalate as their reputation increases. In a real sense, then, the willingness of outside organizations to pay for the services of a member of its faculty could be viewed by the university as a gauge of that individual's value to the profession. Thus it has been suggested that rather than penalizing an individual for receiving a fee for his services, the opposite should be true. However, there is another viewpoint that holds that by basing yearly merit increases upon remunerated services a university would find itself supporting a kind of professional double-dipping. Then, too, it is possible that faculty so encouraged will eschew other types of services, e.g., local or university recitals and lectures. Although these traditionally do not carry separate remuneration, they are often considered to be of more immediate value to certain types of institutions. Clearly the decision whether to stress remunerated service must depend in part upon the perceived role of the university itself. USL, as the largest of the "regional" universities in the Louisiana State System and the second largest university in the state (after Louisiana State University), finds itself in a somewhat ambiguous position. Probably because of its regional role, the administration has taken a position discouraging extra-university remunerated activities under most circumstances. This in turn influenced the position taken on this issue in the present experiment.

It is of course possible to question the value of any system based upon rating the productivity of one faculty member against that of another. However, it is unlikely that such objections will alter significantly the current vogue for evaluative efforts. Given the fact that these systems will very likely continue to be used, the question then becomes one of educating those in charge of the process to specific inequities as they are uncovered.

In this regard, it cannot be stressed too strongly that this experiment was never intended to replace the judgment of that university official who is closest to the faculty, both by virtue of his specific academic training and the nature of his assigned duties, i.e., the department chairman. Rather it is aimed higher, at those administrators for whom the music faculty and its unique problems may constitute only a minor and sometimes imperfectly understood portion of their responsibilities. As such, its singular objective is to provide that "measurability" required for these individuals to equate the activities of the creative musician with those of his research-oriented colleagues both in and out of his department.

It is expected that the particulars of this approach will continue to be the subject of lively debate and continuing detail adjustments. The range of departmental priorities at other universities might well serve to test the flexibility of the plan. Surely the prospect of academic equality for a long-neglected segment of the university community is well worth the effort.

 

APPENDIX 1

The University of Southwestern Louisiana
Faculty Self-Evaluation of Performance Form

ACTIVITIES AND ACCOMPLISHMENTS FROM MARCH 1, 1983 TO MARCH 1, 1984

Instructions: To be filled out in triplicate by each faculty member. Please attach additional pages as necessary. Completed forms must be turned in to the Dean's office by the Department Head no later than April 13, 1984.

NAME________________________________________ RANK_________________________________

DEPARTMENT__________________________________ SOCIAL SECURITY NO._____________________

 

1. Teaching

a. UNDERGRADUATE		CREDIT HOURS	CONTACT HOURS	NO. OF STUDENTS	NO. OF PREPARATIONS
	1. Teaching Load for Fall Semester:	_______	_______	_______	_______
	2. Teaching Load for Spring Semester:	_______	_______	_______	_______
b. GRADUATE
	1. Teaching Load for Fall Semester:	_______	_______	_______	_______
	2. Teaching Load for Spring Semester:	_______	_______	_______	_______
	3. Thesis and/or Dissertation Advisor:

_______(Master's)	_______(Thesis)	_______(Master's)
_______(Ph.D.)	_______(Dissertation)	_______(Ph.D.)
Number and Level of Students	Total Credit Hours of Thesis/Dissertation Scheduled	Total Contact Hours Per Week With Students

c. Released Time? ______________ For what purpose? ______________________________________
_________________________________________________________________________________

d. Average hours per week available for student consultation: ____________________________________

e. List students whose graduate school committee you have served on during the period covered by this report. Indicate whether you were chairman or committee member and whether or not the student completed degree requirements during the time interval.

Student Level

Committee Assignment

Did Student Finish Work

Thesis or Dissertation

Student Name	M	Ed.S.	Ph.D.	Member	Chairman	Yes	No	Yes	No

f. List teaching innovations which you have made during this period, new methods introduced in your courses, new courses developed, new programs introduced, etc.

 

2. Research

a. Are you a member of the Graduate Faculty? Yes _______ No _______

b. List in formal bibliographical style all articles or books which have been published or accepted for publication during the time period covered by this report.

c. List all papers delivered at professional meetings during the time period covered by this report in formal bibliographical style.

d. List other research activities such as exhibits, on-going research projects, new research projects, etc., in which you participated during the time period covered by this report.

e. List other professional activities for this time period including committee assignments and offices held in national, regional, state and local professional organizations. List all professional meetings attended and give dates.

f. List all grant proposals written and submitted to any agency during this time period (include agency or foundation name, title of proposal, date of submission, whether pending, awarded, or denied).

 

3. University Service

a. List assigned formal advising duties and number of advisees assigned to you:

1. In the last summer session
2. In the last fall semester
3. In the current spring semester

b. List committee assignments (departmental, college, and university).

c. List programs in which you have participated, such as Career Day, workshops for area teachers, donated on-campus consulting, etc.

d. List equipment, grants, or scholarships which you were instrumental in securing for the benefit of the University's educational program.

e. List all other activities which you construe to be University service (informal advising, active service in the Faculty Senate, etc.).

 

4. Community Service

a. List activities that you consider a contribution to the welfare of the community (participation in service and philanthropic organizations, contributed off-campus consulting, appointments on city commissions, etc.).

APPENDIX 2

The University of Southwestern Louisiana
Annual Performance Evaluation Form

DEPARTMENT HEADS' CONDENSED EVALUATION FORM

NAME________________________________________ RANK_________________________________

DEPARTMENT__________________________________ COLLEGE_______________________________

Instructions: Each faculty member is to be rated 0 through 10 (0 representing Unsatisfactory and 10 representing Outstanding) in each category for the time period of March 1, 1985 to March 1, 1986. Circle a numerical rating in each category and use this rating to calculate a score. The total score is the sum of these individual scores. Submit this form in triplicate to your Academic Dean (or appropriate Director) by April 11, 1986.

1. Teaching

Justification

10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0

(Attach extra sheets as necessary)

Lectures, demonstrations, innovations, use of teaching aids, class attendance, student relations, student contact, graduate school committees.

______ Rating

×

(4 or 6) Factor

=

______ Score

2. Research & Professional Activities

10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0

Publications, papers read, textbooks, professional activities, grants, ongoing projects.

______ Rating

×

(4 or 2) Factor

=

______ Score

3. Formal Advising & University Service

10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0

Assigned advising, extra responsibilities, self-improvement, recruitment, contributions to student life, committee assignments, participation in programs to serve the diversified clientele of the University.

______ Rating

×

1.5 Factor

=

______ Score

4. Community Service Outside University

10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0

Service organizations, philanthropic endeavors, contributed services.

______ Rating

×

0.5 Factor

=

______ Score

5. Total Score

______ 100.0

Raise Recommendations

Approved Signatures

Priority Within Category Categories I, II, III*

DEPT. HEAD OR DIRECTOR _________________________________________________________________________________

ACADEMIC DEAN ________________________________________________________________________________________

ACADEMIC VICE PRESIDENT ________________________________________________________________________________

Acknowledgement

This will certify that I have inspected this copy of my Performance Evaluation after it has been completed by my Department Head or Director (including assignment of a "Category" and "Priority," if applicable). My signature below does not imply my concurrence with this evaluation, but merely indicates that I have inspected the completed form.

EMPLOYEE SIGNATURE ______________________________________________________________

 

Information Concerning Annual Performance Evaluation Form

I. Calculating the "SCORE" of a faculty member

The score in each evaluation category is determined by multiplying a "Rating" by a "Factor."

A. The "Rating" in each evaluation category is expressed with a number which ranges from 0 to 10. Zero implies an unsatisfactory rating and 10 means an outstanding rating.

B. The "Factor" in each evaluation category is to be a number which expresses the relative importance given to this particular category in the overall rating scheme. There are two sets of factors which may be used:

EVALUATION CATEGORY	THOSE FACULTY WHO ARE STRONGLY RESEARCH ORIENTED	THOSE FACULTY WHO ARE STRONGLY TEACHING ORIENTED
Teaching Research Formal Advising & University Service Community Service	4.0 4.0 1.5 0.5	6.0 2.0 1.5 0.5

C. The "TOTAL SCORE" is obtained by adding the individual score in the four evaluation categories. The system is arranged so that the maximum score possible is 100. Consequently, within each department, the total score will serve to define the relative placement of a faculty member. The "TOTAL SCORE" has no meaning when compared across departmental or college lines.

II. JUSTIFICATIONS should be reasonably complete. Additional sheets should be attached as needed.

III. The number of raise categories for which this evaluation is to be used as a placement vehicle is six (6).

A.	Category I Distinguished		—Faculty member doing an excellent, truly outstanding job in all areas of his/her responsibilities when judged by job description and standards of an ideal college professor (see Faculty Handbook). This category should be limited to a maximum of 5-10% of your faculty unless you and your Dean agree that there are some unusual circumstances in your Department.
B.	Category II Commendable		—Faculty member doing an above average, very good job when judged against his/her job description and standards of an ideal college professor. Categories I and II should normally be limited to a combined maximum of 25% of your faculty.
C.	Category III* Very Good		—Faculty member doing a very good job as judged against his/her job description and the standards of an ideal college professor (limit of approximately 25% of your faculty in this category).
D.	Category III Good		—Faculty member doing a good, satisfactory job as judged against his/her job description and the standards of an ideal college professor. (No percentage limits in this category.)
E.	Category IV Adequate		—Faculty member doing a barely adequate job when judged by his/ her job description and the standards of an ideal college professor. This category rating signifies a need for improvement of his/her performance. (No percentage limits in this category.)
F.	Category V Inadequate		—Faculty member not performing the job expected of USL faculty when judged against his/her job description and the standards of an ideal college professor. This category signifies a "zero" raise. (No percentage limits in this category.)

Priorities for Categories III, IV, and V should not be assigned under this system. However, Categories I, II, and III* recommendations must be placed in priority, i.e.

a.	Category	Priority		b.	Category	Priority		c.	Category	Priority
	I	3			II	2			III*	7

Dollar values cannot be placed on each Category at this time (except Category V which is zero raise) since USL's 1986-87 budget has not been finalized. When the Legislature has passed our budget (sometime in July 1986), and providing that budget contains sufficient funds for merit raises, numerical dollar values will be assigned to Categories I, II, III*, and IV. Additionally, merit raise amounts for this year may be subject to effects of any across-the-board raises voted by the Legislature.

APPENDIX 3A

A Comparison of The Three Basic Types of Research-Related Activities
Presented as A Plan to Obtain An Objective Rating System for The USL School of Music Faculty

---

PUBLICATION

VERBAL PRESENTATION

CREATIVE PERFORMANCE

NUMERICAL VALUE

---

A.	Book	A.	N/A	A.	N/A	A.	12
B.	Monograph	B.	N/A	B.	N/A	B.	10
C.	1. Refereed Scholarly Article/Periodical	C.	Featured (sole) speaker on a recognized scholarly series (or equivalent)	C.	1. Full Solo Recital	C.	8
	2. Major editorial duties on an ongoing basis				2. Published musical composition (major work)		8
D.	Full length/substantial articles in non-refereed professional journals	D.	Presentor/Professional Society	D.	1. Full chamber recital	D.	6
					2. Major conducting responsibilities		6
					3. Partial solo performance		5/4
E.	Abstracts, reviews, summaries in professional journals	E.	1. Presentor/General Audience 2. Respondent or Panelist 3. Moderator of a session	E.	Published composition (minor work)	E.	4
F.	Non-professional publications	F.	N/A	F.	1. Other performances 2. Unpublished but performed musical compositions	F.	2

---

APPENDIX 3B

Qualifying Factors to Determine a Numerical Value
for Research/ Verbal Presentation/Creative Performance Activities

A. For what type of audience is this presentation/performance intended?

1. Local 2. State 3. Regional 4. National

(+ 1) (+ 2) (+ 3) (+ 4)

B. Does this presentation or performance involve essentially new research or music (as determined by whether or not the material has been published/presented elsewhere)?

Yes No

(+ 1) (- 1)

C. Prior to the presentation/performance, what means were employed to assure the quality of the product?

1. Was this presentation refereed, i.e., was an abstract or the complete text required in advance? If a musical performance, was an audition required or was the performance screened or reviewed in any way? (specify)

Yes No

(+ 1) (- 1)

D. Has the teaching load been reduced a credit for involvement in either research or performance-related activities?

Yes No

(- 3) (+ 1)

E. Was remuneration received for this project/performance?

Yes No

(- 1) (+ 1)

Under most circumstances a paid project/performance will receive considerably less credit than a similar project for which no payment is received. Exceptions to this policy may be warranted; however, to qualify for an exception, it is the faculty member's responsibility to state his case in writing.

An example of the above system: A faculty violinist presents a full solo recital in the University Recital Hall of all new material. No pay was involved and the performance was not screened. This faculty member is being assigned a reduced load to compensate for his performance activities.

1. Solo recital: 2. Local audience: 3. New material: 4. Not screened: 5. Reduced load: 6. Unpaid performance:	+ 8 + 1 + 1 - 1 - 3 + 1	points
TOTAL	7	points

These guidelines will allow an objective rating to be assigned to all School of Music faculty members engaged in creative performance and can then be transferred to any system that is devised by the University to give credit for research-related activities. This system as proposed represents a first attempt at an objective evaluation system for those faculty members whose skills are related to the creative arts. It must also be understood that special circumstances will occur from time to time that will cause the above factors to be adjusted by the person making the evaluation.