Emzetpqota~ ] “Epgvvct / Operational Research. An International Journal. Vol. 7, No 1 (2007), pp. 47-59 Measuring student satisfaction with their studies in an International and European Studies Department G. V. Diamantis 1, V . K . B e n o s 2 University of Piraeus, 80 Karaoli & Dimitriou Street, 18534 Piraeus, Greece Abstract The satisfaction the students receive from their studies is of great importance. This paper considers ways through which the measurement of service quality could determine the status of a University department in International and European Studies (IES).
Student satisfaction is reliant on factors such as the curriculum, the range of the academic subjects taught, the academic staff training, the teaching materials, the social and intellectual experiences furnished by the institution. This research shows that the IES department enjoys a high rate of student satisfaction in comparison to the rest of the academic departments in the University and suggests some improvements and adjustments especially regarding the total number of contact hours per semester as well as the teaching methods used.
Keywords: Education, Quality Management, Satisfaction Measurement 1. Introduction The concept of quality is an important feature in life, in the sense that we are all clients and consumers of goods and services and desire to obtain the best possible “bargain” for the money and time we invest in obtaining them. This has led to quality management approaches, designed to produce products and services of good quality and to guarantee this quality for the consumer. It has also stimulated the development of methods of assessment and evaluation of the promised quality.
Language learning and education in general have also been affected. Education authorities are increasingly applying quality assurance and quality control techniques in the management of schools-universities and teaching programmes. Inspection systems are also being developed to promote methods by which high quality can be ensured. Elliott and Shin (2002) state that higher education is 1Department of International and European Studies, e-mail: [email protected] gr 2 Department of Statistics and Insurance Science 48 Operational Research. An International Journal / Vol. 7, No. 1 / January – April 2007 ncreasingly recognising the fact that it is a service industry and as such is placing greater emphasis on meeting the expectations and needs of students. Institutional evaluation represents one of the most modern and interesting issues of higher education systems. The need for such an assessment has been primarily prompted by the innovative nature and the restricted number of similar academic departments world-wide. The study by Politis and Siskos (2004) highlights the strengths and weaknesses that the department has recognised for itself and the effectiveness of the department’s strategic approach to quality management.
The available evidence demonstrates both the department’s capacity for critical selfevaluation and its willingness to act upon that self-evaluation. An analysis of the results sheds further light on the ways in which provision of quality service in universities can be affected by the perceptions the students have of the academic staff and their studies. Elliott and Shin (2002) have shown that student satisfaction has a positive impact on student motivation, student retention and recruiting efforts.
On the other hand Pickering (2004) and Stuart (2004) are arguing that there is a tendency to think that the most important thing is the feedback from the students, from the learners but that it is equally important to get feedback from the academic staff. Students’ satisfaction is of vital importance for every higher education institution because it (1) promotes its internal restructuring; (2) enhances its image; (3) focuses on student expectation and needs; (4) provides data which will assist the students performance in the labour market; and (5) operates as a bridge with other disciplines.
The aim of this paper is to present an original study of evaluating students’ satisfaction received from studies and services offered in an International and European Studies Department (IES) and to show how this evaluation study could improve the quality of education in this kind of studies. The paper is organised as follows: in section 2 a brief presentation of the IES Department is provided, section 3 presents the design of a student satisfaction evaluation study and the methodological issues raised, section 4 summarises and discusses the study results.
Finally in section 5 findings are cited and conclusions are drawn regarding long and short-term goals. 2. The philosophy of the International and European Studies in Greece The need for studies on International Relations and European Union Institutions has been growing in the last decade, having reached a peak after the recent accession of the 15 new member states [EUROSTAT (2005)]. One of the main challenges of the contemporary intemational, political and economic sphere and the process of European Integration, where Greece participates as an equal partner, is the emphatic demand for highly qualified graduates.
Such graduates should G. V. Diamantis, V. K. Benos / Measuring student satisfaction with their studies in an International and European Studies Department 49 possess the skills and knowledge required to occupy positions of responsibility within International and European Organisations, the Greek Public Administration as well as private enterprises, which are active in local and global market. In view of this, graduates need to develop a good command of English as an intemational medium of communication across cultures.
The IES Department we are studying in this paper was founded in 2000. The services provided by the department primarily focus on enabling students to obtain nationally and internationally recognised tertiary qualifications and credentials. The Department’s constitutional mission is the study and promotion of the social sciences from an Intemational and European perspective, by means of interdisciplinary teaching and research. It seeks to provide high quality and progressive study routes in political institutions and processes, international relations, and political thought, risis management and economics lIES (2005)]. The Department’s competitive advantage lies in the fact that its students have to engage in the contemporaneous study of three related academic disciplines or pillars of studies. In this way students develop a greater degree of specialisation, the intended outcomes of learning focus entirely on breadth of knowledge and understanding in each subject area. The three principal areas of concentration in the curriculum are the following: 1. International and European Institutions 2. International and European Politics 3.
International and European Economy The Department’s programme/syllabus acknowledges equal importance to the study of each of these academic disciplines and leads to the award of a single degree. 3. Designing a Students’ Satisfaction Evaluation System 3. 1 The Students’ Satisfaction Criteria The study presented through this section is extracted from the whole evaluation study outlined in Siskos et al. (2005). In order to assess the students’ satisfaction a survey should be conducted by emphasising on the satisfaction of students on several criteria which are considered as the components of students’ global satisfaction.
These criteria are presented in figure 1 and are also analysed in a number of sub-criteria to better explain the satisfaction or dissatisfaction of the students: 50 Operational Research. An International Journal / Vol. 7, No. I / January – April 2007 1. Education: it concerns the curriculum and the teaching quality factors 2. Infrastructure-tangibles: it refers to the university accommodation and technical equipment . Administrative Support: it considers the knowledge and the faculty of the personnel which works in the supportive services .
Image of the Academic Department: it refers to the reliability, prestige and the recognition of university. Figure 1. Hierarchical structure of student’s satisfaction criteria It should be pointed out that the IES Department has recently been granted official accreditation by ASEP (Higher Selection Council). This will further enhance the image of the department and render the degree gained by its students, even more broadly and extensively recognised in all sectors and different tiers of especially the state sector.
In that context the IES department has been trying to develop a strong and distinctive image. G. V. Diamantis, V. K. Benos / Measuring student satisfaction with their studies in an International and European Studies Department 51 Figure 2. A part of the questionnaire regarding Education Programme To collect feedback from students a simple and short questionnaire was drawn where each box in figure 1 corresponds to a question regarding to the student’s satisfaction judgements on the criteria and globally (a part of the questionnaire is presented in figure 2).
The questionnaire was distributed by students during the breaks of lectures for which the completion time varied from 10 to 12 minutes. 3. 2 Methodological issues In order to assess the student satisfaction the customer satisfaction evaluation methodology named MUSA (Multicriteria Satisfaction Analysis) was applied. MUSA follows the principles of ordinal regression analysis where a collective additive value function is inferred from satisfaction judgements expressed by a set of customers [Grigoroudis and Siskos (2002)].
Some of the characteristics of this method are the ease and simplicity of its application, previous successful outcomes, validity of results verified by standard statistical methods (fitting index, stability index, overall prediction level) and yield of important information about customer behaviour. The main advantage of the MUSA method is that it fully considers the qualitative form of customers’ judgements and preferences, as they are expressed in a customer satisfaction survey. This 52 Operational Research. An International J o u m a l / Vol. 7, No. 1 / January – April 2007 ethodology was successfully applied to a wide range of real world evaluation studies for various services [Grigoroudis and Siskos (2002)]. Further advantages of using this method is that we can estimate the following indices: – global satisfaction of the students in a range of 0% to 100%; criteria and sub-criteria satisfaction indices that show in a range of 0% to 100% the level of partial satisfaction of the students for each criterion/sub-criterion; – weights of criteria/sub-criteria which show the relative importance within a set of criteria or sub-criteria; – emanding level of students that takes values in [- 100% (not demanded at all), 100% (highly demanded)]. A questionnaire survey was conducted in the academic year 2002-2003. The sample selected via stratified random sampling represents 27% of the students population; 168 questionnaires were completed. 4. Results of the study 4. 1 The student satisfaction indices The MUSA system showed that the Department of International and European Studies enjoys a high global satisfaction rate, 89. 3% which is considered the highest in comparison to the rest eight departments of the university.
It should be mentioned at this point that the same survey was contacted to all nine departments of the University of Piraeus. All the four criteria which were used in the survey score high. In particular, the students appear to be fully satisfied with the quality of education offered to them, 92. 8%. On the other hand, the satisfaction levels on all four main criteria are also high, with the exception of Administrative Support which amounts to 71. 9 % (Figure 3). Indeed, it should be pointed out that not all satisfaction criteria carry the same weight for the students of the IES Department.
They consider Education the most significant one (41. 1%), the Image of the Academic Department follows with a percentage of 25% whereas Tangibles and the Administrative Support are regarded as less significant (Figure 4). Undoubtedly, the areas of Tangibles and Administrative Support need to be improved since according to the students’ view, their performance lies at a low level. These two criteria do not seem to be particularly critical at the time when the survey was carried out. However, one should not overlook them as it is very likely that in the future the students’ perception in relation to their importance may change.
G. V. Diamantis, V. K. Benos / Measuring student satisfaction with their studies in an International and European Studies Department 53 It should be pointed out that most of the information in this part is based on the third and fourth year students’ sharing of their views on the quality of the education they receive. Figure 3. Satisfaction Criteria lndices Figure 4. Criteria weighting All four criteria show very low demanding level (Figure 5). The students seem to be less demanded for Education and Image of the Academic Department. 54 Operational Research.
An International Journal / Vol. 7, No. 1 / January April 2007 Table 1 summarises all the satisfaction indices obtained by MUSA system by inferring satisfaction on criteria from satisfaction judgements on the corresponding sub-criteria. Concerning the quality of education perceived by the students, it should be pointed out that there is room for improvement particularly as far as the teaching hours per semester which the students consider as not satisfactory (rate of satisfaction 54%); the teaching methods which the students think do not really motivate them (59. %); and the courses overlap which decreases their satisfaction because of the quality of education they receive (54%). Teaching is carried out within the traditional framework of a weekly lecture and seminar. However, more emphasis should be placed on interactive learning especially as far as English Language and Terminology is concerned. This will be possible and facilitated by the size of the seminar groups of approximately 20 students and variation of the input process. Figure 5.
Demanding Levels G. V. Diamantis, V. K. Benos / Measuring student satisfaction with their studies in an International and European Studies Department 55 Table 1: Student” Satisfaction Indices for sub-criteria Education-Sub criteria Courses variety Teaching hours Educational material Course overlap Career orientation Professors knowledge Teaching methods Professors” communicative skills Professors availability Evaluation methods 87. 1% 54. 0% 80. 4% 54. 0% 73. 8% 85. 1% 59. 4% 64. 7% 76. 2% 72. 6 %
Tangibles-Sub criteria University facilities Printed material Laboratory equipment Laboratory opening hours Library sufficiency Library opening hours Library Electronic System Lending procedures Library spaces 51. 7% 72. 1% 52. 2% 43. 0% 54. 3% 82. 9% 62. 9 % 72. 5% 69. 6% Administrative-Support Personnel knowledge Reliability Speed Friendliness 79. 4% 94. 4% 66. 1% 66. 7% Image and reputation of the Academic Dept. Expectations 90. 6% Job market opinion 95. 1% Promotion activities 71. 1% Liaising with job 80. 3 % market
Because of the nature of the departmental subject area there are some laboratories where the student are trained in computer skills and have access to the web. On the other hand, library facilities are more important for them which justifies the apparent imbalance compared with laboratory. As far as the university tangibles are concerned students satisfaction is high regarding Library opening hours (82. 9%) in contrast to the Laboratory opening hours (43%). Subject books and lecture notes, which are distributed free of charge to all students, are useful and appropriate. Periodical stocks are appropriate and accessible.
In contrast to the initial three dimensions namely, education, tangibles and image of the department Administrative Support lies at a lower level (71. 9%), compared to the other services of the department and the university in general. This is due to the low rate of speed (66. 1%) and the low rate of friendliness (66. 7%), the way they are served or treated by the staff employed in different areas of the university, the department secretary, the workshops and the like. By contrast, the services provided to the students by the university support staff are considered highly reliable (satisfaction rate 94. %). 56 Operational Research. An International Journal / Vol. 7, No. 1 / January – April 2007 Both the image and reputation of the academic department, which contribute to its competitive character, rate high at 89. 2% (Figure 3). It should be mentioned that the student expectations from the institution were met to a high degree indicated by the satisfaction rate of 90. 6% in this particular sub-criterion (table 1). However, the students state that there is still room for some improvement regarding the procedures of the university’s rojected public image through promotion activities (71. 1%). Figure 6. Performance~Importancediagram (relative) 4. 2 Students ‘practice in the labour market. The students’ major concern is their employability after graduation; they wonder whether the knowledge they will gain during their studies will assist them in developing their careers. Some of the students will opt for non-academic careers (e. g. government, international organisations, business and media) as articulate, clearthinking individuals, able to critically analyse complex issues.
A large percentage of third and fourth year students have been very keen on doing a two month practice in an institution or ministry related to their area of studies such as the Ministry of Foreign Affairs, Secretary General for European Affairs Directorate for European Countries, Directorate for Common Foreign and Security Policy, Diplomatic Academy, the Ministry of Press, the Ministry of Culture and the European Union. It should be pointed out that doing hands on practice is neither compulsory nor a required degree component. The percentage of the students which have already done G. V. Diamantis, V. K.
Benos / Measuring student satisfaction with their studies in an International and European Studies Department 57 a two month practice in the labour market amounts to 65%. Students’ response has been this positive due to the fact that such practice is an asset because it is regarded as work experience. It is hoped that in the future the students of the Department will be able to do their hands on practice in the private sector as paid trainee professionals. 5. Recommendations, New Research Avenues and Conclusion. The IES department claims to be providing an excellent education in European Union Institutions and International Relations.
The assessors come to the overall conclusion, based on the students’ questionnaire and an analysis of self-assessment, together with the statistical indicators, that the quality of education offered at the department is excellent. Both students and the academic staff agreed that the results of the assessment should be acted upon. The increasing competition between universities internationally, due to new technology and the spread of translational agreements between universities, has led to fierce competition for students/customers, teaching and research staff.
Although some universities choose not to disclose information that affects their strategic position that would reveal their strengths and weaknesses to their competitors [Landrum et al. , (1998); Palacio et al. , (2002)], the IES Department, true to its principles, is willing to share information and knowledge in its quest for excellence. The positive features which support this assessment include (1) the high-quality and academically challenging content of most teaching sessions; and (2) the care and attention given by the teaching staff to students in helping them to acquire academic and applied skills.
Three points which require further attention to overcome a number of weaknesses are listed below: 1. Essential matters which are currently putting academic standards and/or quality at risk, and which require urgent corrective action: fourth year students should undertake compulsory dissertation which is a key means by which students can experience a greater depth of study and learning as well as to demonstrate their ability for independent study. 2.
Advisable matters which have the potential to put academic standards and/or quality at risk and which require either preventive, or less urgent corrective action. Although the specified aim of providing high quality and progressive study routes in the three pathways/pillars is met suggestions address the need for students to follow coherent pathways in their third year of studies. 3. Desirable matters which can potentially enhance quality and/or further secure academic standards. 58 Operational Research. An International Journal / Vol. , No. l / January April 2007 As a direction for future research, more extensive studies should be carried out to examine other factors, which affect the level of satisfaction of students such as comparing and contrasting different departments of similar subject area. Universities and individuals need to improve their performance in order to respond to current challenges. Various inspection schemes can provide a valuable framework to help universities improve communication, productivity, effectiveness and to achieve strategic goals.
However, an ongoing cycle of inspection would be advisable. In practical terms students could be involved in internal audits twice in the course of their studies which is reasonable and manageable. Surveys of this nature normally yield instrumental and reliable results when conducted at regular intervals. Areas for improvement are highlighted below, identified through the review of the assessment and the suggestions made by the students. The need for support and guidance which can lead to autonomous or independent study was one of the suggested areas.
The policy of an IES department for quality assurance, and the measures it has promoted to implement that strategy, empower it, now and in the foreseeable future, to manage its responsibilities for quality and academic standards effectively. The evidence indicates that greater care needs to be taken to ensure that written examination are accompanied by coursework assignments. The role of student-customer is different in every single university. He/she cannot be considered as a passive customer who unquestioningly receives the service which is offered to him/her.
On the contrary he/she participates actively in the formation and improvement of this service [Helms and Key, (1994)]. Discussions with the students yielded the following comments: (1) Although students possess the ability to design and independently execute a “long essay/dissertation” on an approved subject of their own choice, within their field, this is not officially recognised as a requirement in their course of study; (2) The academic staff should discuss student achievement with students; and (3) Timetables are rather heavy.
Although the quality of student progression is commendable, however, with respect to academic standards, there are serious concerns over the quality and academic standards associated with all subjects contributing to the degree. Each academic subject should aim to enable students to acquire subject-specific knowledge and skills. Students highly appreciate the approachability and accessibility of teaching staff and the additional support services provided by the department. The outcomes of this survey demonstrate the important educational gains which are achieved by the quality of education already offered in the IES department.
In the future, the teaching staff of the department is likely to increase its demands for higher levels of faculty performance and accountability. An established and principled assessment procedure which yields such essential and reliable results will inevitably lead to the enhanced teaching and learning effectiveness of the department. It is hoped that what has been suggested in this paper will stimulate further thinking about the complexities of the topic and will evoke further discussion on this significant academic issue.