• ACADEMIS
• Events
• ARACIS

Selected examples of the deliverables from this activity are the following:

 

• Document: “Proposal Concerning Benchmarking Indicators” (Romanian version), year I of the project. After a research on benchmarking and performance standards and indicators, the team of experts in this work package compared the methodology currently in use with the research findings (other European systems etc.). The objective of the comparison was the elaboration of a draft set of general and institution-specific indicators.

• Document: “Primary and Secondary Indicators for Quality Evaluation”, year I of the project. The document was designed to invite the interested parties to a debate on the newly proposed primary and secondary indicators, in light of the research carried out during the project’s first year. A Romanian version of the document is also available (“Indicatori primari și secundari pentru evaluarea calității”).

• Document: “Bilingual (Romanian-English) Glossary of Terms for Higher Education”, year I of the project. The document contains an updated selection and adaptation of academia-specific terms, especially as employed by specialized international bodies and agencies in the field.

• Document: “A proposal for a new Visit Record”, year I and II of the project. The Visit Record kept the three dimensions of quality evaluation established by law (institutional capacity, educational efficacy and quality management), with the needed modifications concerning the old indicators. Thus, the output and outcome indicators gained more importance; the vague and unprecise nature of some indicators was eliminated; some indicators were (re)phrased for better quantification; great attention was given on collecting comparable data; new emphasis was placed on quality management. When applied in different universities evaluated within the project, the Visit Record was constantly amended in accordance with received feed-back.

• Document: “A proposal for an Unitary Discipline Record” (with guidelines for filling in) and “a proposal for a Program Record” (also with specific guidelines for filling in). The documents were elaborated in collaboration with ACPART, taking into consideration the need to correlate the institutional and tertiary indicators with the competencies developed by this particular agency

• Special journal issue: A special issue of the bi-annual journal Quality Assurance Review for Higher Education, year II of the project. This particular issue contains a themed section on the „Analytical Approaches of the Quality Assurance in Higher Education Methods and Instruments”, an articles section on „Community Development”, as well as a review and a series of announcements on national and international conferenes.

• Study Chapter: A chapter within the study the Quality Barometer 2010: The State of Quality in Romanian Higher Education System, entitled „University Practices for Ensuring and Evaluating Higher Education Quality”. This study contains: an analysis of institutional diversity and its desirability, different types of institutional diversity being presented; an analysis of Romanian institutional diversity in higher education; an approach to university autonomy, institutional homogenity and quality assurance; considerations on the need to increase diversity within the higher education system and benchmarking practices for quality ensurance of university programs; an analysis of quality ensurance as it is entailed by institutional benchmarking practices, etc.

• Evaluation Methodology Proposal: A Methodology Proposal for external evaluation for provisional authorization, acreditation and periodical evaluation of PhD Schools, year III of the project.

• Indicators Methodology Proposal: A Proposal for a Methodology for the Construction of New Reference Indicators, year III of the project. The Report, mainly descriptive, offers information on the asumptions and philosophy on which the Methodology for the Construction of New Reference Indicators is based, proposed for the external quality evaluation in Romanian higher education. Also, the document highlights the inferential connection between the nature of the newly proposed reference indicators and the possibility of using them in organizational procedures and practices (ex. Benchmarking exercises). In the context of different systematic empirical data collecting practices at the university level in the Romanian higher education system, the document underlines the complementary relation between ARACIS’ work in the external measuring of quality assurance in higher education and other similar national work, as well on the classification of universities and study programs ranking or the evaluation of scientific research quality.

 

Within this sub-activity the new proposals for general indicators were analyzed and new proposals for their modification were developed, during year I of the project. Also, primary lists of reference indicators were elaborated for the 10 fields of study selected. The work of the experts from these commissions formed the bases of the following sub-activity “Piloting the Selected Indicators and Formulating Performance Indicators”.

The supporting documents for the activities of the expert commissions are:

• Implementing the new reference indicators

• A description of tertiary indicators

• The credit system in Higher Education

 

Within this sub-activity:

 

- The application of the Proposal for a New Visit Record on externally evaluated universities during this time-frame was piloted;

- The following documents were perfected: the lists of tertiary indicators, as well as the distinct sections of the Visit Record (on the normative mandatory requirements - the performance standards and indicators) for the authorization and accreditation of study programs, piloting their application to different study programs levels.

The perfecting of the Visit Record for the authorization/accreditation of study programs was based on a selection from proposed experimental indicators within the project, indicators proposed for use for elaborating a Visit Record for evaluation in institutional accreditation. The selection was undertaken starting from the naming of current indicators in ARACIS Methodology for institutional evaluation. It has been suggested to add other indicators or to complete the current list with tertiary indicators, specific to study fields.

The perfecting of tertiary indicators aimed to collect data considered relevant for each study field, which could be objectively measured and that would eventually result in a clear evaluation of a certain study field. At the same time, a correlation was envisioned between tertiary and primary and secondary indicators, a correlation needed to insure the internal coherence of the evaluation process. Also, this perfecting activity sought to establish a concordance between indicators and the achieved competences after completing a study program.

After the pilot phase and receiving feed-back, a Final Report on the Elaboration and Piloting of Tertiary Indicators was elaborated for each commission of a field study (the reports contained final proposals for the programs authorization/accreditation record). Each of these documents followed an established pattern, with the following structure:

- General Considerations: the main vulnerable points identified for specific standards proposed for modifications; the main novelties introduced by the commission, through tertiary indicators, on the evaluation of study programs relating to approached study fields;

- The Feedback Analysis (presenting the main observations on the piloting of the Tertiary Indicators Record, for undergraduate and Masters levels, focusing on the variables which posed measuring problems; debating the reactions received from universities where the Tertiary Indicators Record was piloted);

- Final Considerations on the evaluation and assurance of quality (underlining the limits of the proposed tertiary indicators; it approaches different issues related to the quality of university study programs (for each approached study domain) that the commission feels would be left out of the reach of proposed tertiary indicators; it refers to the main envisioned difficulties in applying the new tertiary indicators; it approaches the process of evaluation and quality insurance in the context of the National Education Law no.1/ 2011 and of the National Evaluation Exercise for the classification of universities and ranking of study programs).

At the end, a Final Report was put together, containing the recommendations and annexes with the analyzed domains. The Report unites and synthesizes the main recommendations from the expert commissions reunited within the project. The presentation of the recommendation was structured in such a way that it would cover both the limits of the sets of tertiary specific indicators and their correspondence with the institutional indicators system. Also the report presents in table form the main feedback categories received from university study programs where the tertiary indicators had been simulated.

 

Within this sub-activity from the last year of the project, a benchmarking Report was elaborated, containing the analysis of variation spaces for the proposed and validated indicators.

The document reports on the results of the first exercise of benchmarking on data bases ever implemented in higher education in Romania.

The results serve multiple purposes:

a) they offer a general picture of the institutional profile already acknowledged by more than 40 public and private universities;

b) they provide information on the way the values recorded at the university level varies in different spaces associated with a rather wide range of variables and indicators within the quality evaluation process;

c) they offer a procedure that the universities from within the higher education system can use to project personalized strategies for bettering the quality of the higher education they provide.

The empirical data from this report were collected using an Institutional Visit Record with three categories of variables (process, input and output). These variables were obtained through the operationalization of standards of quality insurance present in the current Evaluation Methodology of ARACIS.

• About ACADEMIS
• Project Management
• Autoevaluation and external evaluation
• Quality assurance of higher education
• Benchmarks
• Evaluators’ training
• Promotion, Dissemination, Transfer
• Magazine
• Events
• Useful links
• Contact
• Login