Aims and Results
Institutions: SWITCH, ETHZ, FHO, UNIBAS, UNIBE, UNIGE, UZH, SUPSI, ZFH
This work package focuses on the delivery of an e-assessment portal providing status quo information and best practice to support the decision process about assessment infrastructure at HEI's.
- Identification of information sources and definition of portal structure
- Implementation of portal (internal test environment http://eassessment-test.eduhub.ch/)
- Structured information (information collection sheet) available via portal (https://www.eduhub.ch/resources/eassessment and related search section http://eassessment.eduhub.ch/)
- test phase & feedback
- portal update cycle description
Institutions: SWITCH, ETHZ, FHO, UNIBAS, UNIBE, UNIGE, SUPSI, ZFH
This work package focuses on local e-Assessment information to be linked into the portal, by localizing, preparing, collecting and structuring institution specific information
(scenarios in use, users, responsibilities, term dependencies, rooms, security, legal issues, technology etc.).
Results available in the search section:
- Detailed and structured local information from each institution on specific products in use and e-Assessment in general, available via portal
- Information of non-partner HEI’s and structure ready and available via portal
Institutions: UNIBE-IML, ETHZ, FHO, UNIBAS, UNIGE, SUPSI(FFHS), SWITCH, ZFH
This work package focuses on creating an e-Assessment tool overview (with focus on technology and tools used or in development at Swiss HEIs), as part of the portal.
Results available in the search section:
- Template for the description of e-assessment tools (use complete collection sheet instaed!)
- Description of e-Assessment infrastructure at the IML, UNIBE (MEASURED, e-OSCE, Web-Pool, AWP, Self-Assessment) and of competence-based assessment at FHO, of tools at UNIBAS (OSCE Manager), at UNIGE (Dokeos, Moodle, Chamillo and Evasys), at SUPSI/FFHS (Moodle and others), at ZFH (Self-Assessment, Aufgabenbörse), of SEB with online exams running in Moodle, ILIAS and SIOUX and in combination with 3rd party applications or Virtual Desktop Infrastructure
- Measures to improve security and monitor the exam clients; Fedora Linux in a specialised system configuration for exams
- Tools in use at non-partner Swiss HEIs
Institutions: SWITCH, ETHZ, UNIBE, UNIGE, SUPSI
This work package focuses on supporting and establishing a formal exchange between projects and institutions. A continuos information flow and dissemination are objectives as well as organizing regular community events.
- Community space for exchange: SIG e-Assessment
- Regular community meetings and special events as workshops (2 regular meetings and 2 workshops in Geneva and Berne during 2013, webinars about e-assessment; eduhub days February 2014; next workshop April 3 2014 in Geneva; regular SIG meetings in 2014)
- Provision of regular news about single project results, e-Assessments performed, project deliverables and additional scenarios.
Institutions: ETHZ, FHO, UNIBAS, UNIBE-IML, UNIBE-ILUB, UNIGE, ZFH
This work package focuses on system tests under novel conditions, with not yet tested scenarios or together with new institutions, in order to enhance stability, security, usability and improve functions in view of further dissemination and centralized services.
- evaluation procedures and checklists for use under novel conditions and scenarios (guidelines for faculty members in E and D)
- guideline for creating a survey on online exams and the use of SEB (in E and D)
- on-site support of new institutions
- test results (with LMS Moodle at UNIGE; with Moodle and students hardware at ZFH; with ILIAS and different scenarios at UNIBE (Manual SEB with ILIAS (D); field tests ILIAS with SEB (E, D), field tests ILIAS (D), requirements for bar examinations (D), Comparison of E-Exams with EvaExam and ILIAS (E, D))
- evaluation procedure and checklists for different exam scenarios (federal, local, in German)
- technical and administrative support documents for deployment of system
- iOS compatibility check (--> therefore canceled*: on-site support for institutions running exams; run and evaluate different exams at new institutions; evaluation reports and specifications of necessary improvements; report about evaluation and test at UNIGE)
- technical and administrative support documents (e-OSCE website, OSCE Eval instruction video (in German)), evaluation procedure and checklists to evaluate usability and feasibility in new exam settings (pre-exam and post-exam questionaire, in German), improvements of administrative client (performed improvements, in German);
on-site support for institutions running exams: support provided at UNIBE, UNIL, Göttingen, federal exams;
run and evaluate different exams at new institutions: evaluations done at UNIBE, UNIBAS and Sydney;
- replace pen & paper processes in OSCE with e-OSCE at UNIBAS: done within one OSCE cycle in 2013, both systems in use;
teach staff and examiners: done, high acceptance;
test of interfaces with OSCE Manager: done, interface stabilized, runs perfectly;
evaluation and feedback reports, evaluation of satisfaction with the system, feedback to eOSCE team (reports UNIBAS, UNIBE, in German)
- Proof of Concept: Windows 8 tablets for online exams with SEB (in German); Canceled* due to lack of suitable tablets: user acceptance evaluation form for SEB used on tablets; on-site support of new institutions; collection and evaluation of results of SEB being used on tablets; evaluation procedures and checklists for SEB used on tablet computers
- Canceled* and postponed to spring 2014 (see MEASURED above) : run exams with MEASURED tablet client; feedback on technical, administrative and operating aspects; on-site support for institutions running exams with tablet client; evaluation and feedback reports of exams
- evaluation procedures and checklists for SEB used on student computers (SEB servercanceled* because of considerable delay of development); user acceptance evaluation form for exams on student hardware; on-site support of new institutions; results of SEB used on student hardware; evaluation and test report of SEB used with students hardware at UNIGE
- Canceled* (see MEASURED above): concept and evaluation procedure of MEASURED use on students hardware; implement pilot system and run test exams; evaluation report
- on-site support of new institutions willing to use SEB in exams; evaluation and test report about SEB used with public computers at UNIGE
- Canceled* (see MEASURED above): concept of MEASURED exams on public computers; run and evaluate test exam on public computers
Specialized assessment forms:
- report on new assessment scenarios for e-OSCE; technological and methodological feasibility report; concept paper about implemention of new scenarios on platform; new partners to implement new assessment scenarios: The possibility of adapting the eOSCE system for evaluations of standardized patients in several locations at several points in time was explored. This process - at the time paper and pencil based - would profit from an electronic rating by reducing administration costs and improving reliability of ratings.
A test implementation is being set-up and a proposal for funding was handed in to the Association of Standardized Patient educators.
Competence-oriented assessment pilot:
- report about implementation and execution of a competence-orientated exam based on WP 7.1; communicate to WP 7.2
*Resources for canceled parts have been reallocated to deliverables in WP 4.2
Institutions: ETHZ, FHO, UNIBAS, UNIBE
20 PM (and reallocated resources from WP 4.1)
This work package focuses on improvements for rollout and enhancements based on feedback (already performed field tests and upcoming tests of WP 4.1).
- implementation of requested features; usability and design improvements in SEB versions for Windows and Mac OS X (stable release candidate version 2.0 RC5 for Windows and a preview version Safe Exam Browser SEB 2.0pre3 for Mac OS X 10.7 and 10.8 have been released. The SEB 2.0 stable releases are expected in Q2/2014);
- improvement specification for ILIAS: ILIAS test improvements, requirements for optimized ILIAS e-exam system (E and D)); concept-papers, improvements/developments in ILIAS questions-forms (feature-wiki); rollout-recommendations: released in the recent ILIAS Version 4.4 (Cloze Questions Improvements; Revision of MC Question type; Multiple Choice K/Kprim) or are implemented in the upcoming ILIAS version 4.5 (improvements; Matching Questions improvements; several test results improvements; release expected end of 2014)
- further development of SEB tablet prototypes (proof of concept Windows 8 tablets); porting of SEB on additional tablet OS (according to user demands and tablet market situation) (behind schedule);
- further development of MEASURED tablet prototype: A first prototype was successfully tested: Screenshot of web application on tablet, usability test report, measured data flow diagram.
Canceled: feasibility report about porting to additional tablet OS (according to user demands and hardware availability)
BYOD and LMS support:
- implementation of requested features, usability and design improvements regarding use of student notebooks in SEB versions for Windows, Mac OS X, in SEB Server and in the LMS software interfaces for exam use; implement support in SEB for additional LMS according to user demands:
Improvements in SEB are released. Delayed due to the fact that SEB 2.0 development is behind schedule. All deliverables directly related to SEB-Server had to be cancelled and are not available)
- further develop administrative interface of e-OSCE system; implement usability and feature requests based on feedback and evaluation results; improve import/export functions for exam data (release notes and e-OSCE website)
- Canceled: improve and expand import/export interfaces of MEASURED; evaluation of needs for interfaces to additional systems
- integration of software module for management of MCQ’s inside OSCE Manager (report about supporting question lifecycle, question creation with different quality controls and creation of an assessment; question process diagram)
As the publication of code could allow potential attackers searching for weaknesses of the software please ask for access to code at firstname.lastname@example.org
Media enriched MCQ:
- report on best media enriched MCQ forms (graphical hotspot etc.) (identification and documentation MCQ formats).
- implementation inside the question management module of OSCE Manager: The new MCQ formats has been released within the last OSCE-Manager package (see above)
- BeAXi software: considerations about MCQ, question management and Single-Sign-On (WP 5.2) have been integrated into the concept of the Basel electronical Assessment extended on iPad software.
Virtual desktop infrastructure:
- general concept for exams with VDI with SEB (including process description for planning and conducting exams); manuals with technical details about how to configure and use VDI with SEB
Institutions: UNIBE-IML, ETHZ, FHO, SWITCH, UZH
This work package focuses on the creation and evaluation of business models and according Service-Level-Agreements (SLAs) to assure administrative, technical, legal and monetary sustainability of e-assessment tools at Swiss HEIs.
Institutions: UNIBAS, UNIBE-IML, FHO
This work package will define form and possibilities for the use of existing single-sign-on infrastructures for different e-Assessment systems.
- Overview about the different access mechanisms in current e-Assessment tools
- Feasibility study of access scenarios trough Shibboleth AAI: Presentation SSO in e-Assessment and easy-to-use concept for secure SSO exams
Goal is implementation a proof of concept, using the core security aspect - Public Cryptography and Digital Signature - to store data securely and verify the authenticity of a sender. As the private signing key is sensitive data it should be accessible after authentication. Therefore the system must be an extension to Shibboleth.
- Feasibility study of online/offline system, which supports asynchronous encryption, certificates and access on PC and tablet platforms: Demo server was implemented.
Beside the requirement analysis and the concept, a demo server and iOS, Web and native local client was implemented to demonstrate the fuctionality (ev. later implementation of a symmetric key functionality because this may lower the complexity of the processes althought it is less secure). Demo server is accessible only with UNIBES email address and is already in use as alpha release of the iOS exam application (code available on request email@example.com).
Institutions: FHO, UNIBE-IML, ETHZ, UNIBAS
This work package focuses on implementing measures for facilitated and sustainable deployment of systems to other institutions-
Distribution: documentation (also translated); user-, technical- and administrative manuals (paper, video, online tutorials);
concept paper for distribution/installation and maintenance (updates) of tools and infrastructure: Concept paper for distribution of e-OSCE;
implementation of multilingual interfaces; distribution channels; administrative and legal documents:e-OSCE Multi-Lingual Interface Support (English, German & French), e-OSCE Support documentation, installation instruction and video available through the e-OSCE Website;
e-OSCE Apps available for free download through the Apple AppStore (OSCE Track; OSCE Eval);
OSCE-Manager Multi-Lingual Interface Support (User Interface and documentation in English available); OSCE-Manager supports internationalization. German and English localizations are available. The OSCE-Manager website makes available support documentation, installation instruction and video tutorials
Implementation: feasibility analysis of porting (MEASURED and) e-OSCE to different platforms (e.g. Android);
improvements for facilitated implementation and distribution of MEASURED and e-OSCE;
on-site support for new institutions willing to use SEB with third party applications / student hardware / tablets (see WP 2.1);
e-Assessment workshopat ETHZ with focus on SEB, third party applications and VDI use:
- 06. June: Consulting ZHAW (C. Schmucki) at ETH Zürich (D. Schneider, Th. Halbherr, Th. Piendl)27. May: Workshop FHNW (Cinzia Garcia) at ETH Zürich (D. Schneider, Th. Halbherr, Th. Piendl)
- 24. May Workshop on “E-Assessment: Using the Safe Exam Browser at University” at the University of Geneva (L. Moccozet)
- 15. May: Workshop University Lucerne (M. Comte) at ETH Zurich (D. Schneider, T. Halbherr, Th. Piendl)
- 7. May: Consulting ZHAW (C. Schmucki)at ETH Zürich (D. Schneider, Th. Halbherr, Th. Piendl)
- 30. April: Consulting ZHAW (C. Schmucki)at ETH Zürich (D. Schneider, Th. Halbherr, Th. Piendl)
- 19. April: International Workshop “Online Assessment using the Safe Exam Browser” at ETH Zürich (D. Schneider, T. Halbherr, D. Bauer, Th. Piendl
- 20. March: Workshop University Bern (Hj. Lauener) and ZHAW (C. Schmucki) at ETH Zürich (D. Schneider, Th. Halbherr, Th. Piendl)
Synergies: concept for common support- and distribution structures of MEASURED and e-OSCE: Synergies between MEASURED and eOSCE were optimized during the project. Design elements for eOSCE were adopted for use in MEASURED (tablet version): Final Report.
Institutions: UNIGE, ETHZ, FHO, UNIBE-IML, SUPSI(FFHS)
This work package focuses on providing an overview of competence-oriented assessments at Swiss HEIs (Bologna conform testing/exam methods and infrastructure).
Results available at https://ple-connect.unige.ch/groups/peercollaborativecompetence-e-assessements/:
- Competence-oriented assessment scenarios: experiences; requirements; scenario documentation:
- Development of the new Swiss licensing examination for human medicine: from concept design to first result. It involves the process from concept design to the first results of the implementation of the new exam. It has been submitted to the "Medical Weekly" (UNIBE-IML).
- Requirement analysis for student hardware (BYOD) scenarios for competence‐oriented assessment at the Hochschule für Technik Rapperswil (HHO-HSR). It summarizes requirements and possible hardware configurations for running BYOD e-assements in the various configurations available at HSR.(FHO)
- Development of a competence oriented assessment scenario for computational methods for quantitative finance with MatLab. The scenario settings is based on Moodle, VDI and SEB (ETHZ)
- Reports including survey, needs, requirements, experiments, tests scenario building, research and experience input:
- Definition and categorization of the most important concepts and approaches of competence-based e-assessment and the reasons for a huger inclusion of the competence orientation. The players and some implementation scenarios are described to conclude on the most important trends. (SUPSI)
- Survey and state of the art of peer and collaboration competence assessment methods and softwares, analysis of existing peer assessments scenarios, recommendation for developing peer and collaborative e-assessments, implementation and integration of peer e-assessments modules in a social learning/PLE platform, description of peer e-assessment based scenarios at the University of Geneva. Some parts of the report document have been made available on myple.ch in a dedicated group. The workgroup assessment scenario has been published in a paper presented at Interactive Collaborative Learning 2013. (UNIGE)
- Peer-assessment (typology, survey-technologies, case-studies) (UNIGE)
- Group-work assessment
- VDI scenario requirement document: Description of a general concept for exams to assess competences in the use of specific IT tools (such as programming environments, CAD, simulation, or statistics software) using Virtual Desktop Infrastructure (VDI) and Safe exam Browser (SEB). It includes process descriptions for planning and conducting such exams. The document includes checklists and technical details on how to configure and use VDI with SEB.
nstitutions: ETHZ, FHO, UNIBE, SUPSI(FFHS), UNIGE, UNIBAS, UZH, ZFH, SWITCH
This work package focuses on preparation of competence-oriented assessments with VDI, consolidating results and information on competence-oriented exams already performed and providing recommendations on further steps to take and perspectives for further development in the e-assessment field at Swiss HEIs.
- Preparation: technical, didactical and planning documentation (see WP 7.1) (VDI documentation)
- Experiences: guidelines on competence-oriented assessment with 3rd party applications; corresponding workshop(s): Presentation and workshop at Swiss MoodleMoot 2013: “The ideal Assessment Tool: Valid and Secure E-Assessments with Moodle and Safe Exam Browser (SAMoo)” and “Online-Exams: Challenges in Transitioning from Paper to Online (SAMoo)” ; presentation/poster at delfi 2013; Kompetenzorientiertes Prüfen mit virtueller Desktop-Infrastruktur und Safe Exam Browser; presentation at educause 2013
- e-Assessment Perspectives: Report with summary and future prospects of e-assessment scenarios and solutions at Swiss HEIs: individual reports of UNIGE, ETHZ, UNIBAS, UNIBE (E and D), FFHS, UZH, ZHAW)
WP - Work package
PM - Person month