Daciana Iliescu and Evor Hines, School of Engineering, University of Warwick
The use of EVS (electronic voting systems) in teaching and learning has been extensively reported in the literature. These systems are based on the use of individual ‘handsets’ with which students can provide individual responses to questionnaires in an interactive manner. A variety of hardware technologies could be used for the handsets, such as networked computers, custom-built wired or wireless handsets, as in PRS (personal response systems), and more recently, PDAs and mobile phones. Draper maintains a web page of technologies and supporting resources (Draper 2005).
From a pedagogical perspective, the use of such technologies addresses several issues, such as the lack of interaction between lecturer and students in a lecture setting, the lack of students’ participation, especially in large classes, the delay between assessment delivery and feedback etc. However, their benefits, as Draper & Brown stated, “vary over a considerable range, depending not only on the subject matter being learned but still more also on the pedagogical aims and consequent methods of the lecturer” and that “pedagogy had to be put first, technology second”. (Draper & Brown 2004). . Reported test cases have shown that using an EVS produces produces “large and statistically significant improvements in standardized test results” (Crouch & Mazur 2001), “a small upward shift in exam scores but with a pronounced effect in raising the ‘tail’ of the class” (Boyle & Nicol 2003) and that the learners and teachers “almost always saw it as providing a net benefit to them” (Draper & Brown 2004).
WES and its technology - the SMS server
The system presented here is a short message text (SMS) based student feedback system called WES (Warwick Engineering SMS project) and, at present, is being developed and hosted in the School of Engineering. The access to the server is through a web interface. The present website for the server is http://sms1.eng.warwick.ac.uk and the current number is (+44) (0)7906 390 850.
The server is capable of gathering, storing and processing, as well as sending, SMS text from/to any mobile phone user on any GSM network. The handsets currently used are the students’ own mobile phones. From our campus survey we found that 98% of the student population own a mobile phone and 95% carry it permanently, figures supported by wider literature. In terms of signal strength and network coverage, we found that 90% of the teaching rooms in Engineering had a signal from the major carriers (Vodaphone, O2, Virgin, Orange and 3).
This technological solution to the implementation of an EVS through an SMS based server has several advantages:
- it is based on a popular technology, according to GSM WORLD, more than one billion users worldwide are capable to send/receive texts (S. Buckingham 2000)
- it can be used on an international scale
- there is no theoretical limit on the number of users that can access the sever simultaneously
- it can be used in any location covered by the GSM network
- there is no theoretical limit on the number of messages stored on the system
- it is not limited by third party technology since the hardware and software is locally developed and managed
- it is highly extensible in terms of the number and type of services that can be managed from the server
- it can be integrated with third party services
- if the need arises, it can be replicated easily
- it can be developed further to support new G3 mobile phone technologies (voice, data, video).
In an academic setting, currently the WES can be used to deliver the following services. The first two services are functional and available through the server web site, whilst the others are developed, but not linked in yet.
1. Student online testing
There are three formats of questions supported by the system: multiple choice questions, where the format of the answer is pre-set to a letter, free text and a combination of the two, where the answer consists of the question number followed by the free text. These can be used to run quizzes, feedback forms and interactive questions/answers sessions. There is no theoretical limit on the number of questions however, the length of the answer is currently limited to 130 characters (although it can be extended to support collated messages). The questions can be read out or displayed on a screen in class. At present, although feasible, there is no implemented capability for the questions to be sent directly to students’ phones.
2. Reply and feedback for tests
The answers are analysed and displayed on the web page of the test, with the added possibility of the results and statistics being sent to the lecturer and/or students’ phones. At present there is no acknowledgement to the students of their answers being received.
With this service staff can compose a short-notice announcement or a warning message on a web form and send it as an SMS to registered recipients. Our survey showed that students would be willing to register to receive such messages from the department as an alternative to the current email notification system.
4. Formal and informal voting
The system is linked in with a user registration system, so that a valid password can be checked when the vote is registered. There are, however, issues regarding the security and anonymity of the vote, which could be solved in the future by message encryption. On the server side, the encryption system is supported, however, only top of the range phones can run additional encryption software.
5. Automated radio station
This service was developed to allow users to send in their music preferences, which would be used to search a database of songs and automatically play/broadcast their choice. In the future the idea of this system could also be used to create an SMS interface to an information retrieval system.
From the range of services described above it can be seen that WES is much more than a personal voting or response system. It is, however, still in a developing phase and extensive testing and evaluation has yet to be done. However, we will attempt a comparison between the student testing service and other types of EVCs, based on our evaluation and existing literature (Jones and Marsden 2004).
- anonymity of response, shared with all types of personal systems (considered sometimes an advantage over voting methods such as show of hands)
- automatic processing of answers and immediate results in an electronic form
- compared to wired systems and similar to wireless EVCs, WES has the advantage of mobility and portability, however, it has the added advantage of usage in an outdoor environment with a larger coverage than any of the wireless systems
- WES does not require any prior hardware setup, nor prior knowledge of the function of handsets, as in the case of purpose-built systems
- the keyboard of the handsets allows more flexible testing scenarios than multiple choice questions
- there is no upfront cost associated with handsets or software licenses, nor issues relating to the ownership of handsets
- the SMS technology (like email) does not guarantee set transmission times or guaranteed delivery of the message, therefore some messages may be delayed, blocked or lost
- the cost of the message is transferred to the sender, is based on the number of messages being sent and is network dependent, although possibilities exist in reimbursing such cost and operating volume based tariffs
- not all networks have full coverage and some locations may be protected for electromagnetic radiation, therefore not all users may be capable of getting a signal
- not all users may have a mobile phone, therefore alternative back-up needs to be sought
- nominal voting or answers require prior registration with the system
- the handsets can communicate among themselves prior to any use in an assessment and the source of the answer cannot be confirmed
- the server can be loaded with spam and will require regular clean-up
- there may be issues (unknown at present) with incorrect message parsing and proprietary formats among networks
The article presented the SMS based system developed bin the School of Engineering. It was shown that the system has more uses than just an EVS or feedback system for students.
However, when used in the latter manner to gather immediate feedback, to introduce interactivity and for formative assessment in large classes, WES compares favourably with equivalent systems. The main advantage consists in its flexibility, whilst the major drawbacks are currently related to the existing SMS technology and, more importantly, the cost per message for the user.
The work reported here is partly funded by the University of Warwick TDF and the School of Engineering. The current implementation of the server system and its documentation has been done by Jarl Cornell (Engineering) and Olof Sisak (Mathematics) and the implementation of some of the services by Philip Wright (Computer Systems Engineering), Andrew Taylor (Computer Science) and Nian-ou Pan (Engineering).
J.Boyle and D.Nicol (2003) “Using classroom communication systems to support interaction and discussion in large class settings” Association of Learning Technology Journal (ALT-J) 11, pp43–57.
C.Crouch and E. Mazur (2001) “Peer instruction: ten years of experience and results” American Journal of Physics 69, pp970–977.
S.Draper and M.Brown (2004), “Increasing interactivity in lectures using an electronic voting system”, Journal of Computer Assisted Learning 20, pp81–94.
S.Draper (2005), “EVS technologies, alternatives, vendors”, Interactive Lectures Website http://www.psy.gla.ac.uk/~steve/ilig/tech.html
M.Jones and G.Marsden (2004) “Please turn on your mobile phones - First Impressions of Text-messaging in Lectures”, Computer Science Department Tech Report (07/2004). Waikato University.
E.Wit (2003) “Who wants to be… The use of a Personal Response System in statistics teaching” MSOR Connections 3, pp5–11.