The Evaluation of E-government Web Portal in Turkey through Eye-tacking Method

Author(s):

Ömer Demir(presenting / submitting)Arif Tuluk Kerem Ay Fatma Özçelik Hakan Tüzün

Conference:

ECER 2017

Network:

Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)

Format:

Paper

Session Information

ERG SES D 05, Professionalism and Education

Paper Session

Time:

2017-08-21

13:30-15:00

Room:

W2.10

Chair:

Jonas Almqvist

Contribution

Advances in ICTs have rendered tremendous changes in the understanding of public administration mandatory. To keep pace with these continuous changes, governments have been trying to offer some of their services online over the last two decades. Governments all over Europe and world design and create comprehensive web sites to fulfill these purposes. E-government web portal in Turkey can be given as a representative example of one of these web sites.

E-Government web portal in Turkey:

E-government web portal is a comprehensive web site designed by paying attention to user needs with the aim of offering government services to the citizens. It aims to reduce institutions’ and clerks’ work load, make government services more easily available, and provide quality, fast, non-stop and secure services for all citizens. E-government web portal in Turkey came into use on 18th of January 2008. It became available on mobile platforms on 7th of October 2009. A total of 1739 services are provided with the collaboration of 289 government institutions (E-Government web portal, 2016). The number of users enrolled in the web site is over 31 million now. As for another statistics, 53.2% of the Internet users in Turkey benefit from E-government services (TUIK, 2015). The number of services and users continuously increase. With the help of these improvements, Turkey has been ranked as the 8th amongst 33 countries in terms of being user focused according to e-government measurement study (European Union, 2015, as cited in Demirel, 2016).

Usability:

With the help of proliferation of the tools that can be used, the concept of usability gained traction. ISO (International Organization for Standardization) defined usability as the extent to which a product ensures effectiveness, efficiency, and satisfaction in a specified context of use for achieving specified goals by specified users (ISO 9241-11, 1998). Likewise, Shneiderman and Plaisant (2004) focused more on efficiency and satisfaction components of usability. They explained the concept of usability with the time required for users to learn system to successfully carry out a specified task, retention of the information learnt during the interaction with the system, performance of the system, mistakes made by users, and users’ subjective satisfaction level. In usability studies, many different methods such as participatory-based design, expert evaluation, think aloud procedure, scales, and eye-tracking are used (Yeniad, Mazman, Tüzün, & Akbal, 2011).

Significance:

Determining usability of a web site is of great significance since users’ perceived ease of use and satisfaction indicates the success of design of web sites (Insfran & Fernandez, 2008). However, little is known if citizens from various background in Turkey can effectively and efficiently make use of e-government web portal. If an average citizen cannot utilize this web site, this endeavor is doomed to yield dismal results. All in all, it is lame to try to be e-government without making its citizens e-citizens first. Therefore, if they are not found to be so, relevant precautions such as arranging information literacy courses to address this issue might have to be taken. On the other hand, European e-Government (2016) initiatives aim to ensure digital transformation of member governments by means of modernizing public administration in the upcoming 5 years. In this context, it is very significant to determine if citizens can use this web site. Hence, the purpose of this study is to help authorities improve e-government web portal in Turkey by eliciting usability problems of it. To reach this goal, three research questions were determined.

1) How effectively do authentic participants use E-government web portal?

2) How efficiently do authentic participants use E-government web portal?

3) How satisfied are authentic participants with using E-government web portal?

Method

The method of this study is usability testing. The prime purpose of usability tests is to enhance the usability of the product being tested (Dumas & Redish, 1999). Besides, eye-tracking method was employed amongst usability testing methods. Study Group: In usability studies, it is indicated that 5 authentic users are sufficient to identify 85% of all usability problems (Nielsen, 1993, 2000). Therefore, 7 volunteering authentic users participated in this study. Participants’ ages range between 18 and 49, 5 of whom were male. Data Collection Tools: Five data collection tools were utilized in this study, the first of which was participant information form. It includes personal data of users such as occupation, age, and gender. Second one was the satisfaction form containing 3 open-ended questions. Third one was authentic tasks form which incorporated 5 authentic tasks. These three forms were checked by all researchers and a field expert. The fourth one was the eye-tracking device. Using this device, eye-movements of the participants were recorded. To do that, Tobii T120 desktop eye-tracking device and Tobii Studio 3.3.0 software were used. As for the fifth data collection tool, an observation form was used. That is to say, researchers observed authentic participants trying to complete authentic tasks and took notes. Data Collection Process: First, participants were informed about the aim of the study. After that participants filled in the participant information form. Then participants read the introduction and eye-tracking device was calibrated. After that, authentic tasks were given to the participants. After the study, participants were asked to fill in the satisfaction form. Data Analysis: Completed tasks and tasks’ completion duration were determined. Frequencies, percentage, minimum, maximum, mean, and total durations were reported. Analyzing eye-tracking data, metrics of fixation count, fixation duration, heap map, gaze plot and area of interests were used. Besides, after reaching a consensus with other researchers 5 area of interests were created. These are (1) main page, (2) search and personal, (3) menus, (4) current information, (5) announcements and mostly used. To analyze the qualitative data of the study, first, all the answers given to the questions in satisfaction form were transcribed. In this way, a 395-word document was obtained. Afterwards, every researcher separately analyzed the documents. To keep the data anonymous, names of the participants were coded.

Expected Outcomes

P1 and P4 fixated on the profile area (which is in search and personal area, and necessary to fixate on to be able to complete T1 many times; however, they were not able to complete the T1. This is because they were not able to detect this area was clickable. Therefore, it can be suggested that this area should be larger and has some indications showing that it is clickable. Current information area covers much more area than the menu, and search and personal ones do. What is worse, current information area is located at the very center of the page. Considering the aim of the current information area is to inform people about the newest added services on E-government web portal and no one reported they sign in E-government web portal for that purpose, it can be recommended that current information area should be made smaller and placed anywhere away from the center of the screen. Participants using search bar conducted faster searches. For example, after P6 detected search bar, its search performance soared. Similarly, P2 could not detect the search bar, but knowing its affordances it invented its own method. It used Windows shortcut CTRL+F to carry out search, which is not feasible because only the open page can be searched within this method. In short, menu and search bar areas can be expanded and current information area can be shrinked. Besides, search bar can be made more visible by moving it to the center of the screen. Actually this modification to the E-government web portal uttered by P6: “Tasks are easily get done through search bar. But search bar should be more in sight.” In conclusion, E-government web portal can be designed like Google search engine in order to maximize the visibility and search efficiency.

References

Demirel, F. (2016). Bugün açıklanan ulusal E-devlet ve siber güvenlik strateji ve eylem planı’nda hangi hedefler var? Retrieved from http://webrazzi.com/2016/09/09/bugun-aciklanan-ulusal-e-devlet-ve-siber-guvenlik-strateji-ve-eylem-planlarinda-hangi-hedefler-var/ in 04.01.2017. Dumas, J. S., & Loring, B. A. (2008). Moderating usability tests: Principles and practices for interacting. San Francisco, CA: Morgan Kaufmann. European eGovernment. (2016). European eGovernment Action Plan 2016-2020. Retrieved from https://ec.europa.eu/digital-single-market/en/european-egovernment-action-plan-2016-2020 in 06.01.2017. E-Government web portal. (2016). Devletin kısa yolu. Retrieved from https://www.turkiye.gov.tr/ in 06.01.2017. Insfran, E., & Fernandez, A. (2008). A systematic review of usability evaluation in web development. Web Information Systems Engineering–WISE 2008 Workshops, 81–91. International Organization for Standardization (ISO) 9241-11. (1998). Ergonomics requirements for office with visual display terminals (VDTs)-Part 11: Guidance on usability. Geneva Nielsen, J. (1993). Usability engineering. Boston: AP Professional. Nielsen, J. (2000). Why you only need to test with 5 users. Retrieved from Jakob Nielsen’s Alertbox web site https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ in 31.12.2016. Shneiderman, B., & Plaisant, C. (2004). Designing the user interface: Strategies for effective human–computer interaction (4th ed.). Reading, MA: Addison-Wesley Publ. Co. TUIK-Türkiye İstatistik Kurumu. (2015). Hanehalkı bilişim teknolojileri kullanım araştırması (Sayı: 18660). Retrieved from http://www.tuik.gov.tr/PreHaberBultenleri.do?id=18660 in 06.01.2017.

Author Information

Ömer Demir (presenting / submitting)

Hacettepe University

Computer Education and Instructional Technologies

Ankara

Arif Tuluk

Hacettepe University, Faculty of Education, The Department of Computer Education and Instructional Technology, Turkey

Kerem Ay

Hacettepe University, Faculty of Education, The Department of Computer Education and Instructional Technology, Turkey

Fatma Özçelik

Hacettepe University, Faculty of Education, The Department of Computer Education and Instructional Technology, Turkey

Hakan Tüzün

Hacettepe University, Faculty of Education, The Department of Computer Education and Instructional Technology, Turkey

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.

Session Information

Contribution

Method

Expected Outcomes

References

Author Information

Search the ECER Programme

Navigation

Info for