Usability research for interpreter-centred technology

Front matter

Usability research for interpreter-centred technology

The case study of SmarTerp

Francesca Maria Frittella

Contents

Acknowledgements 1 Introduction 2 Usability engineering

2.1 The concept of usability da 23a EHE 2.2 Usability engineering . .. ct axe Ee. re BAe 2.4 Usability testing . . .. oy keh eee wale a els os

3.4 CAT tool research 2 2» Gono alo a ent 3.4.2 ‘Evaluation research... . es arena

Part II: Satisfaction by Pragmatic Usability Criteria

Do you agree with the following statements? - from 1 (strongly disagree) to 7 (strongly agree)

- The CAI tool was easy to use (perceived ease of use)

- The CAI tool helped me improve the accuracy of my delivery (perceived effectiveness)

- No training is required to use the CAI tool effectively (perceived ease of learning)

- The input provided by the CAI was timely (perceived timeliness)

- I felt that I could rely on the CAI tools support (perceived dependability)

Part Ill: Likelihood to Use the Tool

How likely are you to use ASR-integrated CAI tools as a support during simultaneous interpreting in the near future? - from 1 (very unlikely) to 7 (very likely)

6.4.4 Semi-structured interview protocol

In the SmarTerp usability test, a semi-structured interview complemented the analysis of users’ perceptions through the questionnaire. I decided to include an interview for three main reasons. The first was to expand the analysis of users’ needs and requirements and update the CAI tool features based on such deeper understanding. The second was to shed light on the reasons for students’ perception of the tool’s usability expressed through the questionnaire. The third was to integrate participants’ self-reported data into my interpretation of critical incidents and identification of usage problems, hence adding strength to the analysis. While in usability tests this is often accomplished by asking users to “think aloud” while performing test tasks, this is obviously not possible in the context of a simultaneous interpreting task.

I conducted the semi-structured interviews based on the protocol below. Interview questions in parts I and II of the protocol were aimed at shedding light on the reasons for participants’ responses to the post-test questionnaire. Part III was aimed at gleaning insight into participants’ perception of the tool’s support in the rendition of individual problem triggers. Part IV asked participants to express their judgment on the functions and design features of the tool. It is important to note that although, as discussed earlier, users’ opinions alone cannot be used to reliably derive design principles, within a mixed-method study, users’ recommendations may be compared with researchers’ observations to see whether they converge or not. Part V explored participants’ perception of the tool in a real context of use. During the interview, I adapted the questions (by reformulating them, asking for specifications, asking to report a critical incident, making open-ended statements that invited participants to fill the gap, etc.) to probe into issues of interest without influencing participants’ responses.

Part I: General Satisfaction

Your self-reported satisfaction with the tool was ... [user’s evaluation in the post-task questionnaire] Could you please explain this choice?

In the enrolment questionnaire, your self-reported likelihood of using a CAI tool of this kind was ... In the post-test questionnaire, it was ... Why is that?

Part II: Satisfaction by Pragmatic Usability Criterion

Based on the enrolment questionnaire, your perceived ease of use / effectiveness / ease of learning / timeliness / dependability is (...). Could you motivate your choice? Can you recall an example?

Part III: Satisfaction by Problem Trigger Class

Based on the enrolment questionnaire, your self-reported difficulty with acronyms / named entities / numerals / specialised terms is (...). How do you normally deal with this problem trigger? Do you believe that the tool helped you in the test? How? Can you recall an example?

Part IV: Design-related Recommendations

What did you like the most about the tool and its interface? Why? What did you like the least about the tool and its interface? Why? Part V: Perception of the Tool in Context

In your view, how could a CAI tool of this kind make a difference to you as an interpreter?

How does the tool differ from a human boothmate?

Which one is more reliable?

If you were to choose whether to use it in an online or on-site assignment or both, which option would you choose?

- If on-site or both: would you still use the help of a human boothmate? What would his/her role be in that constellation?

6.5 Participants

6.5.1 Recruitment

The participants in the study were recruited through an open call for participants made circulate through the research team’s professional network and SmarTerp’s communication channels. Prospective participants who expressed their interest in participating in the study received an informed consent to sign. The document included key information about the study objective and scope, the deadlines and the data collection and treatment procedures. They were also asked to fill out a digital enrolment questionnaire. The questionnaire was aimed to profile and select participants based on our selection criteria: (a) 30-50 years of age, (b) more than 10 years of professional experience as an English-Italian conference interpreter in the simultaneous mode; (c) no less than 10 workdays as an English-Italian simultaneous interpreter each year, (d) no identifiable connection with the topic of the source speech (to create equal conditions for all participants). The first ten prospective participants who responded to our call and were found to fulfil our selection criteria were notified of their inclusion and provided access to the online training module.

Part I: Participant’s Personal Information What are your name and surname?

What is your chosen pseudonym? (You can choose any name different from your given name as the pseudonym which will identify your data to protect your privacy. If you do not choose any pseudonym, the research team will assign one to you.)

What is your country/ region?

What's your age group? Options: below 30 - excluded; 30-40; 40-50; above 50 - excluded.

Part II: Qualifications and Professional Activity 6.5 Participants

Do you hold a Master’s degree in Conference Interpreting or equivalent academic qualification? Options: yes / no.

Are you a member of a professional association? If yes, which one(s)? How many years of professional experience as an English-Italian conference interpreter in the simultaneous mode do you have? Options: less than 10 - excluded; 10-19; 20-30; above 30.

On average, how many days do you normally work as an English-Italian simultaneous interpreter each year? Options: less than 10 — excluded; between 10 and 20; between 20 and 30; above 30.

Do you have any fields of specialisation? If yes, which one(s)? What is the country/region of your main clients? If Africa - excluded.

Part III: Current Use of Technology

How many days have you worked in the remote simultaneous interpreting mode over the past 12 months? Options: less than 10 — excluded; between 10 and 20; more than 20.

In how many of your last three on-site simultaneous interpreting assignments did you bring a laptop with you in the booth and use it to browse your glossary or search for unknown terms / information while you were interpreting? Options: none; 1 assignment out of three; 2 assignments out of three; 3 assignments out of three.

Have you ever used a computer-assisted interpreting (CAI) tool? If yes, which one and for what purpose?

Part IV: Perception of Problem Triggers

Do you find it difficult to interpret the following items? Please, rate their difficulty on a scale from 1 (very easy) to 7 (very difficult): acronyms, named entities, numbers, specialised terms.