Using Eye-Tracking to Create Impactful Interpretation Signage for Botanic Gardens and Other Visitor Attractions

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript explores an important topic: utilizing eye-tracking devices to analyze preferences for signs in botanical gardens and zoos. This introduces a novel application for these devices. The subject caught my attention immediately upon reading the title. While the manuscript is engaging and holds potential significance for its field, my review identified weaknesses that need to be addressed before I can recommend it for publication. Most of the issues are found in the methodology, which is crucial for the study's comprehension and future replicability. See below these points:

- [Line 143] The experiment was conducted in some locations, including three campuses of Nottingham Trent University. Please be more specific (for example, art campus?). Involving university students (I suppose they were students) in this study might cause problems. For example, marketing students may look at signs differently than the average person. And since the sample size is relatively small this might bias the results. Thus, it is important that the demographic information also covers this aspect.

- [Line 155] What is a “large screen”? Please provide more information (size, model, brand). Was the same screen used across the experiments and locations? What was the resolution of the device (or devices)? Were the places adequately lit (for example, the screen was protected against flares)? Also, were hue, saturation, brightness, or any other setting affecting image characteristics adjusted and controlled? I am aware that the experiment was conducted outside a lab but still, some variables should be controlled.

- [Line 156] The reference sends the reader to a site where different products are presented. I presume hardware was used besides the software, but this is unclear. Did the participants have their eyes tracked by a webcam? Please provide this information and possible advantages and limitations of the hardware used. 

- [Lines 162-164] Again, it is important to know more about the participants from the campuses. If some were from biology, they presumably would be familiar with the IUCN sign.

- [Page 4, Table 1] Provide information about font size used in all signs. In the control variant, it is stated that the top title is centered. However, the illustration shows it positioned at left.

- [Data analysis section] Add the companies that created and/or own the trademarks of Excel (please capitalize the “e” in the manuscript) and R. Include the name of the R package used in the study. Inform what is considered significant. Also, were subjects isolated from any possible companion while participating in the experiment? This information is important because companions might influence the answer given by the subject.  

- [Line 356] The text is confusing: if the significance level was not attained, how come there was a general preference? Consider rephrasing the sentence. 

- [Lines 377 and 379] This first paragraph can be removed from the manuscript without loss, given that the same information is at the end of the introduction.

- [Lines 380-383] Citations are needed for both initial statements.

- [Line 385] Provide examples of cultures that read from right to left (include citations). 

- [Lines 389-391] It is informed that people tend to see text before pictures. Please discuss if this finding has any implications when creating a sign.

Author Response

The manuscript explores an important topic: utilizing eye-tracking devices to analyze preferences for signs in botanical gardens and zoos. This introduces a novel application for these devices. The subject caught my attention immediately upon reading the title. While the manuscript is engaging and holds potential significance for its field, my review identified weaknesses that need to be addressed before I can recommend it for publication. Most of the issues are found in the methodology, which is crucial for the study's comprehension and future replicability. See below these points:

We thank the reviewer for their comprehensive review and very helpful comments in how to improve the manuscript and replicability

- [Line 143] The experiment was conducted in some locations, including three campuses of Nottingham Trent University. Please be more specific (for example, art campus?). Involving university students (I suppose they were students) in this study might cause problems. For example, marketing students may look at signs differently than the average person. And since the sample size is relatively small this might bias the results. Thus, it is important that the demographic information also covers this aspect.

We have now added a table in Appendix B detailing the split of demographics across the sites. We have also added the following paragraph:

“Data were collected between 19^th and 23^rd July 2021 at several cultural venues in the city of Nottingham, U.K. (Nottingham Contemporary Art Gallery, Wollaton Hall Stately Home, Southwell Minster) and three campuses (City, Clifton and Brackenhurst) of Nottingham Trent University (NTU). As the experiment was conducted within the university summer holidays but before the start of school holidays it was mainly day visitors and university staff (including non-academic staff) who participated. As we did not collect data on respondent’s profession or role this information could not be included as a factor in our analysis. The age profile across the sites was similar with overlapping means (Appendix A). Modal number of visits to cultural venues were also similar across five of the six sites (2-5 visits per year) but with participants from Clifton campus having a higher number (10 or more visits). Participants at the Brackenhurst campus had a higher modal education (doctorate level) and may have had more prior awareness about species information signs and information boards given that the campus specialises in animal and rural sciences and shares a name with the Brackenhurst Botanic Garden. As the city campus is in the vicinity of the design school it is also possible that participants at this location had some background in sign design. To combat impacts of prior knowledge we tested signs that had never been displayed at any of these locations.”  

- [Line 155] What is a “large screen”? Please provide more information (size, model, brand). Was the same screen used across the experiments and locations? What was the resolution of the device (or devices)? Were the places adequately lit (for example, the screen was protected against flares)? Also, were hue, saturation, brightness, or any other setting affecting image characteristics adjusted and controlled? I am aware that the experiment was conducted outside a lab but still, some variables should be controlled.

We have added the following information:

“Signs and questions were presented on a 22” monitor with resolution 1920 x 1080, the same screen was used at all test locations. Although light levels varied at each location, we kept the display settings the same and instead positioned the screen and viewer in a location where the screen was clearly visible and where there was no glare. Whilst visitors viewed the information their eye movements were recorded by a remote tracking bar (SMI RED Remote Eye-tracking Device^30,31) attached to the screen. This was calibrated for each individual participant to ensure that their eye movements were tracked effectively. Calibration allowed for differences in individual’s position relative to the screen including accounting for height differences or being closer or further away. We note some limitations in that if a participant moved substantially away from their initial (calibration) position their gaze pattern could be lost, and individuals with glasses were harder for the SMI RED to follow. This meant that respondents were required to sit relatively still during the experiment which may not reflect natural sign viewing behaviours. We note that using eye track glasses (as we recommend for future experiments) should negate the issue of moving out of the viewing zone and allow for a more natural viewing experience. The SMI RED bar in combination with SMI BeGaze 3.6 software³⁰ provided information about which areas of each sign participants viewed and the duration of viewing.”

- [Line 156] The reference sends the reader to a site where different products are presented. I presume hardware was used besides the software, but this is unclear. Did the participants have their eyes tracked by a webcam? Please provide this information and possible advantages and limitations of the hardware used. 

We have updated the reference to include the specific SMI RED device. And have explained the device in more detail – see above

- [Lines 162-164] Again, it is important to know more about the participants from the campuses. If some were from biology, they presumably would be familiar with the IUCN sign.

We have now included more information about the types of individuals in the difference campuses (see above) unfortunately no information was gathered as to the respondents profession or subject specialism so we are unable to test whether this had an impact.

“We acknowledge that some individuals may have had more awareness of sign design or content than others depending on their prior experiences. This was an unavoidable limitation of our study design and we have tried to negate this as much as possible through varying the order that sign designs are presented.”

 

- [Page 4, Table 1] Provide information about font size used in all signs. In the control variant, it is stated that the top title is centred. However, the illustration shows it positioned at left.

This has been added and corrected

- [Data analysis section] Add the companies that created and/or own the trademarks of Excel (please capitalize the “e” in the manuscript) and R.

done

 Include the name of the R package used in the study.

“We conducted basic analysis using Excel (Microsoft 365), R (CRAN-r version 4.3.3)) and BeGaze ³⁰ (SMI) software. R was used to conduct bootstrapping of averages to determine 95% confidence intervals and the package mgcv (10.32614/CRAN.package.mgcv) was used to conduct GLMs. Statistical significance was determined using alpha False Discovery Rate at a threshold of between 0.05 and 0.005 (see Appendix C1).”

 Inform what is considered significant.

We have now added this information in the main text (see above) and included the outputs from GLMs as Appendix C.

Also, were subjects isolated from any possible companion while participating in the experiment? This information is important because companions might influence the answer given by the subject.  

We have added the following information:

‘If the individual was part of a group they participated alone with no assistance from their companions.’

- [Line 356] The text is confusing: if the significance level was not attained, how come there was a general preference? Consider rephrasing the sentence. 

This has been changed to: “Sign design D1: large image/block text was preferred by 43.14% of respondents”

- [Lines 377 and 379] This first paragraph can be removed from the manuscript without loss, given that the same information is at the end of the introduction.

This paragraph has been removed as suggested

- [Lines 380-383] Citations are needed for both initial statements.

- [Line 385] Provide examples of cultures that read from right to left (include citations). 

This paragraph has been reworded (including adding references) as follows:

The finding that visitors tend to read signs from left to right, top to bottom (regardless of cultural venue, age, gender, education level or frequency of cultural venue visits) suggests that sign reading may follow cultural reading patterns. Our study was conducted in the UK with English speakers. English is read left-right, top-bottom, mirroring the same viewing pattern as was seen in our sign reading. However, this reading pattern is not universal with other languages for example Arabic and Hebrew are read right to left and some Asian dialects are read vertically in columns³⁴. This concept of reading direction was investigated by Spalek and Hammad (2005) who used eye-tracking to compare English (left-right) and Arabic (right-left) speaking individuals and examined their response to viewing an image. The English speakers were shown to have a bias towards viewing the left side of the image first and the Arabic speakers favoured the right first ³⁵. This has important implications for sign design as, if we design signage based on left-right reading patterns, they may not be read in the same way by cultures where text flows right to left (or in another direction). Given that Arabic is the fourth most spoken language in the world (362 million native speakers) ³⁶ and is one of several right-left languages, this is a significant consideration. Therefore, when translating for different audiences, the whole layout of the sign should be considered and not just what text is written.

- [Lines 389-391] It is informed that people tend to see text before pictures. Please discuss if this finding has any implications when creating a sign.

“Our study also found text tends to be observed before pictures, regardless of layout. Eye-tracking studies from advertising campaigns in China found similar results with respondents viewing text before an image no matter the position or orientation ³⁷. Given that China has a different cultural reading pattern to the UK, this finding is interesting as it suggests the text-first observation may be culturally universal. As such, consideration should be given as to what textual information is to be conveyed and what is its purpose or objective. As sign viewing times are short (a few seconds)²⁷ , text needs to be to-the-point to convey a specific message quickly. “

 

We believe that we have addressed all of reviewer 1's comments and thank them for assistance in improving the quality of the article

Reviewer 2 Report

Comments and Suggestions for Authors

This paper has several strengths and weaknesses:

Strengths:

1. It provides empirical data on visitor viewing patterns and information recall from different sign designs through eye-tracking and survey methods.

2. The study examines important factors like text length, layout, use of images, and positioning that can impact sign effectiveness.

3. The methodology involving eye-tracking, surveys, and different sign variations allows for robust analysis of visitor behavior and preferences.

Weaknesses:

1. The study was conducted in a controlled setting with signs displayed on a screen rather than in a real botanical garden/zoo environment, which may not fully reflect natural visitor behavior.

2. The sample size of 51 participants is relatively small, potentially limiting the generalizability of the findings.

Figure 3 the Y axis label should be uppercase to be match with X axis label 

Line 436: preferable to apply a traffic light approach (green being Least Concern and red Critically"

It should be "preferable to apply a traffic light approach (green for Least Concern and red for Critically".

Overall, the grammar is quite good, with only a few minor issues that can be easily corrected.

Also please follow the journal requirements for the references preparation. 

 

 

 

Comments on the Quality of English Language

Overall, the grammar is quite good, with only a few minor issues that can be easily corrected.

Figure 3 the Y axis label should be uppercase to be match with X axis label 

Line 436: preferable to apply a traffic light approach (green being Least Concern and red Critically"

It should be "preferable to apply a traffic light approach (green for Least Concern and red for Critically".

Author Response

This paper has several strengths and weaknesses:

Strengths:

1. It provides empirical data on visitor viewing patterns and information recall from different sign designs through eye-tracking and survey methods.
2. The study examines important factors like text length, layout, use of images, and positioning that can impact sign effectiveness.
3. The methodology involving eye-tracking, surveys, and different sign variations allows for robust analysis of visitor behavior and preferences.

We thank the reviewers for their positive comments relating to the article

Weaknesses:

1. The study was conducted in a controlled setting with signs displayed on a screen rather than in a real botanical garden/zoo environment, which may not fully reflect natural visitor behavior.

We believe we already highlight these limitations in the following paragraph:

“We acknowledge that this study has limitations. Asking a visitor to view a sign, directly in front of them, for 30 seconds is not the normal way of viewing interpretation boards. We acknowledge that visitors are more likely to view signs for a much shorter time ^6,27. Additionally, due to covid restrictions we needed to present signs on a digital screen using Remote Eye tracking (RED) and presented information about a different cultural venue to the one the respondents were visiting. This may have resulted in a different interaction than would have occurred with an in-context information board.”

2. The sample size of 51 participants is relatively small, potentially limiting the generalizability of the findings.

We have now added the following to the start of the above paragraph

“We acknowledge that this study has limitations. The study considers the viewing behaviours of 51 individuals, whilst this has proved useful in testing the technology and giving some indication of viewing behaviour, the sample size is still relatively small and cannot be generalised to the whole visitor population, especially as we didn’t examine the effect of when children, groups or families read signs.”

"We note that this is a relatively small study of 51 individuals and tested sign reading presented on a digital screen. While the respondents were visitors to cultural venues, this way of viewing does not represent natural sign reading behaviour and is a limitation of the study. We therefore recommend that future research employs eye-tracking glasses to examine reading behaviour within the context of an exhibit."

Figure 3 the Y axis label should be uppercase to be match with X axis label 

Done

Line 436: preferable to apply a traffic light approach (green being Least Concern and red Critically"

It should be "preferable to apply a traffic light approach (green for Least Concern and red for Critically".

Done

Overall, the grammar is quite good, with only a few minor issues that can be easily corrected.

Also please follow the journal requirements for the references preparation. 

References have been updated in keeping with the ACS style as used by JZBG

Comments on the Quality of English Language

Overall, the grammar is quite good, with only a few minor issues that can be easily corrected.

Figure 3 the Y axis label should be uppercase to be match with X axis label 

Done

Line 436: preferable to apply a traffic light approach (green being Least Concern and red Critically"

It should be "preferable to apply a traffic light approach (green for Least Concern and red for Critically".

Done

We believe that we have addressed all of reviewer 2's comments and thank them for their review.

Reviewer 3 Report

Comments and Suggestions for Authors

The paper explored the use of a remote eye tracking device to create effective signs in botanical gardens and museums. Some comments.

The Conclusion section must be provided separately.

 Describe in more detail whether artificial intelligence was used to create the QR. For example, as in Versailles (France).

 Presentation of information in the form of a thermometer and speedometer is effective compared to traditional captions.

It is necessary to mark the danger posed by predators in red, for example, as in the parks of South Africa.

 It is necessary to expand the experiment on working with dyslexics and colorblind people. It is incorrect to cite 4 people.

The list of references must be brought into line with the requirements of the journal.

Author Response

Reviewer 3

Comments 1

The paper explored the use of a remote eye tracking device to create effective signs in botanical gardens and museums. Some comments.

The Conclusion section must be provided separately.

response 1: Conclusion has been added as follows:

We suggest that information signs should be designed with a focus on the text presented in the 50^th -70^th word of the sign as this section is most accurately recalled and is focused on before the images. This text should convey the key message or objective of the sign. For left-right reading audiences (e.g., English speakers) sign design should prioritise content in the top upper-left with the least important content in the lower-right. These positions may need to be reversed for right-left reading cultures such as for Arabic speakers, but more research in this area is needed. Threat status was best presented using colour thermometers to show threat levels e.g., green (Least Concern) and red (Critically Endangered). In addition, clear fonts with large pictures and blocks of text were well received by our respondents.

We note that this is a relatively small study of 51 individuals and tested sign reading presented on a digital screen. While the respondents were visitors to cultural venues, this way of viewing does not represent natural sign reading behaviour and is a limitation of the study. We recommend that future research employs eye-tracking glasses to examine reading behaviour within the context of an exhibit.

 comment 2: Describe in more detail whether artificial intelligence was used to create the QR. For example, as in Versailles (France).

response 2: The QRs used on the test signs were only dummy QRs and led to a blank holding page. We have added the following sentence in the methods:

“Dummy QR codes were used as we didn’t expect them to be scanned during the experiment (we note that none of the participants attempted to scan the QR during the tests).”

 comment 3: Presentation of information in the form of a thermometer and speedometer is effective compared to traditional captions.

It is necessary to mark the danger posed by predators in red, for example, as in the parks of South Africa.

response 3: We are not sure what the reviewer is referring to by this comment and were therefore unable to address it further

comment 4:  It is necessary to expand the experiment on working with dyslexics and colorblind people. It is incorrect to cite 4 people.

response 4: We have added a little more context to this explanation in the main text (see below). Note that we did not intend to test dyslexia and its impact on sign reading. It was simply an anecdotal comment that was made and may be of interest for future sign research. We have removed mention of dyslexia in the discussion to avoid any confusion. We hope we have now clarified this in the text

‘Sign design D1: large image/block text was preferred by 43.14% of respondents. Of those who preferred Sign 1, the main reasons stated were that ‘it had one big image’ (n=8), that ‘the colour scheme of white writing on a dark green background was very clear ‘(n = 7) and that ‘the text information was all in one place’ (n=6). Of the four participants who self-declared as dyslexic, three selected D1 as their preference with one individual stating that ‘white [writing] on green [background] was recommended to help dyslexics retain information’. We acknowledge that this finding is anecdotal (the sample size is too small to draw conclusions about dyslexic reading behaviour and this is not something we intended to test), however, we include this finding as it hints that preferences for high contrast texts may be universally favoured and warrants further investigation for future studies.’

comment 5:

The list of references must be brought into line with the requirements of the journal.

response 5: References have been updated in keeping with the ACS style as used by JZBG

We thank reviewer 5 for their comments which we believe that we have now addressed

 

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

All detected issues in this version of the manuscript were resolved. I only suggest the authors include the model and brand of the 22' monitor employed in their research. I want to commend the authors for their hard work on this interesting study. I truly enjoyed reading it.