Perceptual Similarities among Wallpaper Group Exemplars

Abstract

Symmetries are abundant within the visual environment, and many animals species are sensitive to visual symmetries. Wallpaper groups constitute a class of 17 regular textures that each contain a distinct combination of the four fundamental symmetries, translation, reflection, rotation and glide reflection, and together represent the complete set of possible symmetries in two-dimensional images. Wallpapers are visually compelling and elicit responses in visual brain areas that precisely capture the symmetry content of each group in humans and other primates. Here we ask to what extent different exemplars from the same wallpaper group are perceptually similar. We used an algorithm to produce a set of well-matched exemplars from 5 of the 17 wallpaper groups and instructed participants to freely sort the exemplars from each group into as many subsets as they wished based on any criteria they saw appropriate. P1, the simplest of the 17 groups, was consistently rated more self-similar than any other group, while the other four groups, although varying in symmetry content, were comparable in self-similarity. Our results suggest that except for the most extreme case (P1), perceived self-similarity of wallpaper groups is not directly tied to categories of symmetry based on group theory.

1. Introduction

Symmetry exists in an object or pattern if a transformation can be applied that maps the object/pattern onto itself. In the two-dimensional plane, the set of isometries–distance–preserving transformations, see [1]—what can give rise to symmetries are translation, reflection, rotation and glide reflection and their combinations. The wallpaper groups are a set of 17 regular textures, where each has a unique combination of isometries that leave the texture unchanged [1,2,3]. Each wallpaper group therefore contains a distinct combination of four symmetry types (see Figure 1). Symmetries have been recognized as important for human visual perception since the late 19th century [4]. Wallpaper groups are different from stimuli typically used to probe the role of symmetry in visual perception in two ways: First, they contain combinations of four symmetry types, rather than just reflection (also called mirror symmetry), which have been the focus of most vision research. Second, in wallpaper groups, symmetries are repeated to tile the plane and form textures, instead of being positioned at a single image location, as is usually the case with standard stimuli. These differences, and the important fact that wallpaper groups together form the complete set of symmetries possible in the two-dimensional image plane, make wallpapers an interesting stimulus set for studying the perception of visual symmetries.

Brain imaging studies using functional MRI [5] and EEG [6,7] have demonstrated that the human visual system carries detailed and precise representations of the symmetries within the individual wallpaper groups. Specifically, response amplitudes scale approximately linearly with the symmetry content within the wallpaper groups, across all of the possible combinations of reflection, rotation and glide reflection symmetries. Functional MRI evidence from macaque monkeys reveal similar representations in the macaque visual system, and the brain regions responding to symmetry are largely analogous between humans and monkeys, namely the functionally defined regions V3, V4, VO1 and LOC [8].

The wallpaper group representations that have been identified using brain imaging are highly complex, but do not appear to be readily available for driving conscious behaviour: Humans have limited intuitive sense of group membership for wallpaper group exemplars, as evidenced by behavioral experiments showing that although naïve observers can distinguish many of the wallpaper groups [9], they tend to sort exemplars into fewer (4–12) sets than the number of wallpaper groups, often placing exemplars from different groups into the same set [10]. Wallpaper groups are nonetheless visually compelling, and anecdotally we have observed that exemplars from a given group can be quite perceptually diverse. This observation inspired the current study. Here, we use behavioral sorting, a common technique to study perceptual categorization [11,12], to probe the perceptual self-similarity of different exemplars from the same wallpaper group. In previous sorting experiments with wallpaper groups (e.g., [10]), observers were shown exemplars from different wallpaper groups, and their ability to correctly sort exemplars from the same group into the same subset was assessed. Our approach was different: We wanted to know the extent to which exemplars from the same group would be spontaneously organized into subsets, i.e., the self-similarity of exemplars from a given group. We selected five distinct wallpaper groups: P1, P3M1, P31M, P6 and P6M (see Figure 2). All wallpaper groups consist of a lattice that is repeated to tile the plane. P1 is the simplest group, and contains no symmetries other than the translation generated by the repeating lattice. P6 has rotation symmetries of order 6, 3 and 2, but no other symmetries besides translation. P3M1 and P31M both have rotations of order 3, reflections in three distinct directions, and glide reflections in three distinct directions, but differ in terms of how these symmetries are organized in the lattice. P6M is the most complex of the groups, it has rotation symmetries of order 6, 3 and 2, reflections in six distinct directions, and glide reflections in six distinct directions. The lattice structure of the five groups is described in detail on the wallpaper group wikipedia page (https://en.wikipedia.org/wiki/Wallpaper_group, accessed on 12 April 2022). The five groups selected have all been found to have high self-similarity [10], and four of them (P3M1, P31M, P6 and P6M) share the same lattice shape (see Figure 2).

Participants were given 20 exemplars, all belonging to same group (see Figure 3 for a selection of the exemplars, and Section 4 for details on how they were created) and asked to freely sort them into as many subsets as they wished. Participants sorted exemplars belonging to five different wallpaper groups, one group at a time. This approach allowed us to compare the five wallpaper groups, both in terms of how many subsets participants generated, and also in terms of the Jaccard index, a summary statistic capturing the similarity across exemplar pairs for each group. Within each group, we were also able to identify exemplar pairs that were rated as highly similar and highly dissimilar. Our main conclusion is that P1 was systematically more self-similar than the any other groups, while the other four groups could not be distinguished on these measures. We also demonstrate that for all five groups, participants consistently group certain pairs of exemplars together, although the number of consistent pairs varies among groups. Our results open the door to further investigations into the psychological and neural mechanisms that drive perceptual similarity among wallpaper group exemplars, and indeed among exemplars from different classes of structured patterns.

2. Results

Wallpaper group P1 was more self-similar than the other four groups. This was evident in the number of sets generated for this group across participants, which was lower for P1 (median = 3) than for the four other groups (median = 4–5, see Figure 4). We confirmed this observation statistically by running a repeated measures analysis of variance (ANOVA), which revealed a significant effect of group ($F(4,124)=7.330,p<0.0001$). Post hoc pairwise t-tests showed that the mean number of sets was lower for P1 than all other groups ($ps<0.005$), but no other means differed (see

Table 1. Results of post hoc pairwise t-tests on number of sets and Jaccard Indices. The degrees of freedom was 945 for the Jaccard Index test. For the number of sets test, the degrees of freedom had to be adjusted to account for the fact that one participant did not report number of sets for P6M (see Section 4), and ranged between 127.0 and 127.1.

	Number of Sets			Jaccard Index
Pairs	t	p	D	t	p	D
P1 vs. P31M	−2.981	0.0034	−0.734	5.641	<0.0001	0.579
P1 vs. P3M1	−3.423	0.0008	−0.843	7.233	<0.0001	0.742
P1 vs. P6	−4.748	<0.0001	−1.169	7.734	<0.0001	0.794
P1 vs. P6M	−4.553	<0.0001	−1.132	6.946	<0.0001	0.713
P31M vs. P3M1	−0.442	0.6595	−0.109	1.592	0.1117	0.163
P31M vs. P6	−1.767	0.0797	−0.435	2.094	0.0366	0.215
P31M vs. P6M	−1.600	0.1120	−0.398	1.305	0.1921	0.134
P3M1 vs. P6	−1.325	0.1875	−0.326	0.502	0.6160	0.051
P3M1 vs. P6M	−1.163	0.2470	0.289	−0.287	0.7745	−0.029
P6 vs. P6M	0.150	0.8814	0.037	−0.788	0.4307	−0.081

).

Next, we computed the Jaccard index (see Section 4) across participants for every pairwise combination of exemplars in each group. This provides a measure of the similarity between exemplars within each group. P1 had systematically higher Jaccard indices than the four other groups (see Figure 4), as confirmed by a repeated measures ANOVA, which revealed a statistically significant effect of group ($F(4,495)=20.178,p<0.0001$). The post hoc t-tests showed that P1 had higher Jaccard indices than all other groups ($ps<0.0001$; see Table 1). The fact that the group (P1) for which fewer subsets were generated also had higher Jaccard indices than the other groups illustrates the inherent link between the two measures. For wallpaper groups where the 20 exemplars are sorted into fewer subsets, each individual exemplar pair is more likely to be a member of the same subset, and less likely to be a member of distinct subsets. This in turn leads to higher Jaccard indices. Our pairwise t-tests also showed that P31M had lower Jaccard indices than P6 ($p=0.037$). This effect does not pass our Bonferroni-corrected threshold for significance ($\alpha <0.005$, but may nonetheless possibly reflect real differences in how consistently exemplars were grouped together across participants. Shortly, we will explore this idea more in depth. Out of the five groups tested, P1 is the only one that can be reliably differentiated based on our measures, being higher in self-similarity among the exemplars, and thus lower in diversity among the exemplars.

In order to quantify the extent to which exemplars were consistently grouped together, we ran a permutation analysis in which exemplar labels were shuffled among the sets generated for each participant (see Section 4). This provides, for each group, the expected distribution of Jaccard indices for every pairwise combination of exemplars, if exemplars were assigned randomly to subsets. Moreover, the analysis allows us to compute an empirical z-score that expressed the extent to which a given pair of exemplars deviates from the random assignment.

Because the random distribution is generated by shuffling exemplars across the specific sets generated by each participant for each group, this z-score is independent of the number of sets. If for a given group, none of the pairs deviate significantly from the random distribution, it would indicate that no exemplar pairs were consistently grouped together across participants. To estimate the extent to which this is the case, we look at the distribution of z-scores across the pairs for each group, as plotted in Figure 6, and count the number of pairs for each group for which the p-value associated with the threshold exceeds a given $\alpha$ value. At a threshold of $\alpha =0.01$, several pairs survive for all groups, and even at a much more conservative criterion of $\alpha =0.0001$, most groups have more than one pairing that survives (see

Table 2. Number of consistent pairings at two different $\alpha$-levels for the five groups.

Consistent Pairings
Group	$\mathit{p}<\mathbf{0.01}$	$\mathit{p}<\mathbf{0.0001}$
P1	6	1
P31M	17	10
P3M1	12	3
P6	17	11
P6M	15	4

). It is worth noting that the latter threshold ($\alpha =0.0001$) is lower than the $\alpha$ associated with a Bonferroni correction within the group, given that there are 190 pairs per group:

$$\alpha =\frac{0.05}{190}=0.0003$$

Thus, we conclude that for several exemplar pairs, participants are consistent in how they tend to pair the exemplars. It is interesting to consider that this measure of consistency might provide another way of differentiating wallpaper groups in terms of perceptual self-similarity. While groups P31M, P3M1, P6 and P6M have comparable Jaccard scores (see Figure 4), they differ in the number of consistent pairings, with P31M and P6 producing more consistent pairs than the other two (see Figure 6).

The Jaccard indices also allow us to focus on exemplar pairs that have a high level of similarity relative to the rest of the pairs in the set. We do this by identifying outlier pairs from each group in terms of Jaccard indices, as identified with stars in Figure 4. Because the Jaccard indices are computed across participants, these outliers are also among the pairs most consistently sorted together, as identified in Figure 6. For each exemplar in each outlier pair, we can visualize the pairwise similarity (as measured by the Jaccard index) to every other exemplar in the set (see Figure 5). That is, we can visualize portions of the network of perceived similarity within a set of exemplars. Future work could probe the extent to which networks of perceived similarity have similar structure across wallpaper groups and examine what perceptual features best account for participants’ perceptions of exemplar similarity.

3. Discussion

Previous work has demonstrated that the visual cortex of both humans and macaque monkeys carries highly detailed representations of the symmetries within wallpaper groups, as evidenced by systematic differences in the magnitude of the response elicited by different groups [5,6,7,8]. This distinction between groups can also be observed in psychophysical threshold measurements [7], although observers may not have a strong awareness of the wallpaper group membership of individual exemplars [10]. In the current study, we explored a new piece of the story about how wallpaper groups are processed by the visual system, namely the issue of how self-similar different exemplars from the same wallpaper group appear to untrained observers. We tested this by asking participants to spontaneously sort 20 exemplars from each of five wallpaper groups into different subsets.

Our first finding concerns the number of subsets generated for each group. We find that P1 is divided into fewer subsets than the other four groups. This indicates that the limited complexity of this group, which contains only translation symmetry, has a direct effect of the number of distinct subsets. The relationship between complexity/symmetry content and number of subsets produced is not straightforward, however, as indicated by the fact that P6M is not consistently grouped into more subsets than P6, P3M1 and P31M, despite the fact that these other groups all contain fewer symmetries than P6M. We speculate that this lack of further differentiation is a result of an upper limit on how additional complexity can influence perceptual self-similarity. However, future work with additional wallpaper groups, including groups that are relatively low in complexity but high in self-similarity (e.g., P2 and PMM, see [10]), is needed to draw firm conclusions about this hypothesis.

It is important to note that P6, P3M1 and P31M all consistently generate weaker brain activity than P6M, and produce higher thresholds in a symmetry detection task [7]. Our results would therefore suggest that there is no clear relationship between the strength of the visual system’s response to symmetries in the wallpaper group, and the perceptual self-similarity of each individual group. Future work should explore this more closely, and look for neural correlates of similarity among exemplars from the same group.

We also computed Jaccard indices that, for every possible exemplar pair, expresses the frequency of those two exemplars being grouped together. As described above, the average Jaccard index for a group is inherently linked to the number of subsets produced for that group, because fewer subsets mean that exemplars are more likely to be made members of the same pair, and less likely to be made members of the same pair. It is therefore not surprising that we find the same general pattern for Jaccard indices and the number of subsets, namely that P1 has higher indices than the other groups. The advantage of the Jaccard indices, however, is that they allow us to conduct a permutation analysis that quantifies the extent to which pairs of exemplars are consistently grouped together across participants, independent of the number of sets produced for a given group. It is important to note that consistency in the choice of which exemplars to group into subsets is not an unavoidable consequence of our experimental design, and it does not follow naturally from the results described so far. It would be perfectly possible for participants to group the sets together, producing fewer subsets for P1 as observed, but exhibit no consistency across participants at all. That is not what we see, however. Even when setting a conservative threshold, all five groups produce one or more pairs that are consistently grouped together, demonstrating that the sorting of exemplars into subsets is not done randomly or arbitrarily across the participants. Rather, different individuals agree to some extent on which exemplars belong together. Because our measure of consistency is independent of the number of subsets produced for a given group, it allows us to show that although P1 has the highest overall Jaccard indices (as a result of the fewer sets produced for this group), it in fact produces fewer consistent pairs than other groups (see Table 2). Indeed, the participants made few comments about their own sorting strategies, but most observed that P1 exemplars were the most difficult to sort because of the lack of readily apparent features that were consistent across exemplars.

In sum, we find consistencies in the way that untrained human observers sort wallpaper images. Observers sort exemplars with translational symmmetry alone (P1) into smaller numbers of sets than exemplars with rotation or reflection symmetry. On average, pairs of P1 exemplars are sorted together more often than exemplars from other wallpaper groups. At the same time, some specific exemplar pairs from wallpaper groups with 3- or 6-fold rotational or reflection symmetry are sorted together substantially more often than predicted by chance.

We note that that the spontaneous sorting task our observers engaged in has less intrinsic structure than some other tasks used to study similar questions such as oddball detection [9,13,14], and thus may involve somewhat different perceptual and cognitive processes. In particular, wallpaper group exemplars have a reduced dimensionality relative to natural objects. Even so, large scale evaluations of how human observers perceive similarity in natural objects yield dimensions that appear to relate to the strict regularities observed in wallpapers: round shape, patterning and repetition [13]. In future work, it would be interesting to explore whether different behavioral tasks yield comparable similarity spaces, or more generally, how task demands shape similarity judgments.

In conclusion, our results suggest that human observers show sensitivity to the dimensions of 2D symmetry (translation, rotation and reflection) embedded in wallpaper exemplars. However, their sorting behavior demonstrates only weak evidence that group-theoretic measures of symmetry influence the perception of self-similarity. These results contribute to a small, but growing literature on the perception of visual aesthetics [15,16,17,18,19], where symmetry is one of many contributing factors.

4. Materials and Methods

4.1. Participants

A total of 33 participants (9 Male, 24 Female), ranging in age between 18 and 35 completed this study. The participants were recruited from the undergraduate participant pool in the Department of Psychology at the The Pennsylvania State University. All participants had self- reported 20/20 or corrected to 20/20 vision. We obtained written informed consent to participate from all participants under procedures approved by the Institutional Review Board of The Pennsylvania State University (#38536). The research was conducted according to the principles expressed in the Declaration of Helsinki. Participants include n = 11 collected and described in [20], plus an additional group collected at a later date using the same protocol.

4.2. Stimulus Generation

Five wallpaper groups (P1, P6, P3M1, P31M and P6M) were selected for use in the study. The selection was motivated partially by the fact that all five groups has previously been demonstrated to be high in self-similarity [10], and partially by the fact that P3M1, P31M, P6 and P6M all share the same lattice shape. We also found it interesting that while P6, P3M1 and P31M differ in their symmetry content, all are subgroups of P6M with index 2, which means that P6M can be generated by adding one additional transformation to P6, P3M1 or P31M [7]. A total of 20 exemplars from each of these five wallpaper groups were generated using a modified version of the methodology developed by Clarke and colleagues [10] that we have described in detail elsewhere [5]. Briefly, exemplars belonging to each group were generated by starting with a random-noise patch, which was then repeated and transformed to tile the image plane, in accordance with the symmetry axes and geometric lattice specific to each group. The use of noise patches as the starting point for stimulus generation makes it possible to create an almost unlimited number of distinct exemplars from each wallpaper group. To make individual exemplars as similar as possible we replaced the power spectrum of each exemplar with the median across exemplars within a group. These images were printed onto white cardstock and cut into squares, allowing participants to manipulate the orientation of the images during the sorting tasks. Five exemplars from each group are shown (in reduced size) in Figure 3.

4.3. Procedure

Participants were presented with the 20 exemplars of a single wallpaper group (i.e., P1, P3M1, P31M, P6, P6M) and instructed to sort them into subsets by placing them into piles. Participants were advised to sort the exemplars into as many piles as they deemed necessary based on whatever criteria they desired. There were no time constraints placed on this sorting task, and the participants were allowed to move exemplars between piles until they were satisfied with their classification. This method was then repeated for the remaining four wallpaper groups for each participant, with group presentation order randomized between participants. These tasks were carried out on a large table with sufficient space to randomly lay out all twenty exemplars of each set, illuminated by normal overhead room lighting. Upon completion of each sorting task, participants were asked to verbalize which features they used to sort the exemplars. After completion of all five sorting tasks, participants were asked whether they had a distinct method for sorting the images, and whether any wallpaper group was particularly easy or difficult to sort.

4.4. Generating the Jaccard Index

The data were prepared for analysis by creating one binary variable for each subset created by each participant within a sorting task. Then, each exemplar was assigned a value of one (1) if it was included in a subset, or a value zero (0) if it was not. Next, the similarity of each pair of exemplars within a sorting task was calculated using the Jaccard index, as a measure of similarity and diversity for the binary data. This index is calculated by the equation

$$J=\frac{x}{x+y+z}$$

with x representing the number of subsets that contained both exemplars, and y and z the number of subsets that contain only one exemplar of the pair [21], across participants. Thus, the Jaccard index is the ratio of the number of subsets containing both exemplars of a pair to the number of subsets containing at least one of the exemplars of a pair, thereby excluding subsets with joint absences.

4.5. Statistical Analysis

We tested for differences between the five wallpaper groups tested in terms of number of sets produced and Jaccard indices by running repeated measures analyses of variance (rmANOVA) with group as a fixed factor and participant as a random factor. We then tested the extent to which differences between specific pairs of wallpaper groups contributed to any rmANOVA effects found, by running post hoc paired t-tests comparing every possible pairing of the wallpaper groups, for both number of sets and Jaccard Indices. Because there were 10 possible pairings of the groups, we applied the Bonferroni-correction and adjusted our $\alpha$-level so that each t-test was only considered significant if $p<0.005$. The number of datasets was missng for one participant for the wallpaper group P6M; thus, the degrees of freedom used in the paired t-tests was adjusted using the Kenward–Roger technique—the default in the emmeans package in R.

We ran a permutation analysis in order to quantify the extent to which pairs of exemplars were consistently grouped together, across participants. This involved generating a randomized dataset, as follows. For each participant and wallpaper group, we randomized which specific exemplars were sorted together. This retained the basic structure of each participants’ sorting data—the number of subsets created—but randomized the relationship between specific wallpaper exemplars that were sorted together across the participants. We then created 1000 such permuted datasets, and calculated the Jaccard index for each exemplar pair within each group for each of the permuted datasets. This permitted the calculation of an empirical Jaccard index based on the permuted data from which distributional statistics such as z could be calculated. The observed Jaccard indices for each exemplar pair were then compared to the empirically-derived reference distribution to determine which exemplar pairs were sorted together more frequently than chance would predict.