Next Article in Journal
Tripartite Dynamic Zero-Sum Quantum Games
Next Article in Special Issue
Novel Features for Binary Time Series Based on Branch Length Similarity Entropy
Previous Article in Journal
Thermal Management and Modeling of Forced Convection and Entropy Generation in a Vented Cavity by Simultaneous Use of a Curved Porous Layer and Magnetic Field
Previous Article in Special Issue
Entropy Analysis of COVID-19 Cardiovascular Signals
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Entropy
Volume 23
Issue 2
10.3390/e23020153
entropy-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Information Theory Approach to Aesthetic Assessment of Visual Patterns

by
Abdullah Khalili
* and
Hamid Bouchachia
Department of Computing and Informatics, Bournemouth University, Poole BH12 5BB, UK
*
Author to whom correspondence should be addressed.
Entropy 2021, 23(2), 153; https://doi.org/10.3390/e23020153
Submission received: 5 December 2020 / Revised: 8 January 2021 / Accepted: 17 January 2021 / Published: 27 January 2021
(This article belongs to the Special Issue Information Entropy Algorithms for Image, Video, and Signal Processing)
Browse Figures
Versions Notes

Abstract

:
The question of beauty has inspired philosophers and scientists for centuries. Today, the study of aesthetics is an active research topic in fields as diverse as computer science, neuroscience, and psychology. Measuring the aesthetic appeal of images is beneficial for many applications. In this paper, we will study the aesthetic assessment of simple visual patterns. The proposed approach suggests that aesthetically appealing patterns are more likely to deliver a higher amount of information over multiple levels in comparison with less aesthetically appealing patterns when the same amount of energy is used. The proposed approach is evaluated using two datasets; the results show that the proposed approach is more accurate in classifying aesthetically appealing patterns compared to some related approaches that use different complexity measures.

1. Introduction

The study of aesthetics started with the work of ancient Greek, and today it is an active research topic in fields as diverse as neuroscience [1], psychology [2], and computer science. Baumgarten [3] suggested that aesthetic appreciation is the result of objective reasoning. Hume [4] took the opposing view that aesthetic appreciation is due to induced feelings. Kant argued that there is a universality aspect to aesthetic [5]. Shelley et al. [6] studied the influence of subjective versus objective factors in aesthetic appreciation. Recent studies on empirical aesthetics [7] show that there is a general agreement on what is considered beautiful and what is not, despite the subjectivity of aesthetic appeal. Measuring the aesthetic appeal of images is beneficial for many applications, such as recommendation and retrieval in multimedia systems. It also plays a key role in enhancing human–computer interaction by improving the attention, engagement, and the overall user experience. The development of a model of aesthetic judgment is also a major challenge in evolutionary art [8,9], where only images with high aesthetic quality should be generated. Automating the aesthetic assessment is still an open problem, and the development of models of aesthetic judgment is the main challenge. In this paper, a novel approach to classifying aesthetically appealing images will be presented. The main contribution of this paper is showing that aesthetically appealing patterns are more likely to deliver a higher amount of information over multiple levels in comparison with less aesthetically appealing patterns when the same amount of energy is used. The proposed approach is evaluated using two datasets.

2. Related Work

Datta et al. [10] extracted 56 visual features from an image and used them to train a statistical model to classify the images as “beautiful” or “ugly”. Some examples of the used features include: mean pixel intensity, relative color frequencies, mean pixel hue, and mean pixel saturation. They also used photographic rules of thumb such as the rule-of-thirds. Other features related to aspect ratio, texture, and low depth-of-field were also used. Ke et al. [11] used features that describe the spatial distribution of color, edges, brightness, and blur. Aydin et al. [12] computed perceptually calibrated ratings for a set of meaningful and fundamental aesthetic attributes such as depth, sharpness, tone and clarity, which together form an “aesthetic signature” of the image. Other works have also investigated the role of photographic composition [13,14,15,16], colour compatibility [17,18,19], and the use of other features such as object types in the scene [20]. Recently, convolutional neural networks (CNNs), which can automatically learn the aesthetic features, have been applied to the aesthetic quality assessment problem [21,22,23,24]; promising results were reported.
This research is more related to the information-theory-based approaches. Birkhoff [25] proposed an aesthetic measure, where the measure of aesthetic quality is in a direct relation to the degree of order O, and in a reverse relation to the complexity C, M = O/C. Eysenck [26,27,28] conducted a series of experiments on Birkhoff’s model; he argued that the aesthetic measure has to be in direct relation to the complexity rather than an inverse relation M = O × C. Javid et al. [29] conducted a survey on the use of entropy to quantify order and complexity; they also proposed a computational measure of complexity. Their measure is based on the information gain from specifying the spatial distribution of pixels and their uniformity and non-uniformity. Franke [30] proposed a model based on psychological experiments, which showed that working memory cannot take in more than 16 bits/s of visual information. He argued that artists should provide an information flow of about 16 bits/s for their works to be perceived as aesthetically appealing and harmonious; see [31] for more recent developments. Al-Rifaie et al. [32] proposed a nature-inspired, swarm intelligence technique to quantify symmetrical complexities in visual patterns. The technique is then used to investigate aesthetically appealing patterns. Javid et al. [33] investigated the use of Kolmogorov complexity and mean information gain to distinguish 2D patterns. The measures were able to distinguish between random patterns and non-random patterns. Datasets such as [34,35,36,37] are collected from communities where images are uploaded and scored in response to photographic challenges. The main limitation of these datasets is that the images are very rich, diverse, and highly subjective, which will make the aesthetic assessment process very complicated. Therefore, the datasets in [38] and [39] will be used in this paper to test the proposed approach. Using simple visual patterns in these two datasets was necessary to simplify the process and filter out unnecessary information as much as possible. The second reason for using these two datasets was to reduce the subjectivity of the assessment process as much as possible by using very simple patterns instead of using real-world images that have a more subjective nature, such as other datasets [34,35,36,37].

3. Proposed Approach

In this section, simple visual patterns will be studied. The images of the dataset in [38], and the images of the dataset in [39] will be used, the dataset in [38] contains two groups of images: the first one is “more aesthetically appealing” images (Figure 1), and the second one is “less aesthetically appealing” images (Figure 2). These two groups are rated by ten persons. The ten persons were asked to give a binary classification of whether each pattern is beautiful or not. If the score (the number of persons who selected the pattern as beautiful) is higher than the average score, then the pattern belongs to the first group, otherwise it belongs to the second group. The dataset contains simple visual patterns generated by the same physical process. The propagation of waves inside geometrical structures could produce very interesting interference patterns, particularly inside symmetrical shapes. The resulted pattern represents the wave interference pattern inside a closed box. Three waves were initiated at the center of the box at different time instances. The first wave was initiated when the value of the counter was 1, the second wave was initiated when the value of the counter was 5000, and the third wave was initiated when the value of the counter was 10,000. The size of the images is 116 × 116 pixels. The images are grayscale images with 256 possible values.
To analyze the images of Figure 1 and Figure 2, if we start from the center of the image to the boundary, we notice that the number of transitions between lighter and darker values is larger for images in Figure 1; furthermore, the intensity of the transitions is higher. This will result in increasing the high-energy part of the distribution of the gradient of the image. Moreover, we notice that the high-energy part of the distributions of the images of Figure 1 is larger than the high-energy part of the distributions of the images in Figure 2 when both have the same amount of energy, and since the largest part of the distribution is located in the low-energy region, this means that increasing the high-energy part of the distribution will increase the entropy.
The basic idea of the proposed approach is that aesthetically appealing patterns have a balance between randomness and regularity, and aesthetically appealing patterns are those which are closer to this optimal point. The entropy and energy will be used as measures of this balance. The resulted distribution of this optimization process can be uniquely identified by maximizing the entropy, given that the energy levels are constant, and the total energy is constant.
The main difference of the proposed approach in comparison with existing approaches to aesthetic assessment of visual patterns is the use of a statistical mechanics formulation. The main reason for using this formulation is that it provides a link between the energy and the entropy, which was a crucial link to constrain the complexity of the pattern by the energy, and hence achieve a balance between randomness and regularity; this balance was also suggested by many researchers [40,41,42,43]. The approach does not assume any link to statistical mechanics, it only uses the same mathematical formulation.
Figure 3 shows the Maxwell–Boltzmann distribution, Figure 4 shows the distribution of the gradient of one image in the dataset; the same distribution has shown up for all the images in the dataset. We can observe the similarity between the resulting distribution and the Maxwell–Boltzmann distribution. Furthermore, using the above analysis, our problem now is exactly the same problem that Boltzmann [44] solved to derive the distribution of the energies of gas particles at equilibrium. Boltzmann argued that the Maxwell-Boltzmann distribution [45,46] is the most probable distribution and it will arise by maximizing the multiplicity (which is the number of ways the particles can be arranged); assuming that the number of particles is constant, as described by (1), the energy levels that the particles can take are constant, as described by (2), and the total energy is constant, as described by (3). The multiplicity is given by (4), and the entropy is given by (5)
∑ i n i = C o n s t a n t
ε 1 ,   ε 2 ,   … ,   ε N     a r e   c o n s t a n t
Energy   = ∑ i n i ε i = C o n s t a n t
Ω =   N ! n 1 ! n 2 ! … . n n !
Entropy   = log ( Ω )
where N is the total number of particles, n i is the number of particles at the ε i energy level. Maximizing the entropy is equivalent to maximizing the multiplicity. By taking ln (Ω), we get
ln ( Ω )   = ln ( N ! ) − ∑ i ln ( n i ! )
Using Stirling approximation, Equation (6) can be rewritten as follows
ln ( Ω ) = N ln ( N ) − N − ∑ i [ n i ln ( n i ) −   n i ]
The Maxwell–Boltzmann distribution gives the number of particles at each energy level. Using the Lagrange multiplier method to maximize the entropy using the constraints in (1)–(3), we get
n i =   e − α − β ε i
where α , β are the Lagrange multipliers. The distribution in 3D and 2D spaces can be written in the form given by (9) and (10), respectively,
f ( v ) = ( m 2 π k T ) 3 2 4 π v 2   e −   m v 2 2 k T
f ( v ) = ( m 2 π k T ) 2 π v   e −   m v 2 2 k T
where v is the speed of the particle, m is the mass of the particle, T is the temperature and k is Boltzmann constant. The distribution is shown in Figure 3.
Similarly, for images, the energy levels ε 1 ,   ε 2 ,   … ,   ε n are the values which the pixels can take; for grayscale images, the values are 0, 1, 2, …, 255. The energy levels must be constant, as described in (11); n i is the number of pixels at the energy level   ε i , the total number of pixels should also be constant, as described in (12). Finally, the total energy, which is given by (13), must also be constant.
ε 1 ,   ε 2 ,   … ,   ε n   a r e   c o n s t a n t
∑ i n i = C o n s t a n t
Energy   = ∑ i n i ε i = C o n s t a n t
The constraints given in (11)–(13) are exactly the same constraints used by Boltzmann to derive the Maxwell–Boltzmann distribution, and by maximizing the entropy, the same distribution as given by (8)–(10) will arise. Maximizing the entropy will result in a flat distribution; however, the constant energy constraint will produce a balance between order and randomness. Maximizing the entropy using constant energy can then be seen as delivering the highest possible amount of information using the same amount of energy. Figure 4 shows the distribution of the gradient of an image in the dataset. The Matlab gradient function is used to calculate the gradient, and then the resulting values are converted to polar format. Figure 5 shows the distribution of the gradient of the gradient of the same image.
The same distribution has appeared for all the gradient of the images, and the gradient of the gradient of the images, which may suggest that the same law must be satisfied at each level. The multiple-levels approach will be used to cope with energy and entropy limitation in representing the spatial arrangement of the pattern. Due to the complexity of the structure of the visual patterns, the gradient over multiple levels will be used to represent the spatial arrangement of the visual patterns, where the first level represents the image, the second level represents the gradient of the image, and the third level represents the gradient of the gradient of the image. The measures of aesthetic quality M propose that the sum of the entropies of the three levels should be maximum when the energies of the three levels are the same. The measure is given by (14)
M = ∑ i = 1 , 3 Entropy ( Li )
where L1 is the image, L2 is the gradient of the image, and L3 is the gradient of the gradient of the image. Entropy is Shannon entropy (using Stirling approximation, Shannon entropy can be used instead of Boltzmann entropy), and the energies of the three levels must be the same. Figure 6 shows the M values of images in Figure 1 and Figure 2, along with other images in the same category.
However, comparing images that have the same energy at each level is rather limited; furthermore, the above analysis does not say anything about the relation between the energies of different levels. Figure 7 shows the sum of the distances between the energies of different levels for images in Figure 1 and Figure 2, along with other images in the same category.
The blue circles represent the images of Figure 1, and the red stars represent the images of Figure 2, along with other images in the same category. The distances of aesthetically appealing images are different from the distances of the less aesthetically appealing images. To relax the above constraint, and to be able to compare images that have the same first-level energy only, the aesthetically appealing images at different energy levels of Figure 1 are used as reference images, and the distances between the energies of the tested image should be as close as possible to the distances of the reference image Ri, as described by (15); furthermore, the equation described by (14) should be also satisfied. In other words, M should be maximized and Md should be minimized
Md = | ∑ i Distance ( Ri )   −   ∑ i Distance ( Li ) |
where Distance(Ri) is the distance between the energy of the ith level and the energy of the i + 1 level, and the energy of the first level only should be the same. The metrics will be calculated on the center part of the image, since it gets most of the attention, where 20 pixels from each side of the image will be neglected. Figure 8 shows the combination of the two metrics where the sum of the entropies and the energies of the three levels is shown after scaling each energy and entropy to value between 0 and 1.

4. Results

Due to the small number of images in the two datasets, the proposed approach cannot be compared to deep-learning-based approaches [47,48,49,50,51]. The proposed approach will be compared with three related approaches; the first one is based on the Birkhoff model [52,53], where Shannon entropy and image compressibility are used to represent the order and complexity of the Birkhoff model. Figure 9 shows the Shannon entropy and image compressibility (the ratio between the original and the compressed image using the JPEG method) for the images of Figure 1 and Figure 2. The results show that the two groups of images cannot easily be classified using this approach.
Then, the proposed approach will be compared with an approach based on Benford law [54], where the histogram of the image is compared with the histogram described by Benford law. Figure 10 shows the difference between the histograms of the images of Figure 1 and Figure 2, and the histogram described by Benford law. The results also show that the two groups of images cannot be easily classified using this approach.
To further test the proposed approach, we will test it on the dataset proposed in [39]. Figure 11 shows the patterns of the set. In Figure 11a, the first two lines represent asymmetrical patterns and the last two lines represent symmetrical patterns. Fifty-five persons rated the patterns; the patterns start from not beautiful (left), and move to beautiful (right) line by line. In Figure 11b, the first three lines represent symmetrical patterns and the last three lines represent asymmetrical patterns, ordered in lines from the most beautiful pattern starting in the upper left corner to the least beautiful pattern. The number next to each pattern in Figure 12, Figure 13, Figure 14 and Figure 15 represents the line number and the position of the pattern in the line (starting from left to right). For instance, 43 is the third pattern in line four.
Figure 12 shows the energy and the entropy of the first level; the results show that the symmetrical patterns of line 3 and line 4 have higher entropy than the asymmetrical patterns when the same energy is used. This matches with the rating given by the fifty-five persons and with several studies [55,56,57,58], which showed consistent preferences for symmetry. The patterns 41, 42, and 43 have roughly the same energy, but the entropy of 43 is larger than the entropy of 42, which is larger than the entropy of 41.
Figure 13 shows the sum of the entropies of the first two levels after converting all levels to black and white images; again, the symmetrical patterns of line 3 and line 4 have a higher sum than the other patterns when the same energy is used. For instance, patterns 13, 32, and 33 have roughly the same energy, but the sum of 33 is larger than the sum of 32, which is larger than the sum of 13. This also matches with the rating of the fifty-five persons. We can also see that the patterns 11 and 21 have lower sum than the other patterns.
Figure 14 shows the distance between the energies of the first two levels. The symmetrical patterns of line 3 and line 4 have a lower distance than the other patterns when the same energy is used. For instance, the patterns 13, 32, and 33 have roughly the same energy, but the distance of 33 is lower than the distance of 32, which is lower than the distance of 13. The patterns 41, 42, and 43 also have roughly the same energy, but the distance of 43 is lower than the distance of 42; however, 42 has higher distance than 41. We can also see that the patterns 11 and 21 have higher distance than the other patterns. These results show a close match with the rating given by the fifty-five persons.
Figure 15 shows the results using the images in Figure 11b; again, the symmetrical patterns of the first three lines show a higher sum than other asymmetrical patterns when the same energy is used; however, there are some differences between the sum and the ranking of the users within these two groups.
Figure 16 shows the results of applying an information-gain-based approach proposed in [29] for the images in Figure 11a. The results show a link between information gain and empirical aesthetic judgement in the case of the asymmetrical patterns, but not for the symmetrical patterns. We can see that the ordering of the information gain for the asymmetrical patterns agrees with the users’ rating. However, this is not the case for the symmetrical patterns.
Figure 17 and Figure 18 show the algorithmic complexity as approximated by the Block Decomposition Method (BDM) [59,60] after converting the patterns to black and white, since the method does not yet support a large number of values. The method performs better on the first dataset than the second dataset, where we can see that aesthetically appealing patterns tend to have a higher complexity. However, it is still clear that this method is less accurate in comparison with the proposed approach.

5. Discussion

The results show that the proposed model is more accurate at classifying aesthetically appealing visual patterns. The results suggest that aesthetically appealing patterns of the two datasets are more likely to deliver a higher amount of information in comparison with less aesthetically appealing patterns when the same amount of energy is used. The results also suggest that the distances between the energies of the levels are more likely to be different for aesthetically appealing patterns. One limitation of the proposed approach is that few aesthetically appealing patterns show a lower M value and higher Md value than the less aesthetically appealing patterns, as can be seen in Figure 8. Future work will improve the proposed model to increase the classification accuracy.
To give a more intuitive analysis of the results, we will take two extreme cases: the first one is an image with only one color, and the second one is an image with equal probabilities for all colors. The first case will produce a distribution of one pulse at one energy level, while the second case will produce a flat distribution. In the case of music, the first case will give a piece with only one note repeated many times, and the second case will produce a piece with all possible notes; in both cases, no aesthetically appealing patterns will be produced, as the first pattern will be too regular and the second one will be too random. The aesthetically appealing patterns represent a balance between these two extreme cases, and the closer we get to the Maxwell–Boltzmann distribution, the higher the aesthetic score of the pattern. Now, if we take one aesthetically appealing pattern and rearrange the pixels randomly, we will obtain a random pattern that has the same distribution; however, the gradient (which decodes the spatial distribution of pixels) of this random pattern will produce a distribution closer to the flat distribution than the gradient of the original pattern. Similarly, if we arrange the aesthetically appealing pattern such that the pixels with the same values are close to each other, the gradient of the resulted pattern will produce a distribution closer to a pulse than the gradient of the original pattern, and again the distribution of the gradient of aesthetically appealing patterns represents a balance between these two extreme cases, and the closer we get to the Maxwell–Boltzmann distribution, the higher the aesthetic score of the pattern.
The proposed approach agrees with the intuition of many scientists, who link the concept of beauty with the ability to cover the largest possible number of empirical facts using the smallest possible number of axioms or hypotheses [61,62]. Similarly, the proposed approach suggests that aesthetically appealing patterns should deliver the largest possible amount of information using the same amount of energy. The relation between aesthetically appealing patterns and the balance between randomness and regularity was also suggested by many researchers. The proposed approach uses a statistical mechanics formulation that links the energy and the entropy, which was a crucial link to constrain the complexity of the pattern by the energy, and hence achieve a balance between randomness and regularity.
The proposed approach has shown an interesting link between information theory and the aesthetic rating of the users of the two datasets. The meaning and the deeper relation of the link between information theory and aesthetic are to be further investigated in future work. Finding the most fundamental law or the optimization process that underlies aesthetically appealing patterns would be of great interest for the research in this area and for many applications [63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79]. It is interesting to see whether the proposed approach has any link to the aesthetic judgment mechanism in the brain, and how is that related to information theory. Pursuing these research directions holds a great promise for a deeper understanding of many important phenomena.

6. Conclusions

A novel approach to classify aesthetically appealing images was presented in this paper. The proposed approach showed that aesthetically appealing images of the two datasets are more likely to deliver a higher amount of information over multiple levels in comparison with less aesthetically appealing images when the same amount of energy is used. The results have shown that the proposed approach was more accurate in classifying aesthetically appealing patterns. Future work will try to apply this approach to other types of images.

Author Contributions

Conceptualization, A.K.; methodology, A.K.; validation, H.B., A.K.; investigation, A.K.; writing—original draft preparation, A.K.; writing—review and editing, H.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chatterjee, A. Neuroaesthetics: A Coming of Age Story. J. Cogn. Neurosci. 2011, 23, 53–62. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Leder, H.; Belke, B.; Oeberst, A.; Augustin, D. A model of aesthetic appreciation and aesthetic judgments. Br. J. Psychol. 2004, 95, 489–508. [Google Scholar] [CrossRef] [PubMed]
  3. Hammermeister, K. The German Aesthetic Tradition; Cambridge University Press: Cambridge, UK, 2002. [Google Scholar]
  4. Gracyk, T. Hume’s aesthetics. In Stanford Encyclopedia of Philosophy; Stanford University: Stanford, CA, USA, 2011. [Google Scholar]
  5. Burnham, D. Kant’s aesthetics. In Internet Encyclopedia of Philosophy; IEP: Martin, TN, USA, 2001. [Google Scholar]
  6. Shelley, J. The concept of the aesthetic. In Stanford Encyclopedia of Philosophy; Stanford University: Stanford, CA, USA, 2012. [Google Scholar]
  7. Vessel, E.A.; Rubin, N. Beauty and the beholder: Highly individual taste for abstract but not real-world images. J. Vis. 2010, 10, 18. [Google Scholar] [CrossRef] [PubMed]
  8. McCormack, J. Facing the Future: Evolutionary Possibilities for Human-Machine Creativity; Springer Science and Business Media LLC: Berlin/Heidelberg, Germany, 2008; pp. 417–451. [Google Scholar]
  9. Latham, W.H.; Todd, S. Computer sculpture. IBM Syst. J. 1989, 28, 682–688. [Google Scholar] [CrossRef]
  10. Datta, R.; Joshi, D.; Li, J.; Wang, J.Z. Studying Aesthetics in Photographic Images Using a Computational Approach. In Lecture Notes in Computer Science; Springer: Springer, Berlin, Heidelberg, 2006; pp. 288–301. [Google Scholar]
  11. Ke, Y.; Tang, X.; Jing, F. The Design of High-Level Features for Photo Quality Assessment. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition—Volume 2 (CVPR’06), New York, NY, USA, 17–22 June 2006; pp. 288–301. [Google Scholar] [CrossRef] [Green Version]
  12. Aydin, T.O.; Smolic, A.; Gross, M. Automated Aesthetic Analysis of Photographic Images. IEEE Trans. Vis. Comput. Graph. 2015, 21, 31–42. [Google Scholar] [CrossRef]
  13. Bhattacharya, S.; Sukthankar, R.; Shah, M. A framework for photo-quality assessment and enhancement based on visual aesthetics. In Proceedings of the International Conference on Big Data and Internet of Thing—BDIOT2017, Firenze, Italy, 25–29 October 2010; pp. 271–280. [Google Scholar]
  14. Liu, Y.-J.; Luo, X.; Xuan, Y.-M.; Chen, W.-F.; Fu, X.-L. Image Retargeting Quality Assessment. Comput. Graph. Forum 2011, 30, 583–592. [Google Scholar] [CrossRef] [Green Version]
  15. Liu, L.; Jin, Y.; Wu, Q. Realtime aesthetic image retargeting. Comput. Aesthet. 2010, 10, 1–8. [Google Scholar]
  16. Liu, L.; Chen, R.; Wolf, L.; Cohen-Or, D. Optimizing Photo Composition. Comput. Graph. Forum 2010, 29, 469–478. [Google Scholar] [CrossRef]
  17. O’Donovan, P.; Agarwala, A.; Hertzmann, A. Color compatibility from large datasets. Acm Trans. Graph. 2011, 30, 1–12. [Google Scholar] [CrossRef]
  18. Cohen-Or, D.; Sorkine, O.; Gal, R.; Leyvand, T.; Xu, Y.-Q. Color harmonization. In Proceedings of the IGGRAPH06: Special Interest Group on Computer Graphics and Interactive Techniques Conference, Boston, MA, USA, 30–31 July 2006; pp. 624–630. [Google Scholar]
  19. Nishiyama, M.; Okabe, T.; Sato, I.; Sato, Y. Aesthetic quality classification of photographs based on color harmony. In Proceedings of the CVPR 2011, Providence, RI, USA, 20–25 June 2011; pp. 33–40. [Google Scholar]
  20. Dhar, S.; Ordonez, V.; Berg, T.L. High level describable attributes for predicting aesthetics and interestingness. In Proceedings of the CVPR 2011, Providence, RI, USA, 20–25 June 2011; pp. 1657–1664. [Google Scholar]
  21. Lu, X.; Lin, Z.; Jin, H.; Yang, J.; Wang, J.Z. Rapid: Rating pictorial aesthetics using deep learning. In Proceedings of the 22nd ACM international conference on Multimedia, Orlando, FL, USA, 7 November 2014; pp. 457–466. [Google Scholar]
  22. Kao, Y.; Wang, C.; Huang, K. Visual aesthetic quality assessment with a regression model. In Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Institute of Electrical and Electronics Engineers (IEEE), Quebec City, QC, Canada, 27–30 September 2015; pp. 1583–1587. [Google Scholar]
  23. Lu, X.; Lin, Z.; Shen, X.; Mech, R.; Wang, J.Z. Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 990–998. [Google Scholar]
  24. Mai, L.; Jin, H.; Liu, F. Composition-preserving deep photo aesthetics assessment. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 497–506. [Google Scholar]
  25. Birkhoff, G.D. Aesthetic Measure; Harvard University Press: Cambridge, MA, USA, 2013. [Google Scholar]
  26. Eysenck, H.J. An Experimental Study of Aesthetic Preference for Polygonal Figures. J. Gen. Psychol. 1968, 79, 3–17. [Google Scholar] [CrossRef]
  27. Eysenck, H.J. The empirical determination of an aesthetic formula. Psychol. Rev. 1941, 48, 83. [Google Scholar] [CrossRef]
  28. Eysenck, H.J. The experimental study of the ’good Gestalt’—A new approach. Psychol. Rev. 1942, 49, 344–364. [Google Scholar] [CrossRef]
  29. Javid, M.A.J.; Blackwell, T.; Zimmer, R.; Al-Rifaie, M.M. Correlation between Human Aesthetic Judgement and Spatial Complexity Measure. In Proceedings of the Evolutionary and Biologically Inspired Music, Sound, Art and Design, Porto, Portugal, 30 March 30–1 April 2016; Volume 9596, pp. 79–91. [Google Scholar]
  30. Franke, H.W. A Cybernetic Approach to Aesthetics. Leonardo 1977, 10, 203. [Google Scholar] [CrossRef]
  31. Martín, F.M.D.P. The thermodynamics of human reaction times. arXiv 2009, arXiv:0908.3170. [Google Scholar]
  32. Al-Rifaie, M.M.; Ursyn, A.; Zimmer, R.; Javid, M.A.J.; Correia, J.; Ciesielski, V.; Liapis, A. On Symmetry, Aesthetics and Quantifying Symmetrical Complexity. In Proceedings of the Lecture Notes in Computer Science, Amsterdam, Netherlands, 19–21 April 2017; Volume 10198, pp. 17–32. [Google Scholar]
  33. Ali Javaheri Javid, M.; Blackwell, T.; Zimmer, R.; Majid al-Rifaie, M. Analysis of information gain and Kolmogorov complexity for structural evaluation of cellular automata configurations. Connect. Sci. 2016, 28, 155–170. [Google Scholar] [CrossRef]
  34. Murray, N.; Marchesotti, L.; Perronnin, F. AVA: A large-scale database for aesthetic visual analysis. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 16–21 June 2012; pp. 2408–2415. [Google Scholar]
  35. Jin, X.; Wu, L.; Zhao, G.; Zhou, X.; Zhang, X.; Li, X. IDEA: A new dataset for image aesthetic scoring. Multimed. Tool Appl. 2020, 79, 14341–14355. [Google Scholar] [CrossRef]
  36. Muller, T.D.; Clough, P.; Caput, B. Experimental Evaluation in Visual Information Retrieval; The Information Retrieval Series; Springer: Berlin/Heidelberg, Germany, 2010. [Google Scholar]
  37. Tang, X.; Luo, W.; Wang, X. Content-Based Photo Quality Assessment. IEEE Trans. Multimed. 2013, 15, 1930–1943. [Google Scholar] [CrossRef] [Green Version]
  38. Khalili, A.M. On the mathematics of beauty: Beautiful images. arXiv 2017, arXiv:1705.08244v5. [Google Scholar]
  39. Jacobsen, T. Beauty and the brain: Culture, history and individual differences in aesthetic appreciation. J. Anat. 2010, 216, 184–191. [Google Scholar] [CrossRef]
  40. Manaris, B.; Romero, J.; Machado, P.; Krehbiel, D.; Hirzel, T.; Pharr, W.; Davis, R.B. Zipf’s law, music classification, and aesthetics. Comput. Music J. 2005, 29, 55–69. [Google Scholar] [CrossRef]
  41. Arnheim, R. Art and Visual Perception: A Psychology of the Creative Eye; University of California Press: Berkeley, CA, USA, 1954. [Google Scholar]
  42. Arnheim, R. Towards a Psychology of Art/Entropy and Art an Essay on Disorder and Order; The Regents of the University of California: Oakland, CA, USA, 1966. [Google Scholar]
  43. Arnheim, R. Visual Thinking; University of California Press: Berkeley, CA, USA, 1969. [Google Scholar]
  44. Boltzmann, L. Über die Beziehung zwischen dem zweiten Hauptsatz der mechanischen Wärmetheorie und der Wahr-scheinlichkeitsrechnung respektive den Sätzen über das Wärmegleichgewicht. Sitz. Kaiserlichen Akad. Wiss. Wien Math.-Nat. Cl. 1909, 76, 373–435, Reprinted in Wiss. Abh. 1909, II, 164–223. [Google Scholar]
  45. Maxwell, J.C.V. Illustrations of the dynamical theory of gases.—Part I. On the motions and collisions of perfectly elastic spheres. Lond. Edinb. Dublin Philos. Mag. J. Sci. 1860, 19, 19–32. [Google Scholar] [CrossRef]
  46. Maxwell, J.C. Illustrations of the dynamical theory of gases. Part II. On the process of diffusion of two or more kinds of moving particles among one another. Lond. Edinb. Dublin Philos. Mag. J. Sci. 1860, 20, 21–37. [Google Scholar] [CrossRef]
  47. Talebi, H.; Milanfar, P. NIMA: Neural Image Assessment. IEEE Trans. Image Process. 2018, 27, 3998–4011. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  48. Bodini, M. Will the Machine Like Your Image? Automatic Assessment of Beauty in Images with Machine Learning Techniques. Inventions 2019, 4, 34. [Google Scholar] [CrossRef] [Green Version]
  49. Chong, N.; Wong, L.K.; See, J. GANmera: Reproducing Aesthetically Pleasing Photographs Using Deep Adver-sarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–17 June 2019. [Google Scholar]
  50. Sandoval, C.; Pirogova, E.; Lech, M. Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings. IEEE Access 2019, 7, 41770–41781. [Google Scholar] [CrossRef]
  51. Cetinic, E.; Lipic, T.; Grgic, S. Fine-tuning Convolutional Neural Networks for fine art classification. Expert Syst. Appl. 2018, 114, 107–118. [Google Scholar] [CrossRef]
  52. Rigau, J.; Feixas, M.F.; Sbert, M. Informational Aesthetics Measures. IEEE Eng. Med. Boil. Mag. 2008, 28, 24–34. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Sahyun, M.R.V. Aesthetics and entropy III. Aesthetic measures. Preprints 2018. [Google Scholar] [CrossRef]
  54. Heijer, E.D.; Eiben, A.E. Using aesthetic measures to evolve art. In Proceedings of the IEEE Congress on Evolutionary Computation, Barcelona, Spain, 18–23 July 2010. [Google Scholar] [CrossRef]
  55. Gartus, A.; Leder, H. The Small Step toward Asymmetry: Aesthetic Judgment of Broken Symmetries. i-Perception 2013, 4, 361–364. [Google Scholar] [CrossRef] [PubMed]
  56. Hofel, L.; Jacobsen, T. Electrophysiological indices of processing symmetry and aesthetics: A result of judgment categori-zation or judgment report? J. Psychophysiol. 2007, 21, 9–21. [Google Scholar] [CrossRef]
  57. Tinio, P.P.L.; Leder, H. Just how stable are aesthetic features? Symmetry, complexity and the jaws of massive familiariza-tion. Acta Psychol. 2009, 130, 241–250. [Google Scholar] [CrossRef] [PubMed]
  58. Tinio, P.P.; Gerger, G.; Leder, H. Birds of a feather… Generalization of facial structures following massive familiarization. Acta Psychol. 2013, 144, 463–471. [Google Scholar] [CrossRef] [PubMed]
  59. Delahaye, J.-P.; Zenil, H. Numerical evaluation of algorithmic complexity for short strings: A glance into the innermost structure of randomness. Appl. Math. Comput. 2012, 219, 63–77. [Google Scholar] [CrossRef] [Green Version]
  60. Soler-Toscano, F.; Zenil, H.; Delahaye, J.-P.; Gauvrit, N. Calculating Kolmogorov Complexity from the Output Frequency Distributions of Small Turing Machines. PLoS ONE 2014, 9, e96223. [Google Scholar] [CrossRef] [Green Version]
  61. Simplicity, Stanford Encyclopedia of Philosophy. Available online: https://plato.stanford.edu/entries/simplicity/ (accessed on 7 January 2021).
  62. Simplicity in the Philosophy of Science, Internet Encyclopedia of Philosophy. Available online: https://iep.utm.edu/simplici/ (accessed on 7 January 2021).
  63. Miniukovich, A.; De Angeli, A. Quantification of interface visual complexity. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces—AVI’14, Como, Italy, 27–30 May 2014; pp. 153–160. [Google Scholar]
  64. Brachmann, A.; Redies, C. Computational and Experimental Approaches to Visual Aesthetics. Front. Comput. Neurosci. 2017, 11, 102. [Google Scholar] [CrossRef]
  65. Ahmed, S.U.; Al Mahmud, A.; Bergaust, K. Aesthetics in Human-Computer Interaction: Views and Reviews. In Human-Computer Interaction; Jacko, J.A., Ed.; Springer: Berlin/Heidelberg, Germany, 2009; Volume 5610. [Google Scholar]
  66. Maity, R.; Bhattacharya, S. Is My Interface Beautiful?—A Computational Model-Based Approach. IEEE Trans. Comput. Soc. Syst. 2019, 6, 149–161. [Google Scholar] [CrossRef]
  67. Maity, R.; Bhattacharya, S. A Quantitative Approach to Measure Webpage Aesthetics. Int. J. Technol. Hum. Interact. 2020, 16, 53–68. [Google Scholar] [CrossRef]
  68. Miniukovich, A.; Marchese, M. Relationship between Visual Complexity and Aesthetics of Webpages. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25 April 2020; pp. 1–13. [Google Scholar]
  69. Cetinic, E.; Lipic, T.; Grgic, S. A deep learning perspective on beauty, sentiment, and remembrance of art. IEEE Access. 2019, 7, 73694–73710. [Google Scholar] [CrossRef]
  70. Santos, I.; Castro, L.; Rodriguez-Fernandez, N.; Torrente-Patiño, Á.; Carballal, A. Artificial Neural Networks and Deep Learning in the Visual Arts: A review. Neural Comput. Appl. 2021, 1–37. [Google Scholar] [CrossRef]
  71. Yue, L.; Chao, G.; Yi-Lun, L.; Fan, Z.; Fei-Yue, W. Computational aesthetics of fine art paintings: The state of the art and outlook. Acta Automatica Sinica. 2020, 46, 2239–2259. [Google Scholar]
  72. Takimoto, H.; Omori, F.; Kanagawa, A. Image Aesthetics Assessment Based on Multi-stream CNN Architecture and Saliency Features. Appl. Artif. Intell. 2021, 35, 25–40. [Google Scholar] [CrossRef]
  73. Deng, Y.; Loy, C.C.; Tang, X. Image aesthetic assessment: An experimental survey. IEEE Signal Process. Mag. 2017, 34, 80–106. [Google Scholar] [CrossRef]
  74. Debnath, S.; Changder, S. Computational Approaches to Aesthetic Quality Assessment of Digital Photographs: State of the Art and Future Research Directives. Pattern Recognit. Image Anal. 2020, 30, 593–606. [Google Scholar] [CrossRef]
  75. Deng, Y.; Loy, C.C.; Tang, X. Aesthetic-driven image enhancement by adversarial learning. In Proceedings of the 26th ACM international conference on Multimedia, Seoul, Korea, 22–26 October 2018; pp. 870–878. [Google Scholar]
  76. Wang, W.; Shen, J.; Ling, H. A deep network solution for attention and aesthetics aware photo cropping. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 41, 1531–1544. [Google Scholar] [CrossRef] [PubMed]
  77. Zhai, G.; Min, X. Perceptual image quality assessment: a survey. Sci. China Inf. Sci. 2020, 63, 1–52. [Google Scholar] [CrossRef]
  78. Jin, X.; Wu, L.; Li, X.; Chen, S.; Peng, S.; Chi, J.; Ge, S.; Song, C.; Zhao, G. Predicting aesthetic score distribution through cumulative jensen-shannon divergence. In Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018. [Google Scholar]
  79. Niu, Y.; Zhong, Y.; Guo, W.; Shi, Y.; Chen, P. 2D and 3D image quality assessment: A survey of metrics and challenges. IEEE Access. 2018, 7, 782–801. [Google Scholar] [CrossRef]
Entropy 23 00153 g001 550
Figure 1. Images in the first group.
Figure 1. Images in the first group.
Entropy 23 00153 g001
Entropy 23 00153 g002 550
Figure 2. Images in the second group.
Figure 2. Images in the second group.
Entropy 23 00153 g002
Entropy 23 00153 g003 550
Figure 3. The Maxwell–Boltzmann distribution for different temperature values.
Figure 3. The Maxwell–Boltzmann distribution for different temperature values.
Entropy 23 00153 g003
Entropy 23 00153 g004 550
Figure 4. The distribution of the gradient of one image in the dataset.
Figure 4. The distribution of the gradient of one image in the dataset.
Entropy 23 00153 g004
Entropy 23 00153 g005 550
Figure 5. The distribution of the gradient of the gradient of one image in the dataset.
Figure 5. The distribution of the gradient of the gradient of one image in the dataset.
Entropy 23 00153 g005
Entropy 23 00153 g006 550
Figure 6. The M values of images in Figure 1 and Figure 2.
Figure 6. The M values of images in Figure 1 and Figure 2.
Entropy 23 00153 g006
Entropy 23 00153 g007 550
Figure 7. The sum of the distances between the energies of different levels for images in Figure 1 and Figure 2.
Figure 7. The sum of the distances between the energies of different levels for images in Figure 1 and Figure 2.
Entropy 23 00153 g007
Entropy 23 00153 g008 550
Figure 8. The sum of the entropies and energies of the three levels of images in Figure 1 and Figure 2.
Figure 8. The sum of the entropies and energies of the three levels of images in Figure 1 and Figure 2.
Entropy 23 00153 g008
Entropy 23 00153 g009 550
Figure 9. Shannon entropy vs image compressibility for images of Figure 1 and Figure 2.
Figure 9. Shannon entropy vs image compressibility for images of Figure 1 and Figure 2.
Entropy 23 00153 g009
Entropy 23 00153 g010 550
Figure 10. The difference between the histograms of the images of Figure 1 and Figure 2, and the histogram described by Benford law.
Figure 10. The difference between the histograms of the images of Figure 1 and Figure 2, and the histogram described by Benford law.
Entropy 23 00153 g010
Entropy 23 00153 g011 550
Figure 11. Patterns from the dataset proposed in [39], (a) ordered from not beautiful (left) to beautiful (right) line by line. (b) Ordered from beautiful (left) to not beautiful (right) line by line.
Figure 11. Patterns from the dataset proposed in [39], (a) ordered from not beautiful (left) to beautiful (right) line by line. (b) Ordered from beautiful (left) to not beautiful (right) line by line.
Entropy 23 00153 g011
Entropy 23 00153 g012 550
Figure 12. The energy and the entropy of the first level of the images in Figure 11a.
Figure 12. The energy and the entropy of the first level of the images in Figure 11a.
Entropy 23 00153 g012
Entropy 23 00153 g013 550
Figure 13. The sum of the entropies of the first two levels of the images in Figure 11a.
Figure 13. The sum of the entropies of the first two levels of the images in Figure 11a.
Entropy 23 00153 g013
Entropy 23 00153 g014 550
Figure 14. The distance between the energies of the first two levels of the images in Figure 11a.
Figure 14. The distance between the energies of the first two levels of the images in Figure 11a.
Entropy 23 00153 g014
Entropy 23 00153 g015 550
Figure 15. The sum of the entropies and the energies of the three levels of the images in Figure 11b.
Figure 15. The sum of the entropies and the energies of the three levels of the images in Figure 11b.
Entropy 23 00153 g015
Entropy 23 00153 g016 550
Figure 16. The mean information gain of the images in Figure 11a.
Figure 16. The mean information gain of the images in Figure 11a.
Entropy 23 00153 g016
Entropy 23 00153 g017 550
Figure 17. The energy and complexity (BDM) of the images in Figure 1 and Figure 2.
Figure 17. The energy and complexity (BDM) of the images in Figure 1 and Figure 2.
Entropy 23 00153 g017
Entropy 23 00153 g018 550
Figure 18. The energy and complexity (BDM) of the images in Figure 11a.
Figure 18. The energy and complexity (BDM) of the images in Figure 11a.
Entropy 23 00153 g018
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Khalili, A.; Bouchachia, H. An Information Theory Approach to Aesthetic Assessment of Visual Patterns. Entropy 2021, 23, 153. https://doi.org/10.3390/e23020153

AMA Style

Khalili A, Bouchachia H. An Information Theory Approach to Aesthetic Assessment of Visual Patterns. Entropy. 2021; 23(2):153. https://doi.org/10.3390/e23020153

Chicago/Turabian Style

Khalili, Abdullah, and Hamid Bouchachia. 2021. "An Information Theory Approach to Aesthetic Assessment of Visual Patterns" Entropy 23, no. 2: 153. https://doi.org/10.3390/e23020153

APA Style

Khalili, A., & Bouchachia, H. (2021). An Information Theory Approach to Aesthetic Assessment of Visual Patterns. Entropy, 23(2), 153. https://doi.org/10.3390/e23020153

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop