The search for the formula of beauty pervades the cultural history of man. Throughout the centuries, philosophers, artists and scientists have considered units of measurement, proportions and compositional rules to describe beauty – in vain. Whether for architecture, paintings, music or poetry: despite great efforts, no satisfactory result has yet been produced as a broadly applicable, timeless formula of beauty.

This enduring challenge stems from beauty's inherently subjective nature. Aesthetic judgments remain deeply personal and culturally bound, shifting with trends and epochs in ways that seem to resist systematic analysis. However, we are living in a time that allows us to reinvestigate the measurement of beauty from a new perspective: artificial intelligence.

Indeed, measurements of beauty, desirability and, more generally, relevancy operate ubiquitously around us. Google’s image search effectively ranks the relevancy and desirability of images – as the probability that a user will select them. Algorithms test the ‘hit’ potential of pop songs to determine whether they will be released. Online advertisements are continuously served based on predictions of how well they will match a user’s preferences. These developments suggest that computational approaches to aesthetic evaluation are not only possible but already integral to our digital landscape.

In architecture, establishing a formula for beauty has an increased relevance today, as the advent of digital fabrication broadens the scope of forms that can be produced at no extra cost. Ornament and articulations are no longer the “wasted manpower” that Adolf Loos critiqued in his essay “Ornament and Crime” (Loos, 1912). Instead, they become expressions of purely aesthetic judgement. The more we know about what lies at the heart of these aesthetic consequences, the stronger the impact on the shaping of our environment through architecture.

In collaboration with TNG Technology Consulting GmbH, we initiated a project exploring whether machine learning can transform computers into arbiters of beauty for algorithmically generated architectural forms. This research unfolds in two phases: first, training algorithms to evaluate aesthetic quality in computational designs; second, enabling these systems to generate original forms based on their acquired aesthetic knowledge.

The history of the search for a formula for beauty spans not only many centuries but also many cultures. An exhaustive survey is beyond the scope of this essay, yet specific milestones show the evolution of this quest.

In ancient Rome, architects such as Vitruvius published manifestos with ideal sets of compositional proportions, defining the interrelation of parts with one another and with the whole. Renaissance Italy embraced this theory through Leon Battista Alberti, who complemented it with harmonic ratios from music: “We shall, therefore, borrow all our rules for finishing our proportions from the musicians, who are the greatest masters of this sort of numbers, and from those things wherein nature shows herself most excellent and complete” (Alberti 1965, 196).

The emphasis on mathematical composition proved equally strong in art. Indeed, Leonardo da Vinci wrote in his notes: “Let no one read me who is not a mathematician” (Da Vinci). Far more elaborate geometric compositional principles can be found in Timurid and Safavid treatises such as the Topkapi Scroll. Indeed, Islamic architecture of the period was oftentimes constructed not only to follow set proportions, but according to rules that specified sequences of operations. The resulting buildings featured elaborate details at multiple scales; beauty was intrinsically linked to architectural complexity.

The knowledge that order and complexity influence people’s appreciation of beauty began to be systematically studied in the late 19th century, most notably by Gustav Fechner and by George David Birkhoff (Cucker 2013, 116-120). In 1970, Daniel Berlyne presented his influential framework that posited that people are naturally attracted to the novel and the complex (Berlyne 1968, 1970). Complexity was defined through aspects such as pattern regularity, number of elements, their heterogeneity, and the irregularity of the forms. Extensive research has tested Berlyne's hypotheses, whose validity appears to depend not only on the objects being evaluated but even more on the chosen definition of complexity. Even today, the question of whether beauty can be measured numerically and expressed in formulas is a matter of ongoing debate.

In 2015, Microsoft launched a how-old.net, a web page where users could upload photos of their face and receive an estimate of their age. The estimated ages were so accurate that the site received some 50 million visitors in a single week. Concurrently, researchers from Yahoo Labs proposed an application for measuring the beauty of faces in portraits (Redi et al., 2015). More recently, in 2016, Beauty.AI launched an international beauty contest in which contestants were judged by machines.

These approaches share a common foundation in supervised machine learning. In machine learning, rules or theories are not specified a priori. Rather, statistical techniques enable machines to 'learn' optimal prediction methods based on various input data. In supervised learning approaches, computers receive inputs and ideal output values from thousands or millions of training objects. Inputs take the form of vectors describing any number of object attributes. The computer learns to map portions of the input vector to the output values. Once learning is complete, the effectiveness of the learned function can be tested with separate validation objects distinct from the training set.

Our project pursues this supervised machine learning approach. The computer receives training data consisting of 1000 abstract forms, each with a pre-assessed ranking of its beauty (described below). A convolutional neural network then attempts to identify features in the abstract forms that correlate with their beauty ratings.

The experiment was designed together with TNG Technology GmbH, a Munich-based consultancy and software developer with expertise in artificial intelligence. All machine learning work was implemented by TNG Technology Consulting, under the leadership and expertise of Mathias Burger and Dr Andreas Hille. The results were presented at the Big Techday 11 conference in Munich (Big TechDay, 2018).

Objects to be Assessed

As Wolfgang Klein of Max Planck Institute points out, a promising technique may be to “begin small and not try to immediately explain why Goethe’s Marienbad Elegy, Beethoven’s piano sonata Appassionata or Rembrandt’s Night Watch are considered beautiful by scores of people, but not by all” (Klein 2018).

Following this approach, our object selection was guided by several criteria:

• • Keep objects entirely free of function; only the form would be evaluated
• • Maintain simplicity, while creating objects that allow for a high degree of differentiation
• • Ensure purely geometrical objects without intentional figurative connotations
• • Ensure comparability of objects
• • Use algorithmically generated objects; this will subsequently allow a synthesis of beauty in case the measuring experiment is successful.

Based on these criteria, we employed a modified Catmull-Clark subdivision process (Catmull and Clark, 1978) to generate the objects. The subdivision process was modified to create non-figurative objects with a high degree of differentiation and articulation. Weights were introduced to control vertex positions as interpolations of their parent vertices. Additional weights enabled extrusion of vertices along face, edge, and vertex normals. All generated forms are cubes that have been subdivided over 8 iterations, and consist of approximately 400,000 faces. In total,1000 forms were generated and rendered using this process (See image gallery above).

Defining Beauty

Rather than formulating a precise definition of beauty, in this approach we equated beauty with appeal. To measure appeal, we selected a fast and simple paired assessment: two forms are presented on screen side by side, and viewers select which form appears more appealing. We built a small website accessible to volunteers within the TNG Technology Consulting group. In total, 17,000 binary comparisons were made.

This setup assumes that the assessment of beauty is a conscious decision. In a subsequent project, it would be conceivable to measure sub-conscious factors in parallel. For instance, one could measure the time a person requires to rate an object, their distance from the screen, or even eye movement patterns.

Convolutional Neural Network Setup

After each object was rated according to its appeal, three vectors for each object were produced to serve as inputs for the learning algorithm:

• 1. A rendered image of the form, equivalent to the one shown to the volunteers to rate, down-sampled to 64 x 64 pixels, 8-bit greyscale.
• 2. 3D measurements of the form — expressed as rendered maps – of curvature, normal orientation, area, edge length, etc – down-sampled to 64 x 64 pixels, as shown in Figure 1.
• 3. Generative parameters consisting of the specific weights that were used in the subdivision process to generate each form — total of 48 8-bit numbers.

We trained a convolutional neural network using Google's TensorFlow framework . Based on the 64 x 64-pixel image data, we set a 3 x 3 kernel size. Corresponding to the relatively small dataset and image size, we crafted small dense and convolutional neural networks that were regularised so that they don’t run into overfitting problems. To ease training and avoid issues like the “dying ReLU” problem, we used the ELU [1] activation function.

Different input vectors and neural network setups were iteratively tested. The best results were achieved using the rendered images of the form — the same images that were shown to the human evaluators, yet down-sampled to 64 x 64 pixels.

Results and Validation

Using this training set, the computer eventually achieved 71% accuracy in selecting the more appealing of two forms. We wrote a program that could take a newly generated form and use the evaluation algorithm to compare it against every form in the library of assessed forms, thereby creating a percentile ranking for the new form.

This enables the computer to perform iterative optimizations of generated forms. Using a tabu-search algorithm (Glover 1986), the computer can successively adjust its generative subdivision weights to maximise beauty of the form. Three forms showing local maxima derived with tabu-search are shown in Figure 2.

For years it has been possible to use algorithms to generate forms. Using a convolutional neural network, the computer can also evaluate the forms it produces based on previous assessments of beauty.

It is crucial to note that this measurement is far from universal; it functions only for forms produced using the exact algorithm with which the neural network was trained. Even when simply shifting subdivision weighting values outside trained bounds, evaluation quality diminishes rapidly.

While the computer has found a way to measure beauty, we remain to understand how this measurement is performed. Indeed, many machine learning algorithms function as a “black box”: we can provide this box with an input and view its output, but we are unable to see what happens inside. Only recently have explanation methods been advanced that at a minimum would show which sections of an image most contributed to a decision.

Beauty will likely remain a mystery beyond our complete understanding. et if we can finally measure it, then synthesizing it in architecture should be entirely feasible. The computer can evolve from a tool of efficiency into true partner in design that helps architects achieve higher compositional goals. In the end, will the computer generate its own taste? Our built environment stands to gain.

• Alberti, Leon Batista. 1991. The Ten Books on Architecture, ed. Joseph Rykwert, MIT University Press
• Berlyne, D. E. 1970. “Novelty, Complexity, and Hedonic value”, Perception & Psychophysics, 8: 279-286.
• Berlyne, D. E., J. C. Ogilvie, & L. C. C. Parham. 1968. “The Dimensionality of Visual Complexity, Interestingness, and Pleasingness”. Canadian Journal of Psychology, 22: 376-387.
• Big Techday conference, Munich, June 2018, https://www.bigtechday.com/rueckblick/big-techday/11
• Catmull, E., and J. Clark. 1978. “Recursively generated B-spline surfaces on arbitrary topological meshes”. Computer-Aided Design, Issue 10.
• Cucker, Felix. 2013. Manifold Mirrors: The Crossing Paths of Arts and Mathematics. Cambridge: Cambridge University Press.
• Da Vinci, Leonardo. Notebooks of Leonardo da Vinci, Volume 1, Prolegomena and General Introduction to the Book on Painting.
• Glover, Fred. 1986. “Future Paths for Integer Programming and Links to Artificial Intelligence”. Computers and Operations Research 13 (5): 533–549.
• Hansmeyer, Michael. 2009. “From Mesh to Ornament. Subdivision as a Generative System”. eCAADe 2010 Conference, Future Cities: 285-293.
• Klein, Wolfgang, et al. 2008. Beauty Beyond Measure, Max Planck Institute
• Loos, Adolf. 1912. “Ornement et Crime”, Les Cahiers d’aujourd’hui
• Redi, Miriam, Nikhil Rasiwasia, Gaurav Aggarwal, and Alejandro Jaimes. 2015. “The Beauty of Capturing Faces: Rating the Quality of Digital Portraits”. FG 2015, IEEE International Conference on Automatic Face and Gesture Recognition May 4-8, Lubljana, Slovenia.