“Capturing Pixels: The Art and Science of Algorithmic Photography in ACM’s Lens”

Employing the phrase “algorithmic photography” may elicit perplexity from a number of individuals, yet British digital creator Alex May is a notable exception. It seems that he has both coined the term and developed the technique.

This should not be conflated with AI-generated visuals (though there is a relationship) or what may initially seem like time-lapse photography. The algorithmic photography that May engages in skilfully captures a still image from digital video footage he records.

While this may sound akin to any conventional YouTube screen capture, it is decidedly different.

In an algorithmic still image, specific movement is encapsulated as a single moment in time, artistically expressed through vibrant and impressionistic swirls set against a still background, which could feature edifices and thoroughfares in an urban landscape.

The creativity stems from the implementation of algorithms that May devises to seize colors as they evolve during what typically spans a five-minute recording, while also filtering time and movement. Often, May employs algorithms that could effectively benefit computer vision systems; however, instead of constructing a vision machine, he is imparting a fresh perspective to a relatively timeworn artistic practice: photography.

“For several years, I’ve been using the phrase ‘algorithmic photography’ to articulate this method I devised, which is a digital procedure yet retains many ties to photography and its historical context,” remarks May, who resides in Brighton on the southern coast of the U.K. “It represents a balance between how we perceive the world through our individual lens and through technological means.”

May has long navigated the nexus of art and technology, beginning to write computer code in the early 1980s using assembly language while maintaining a genuine passion for photography. He boasts an impressive portfolio as a digital creator, having been a video jockey in the international club scene during the early 2000s, and later sculpting video pieces showcased in renowned art and science institutions including London’s Tate Modern museum and The Francis Crick Institute.

Among his influences is Eadweard Muybridge, the British photographer who, during the 1870s, pioneered techniques for higher shutter speeds, which debunked the widespread belief (as illustrated in that era) that a galloping horse’s legs extended outward; rather, Muybridge demonstrated through a series of still images that curled inward when translated into motion.

“It dawned on me at a young age that this technology enables us to perceive beyond what our remarkable physical capabilities allow us to observe in our quest for understanding and how things function,” he reflected.

By leveraging algorithms, May is now advancing still photography to encapsulate motion, elevating his idol Muybridge to a new plateau. “For me, photographs have invariably seemed quite foreign, as they capture an instant in the world, while we don’t experience the world in that way since we are constantly in motion,” May expressed. “Capturing motion in a photograph resonates with how we actually experience our surroundings.”

Fittingly for someone whose creations bridge art and technology, May’s themes occasionally revolve around the scientific realm, illustrating not the fundamental mechanics of science itself, but rather what he perceives as the artistry in the work of scientists.

For instance, May holds a current residency at Vienna’s University of Natural Resources and Life Sciences, commonly referred to as BOKU, where he algorithmically captures images of scientists conducting research with yeast for an immunology project. In one such image, shown below, he incorporates colors and elongation to depict Dr. Özge Ata of BOKU University to express her dynamism against the static backdrop of laboratory furnishings.

“This yields an impressionistic portrayal of the science in progress,” May noted. “These are actual scientists, genuinely engaging in scientific endeavors, and I’m employing the algorithmic photography approach to capture that in a creative manner, illustrating the fluidity of their actions, suggesting a dance performance.”

May is equally adept at photographing urban scenes or landscapes. In a whimsical depiction of the Philadelphia Museum of Modern Art, showcased below, he utilizes swirls of color to highlight the numerous runners who, within a five-minute timeframe, have ascended the steps in a manner reminiscent of Rocky from the film of the same name.

One of his cherished works showcases the iconic Brighton Pier in his home city (below), which contrasts the joy of tourists against the alarming presence of birds notorious for swooping down to snatch food from the hands of unsuspecting guests.

“That

“The visual for me encapsulates the strain between the birds in the realm of the skies and the humans in the realm of the earth,” expressed May. “There is genuinely tension in the atmosphere, with individuals enjoying themselves while birds spiral around searching for an opportunity to snatch a snack. It’s something distinctly Brighton and appealing to tourists.” May recorded the footage using a compact GoPro Hero 6 — characteristic of the portable video camera he typically utilizes — before applying algorithms for the

Considering the “photography” aspect of algorithmic photography, some enduring challenges persist, such as obtaining favorable camera perspectives. As a passionate urban photographer, May asserts that acquiring a view from above (in other words, ascending to a certain elevation) can be crucial. “However, in some urban areas, it’s quite challenging to rise above street level unless you’re actually entering a building, in which case you need to have a valid reason to be there.”

The elevated High Line park in lower Manhattan provided valuable assistance as May captured the flow of iconic yellow taxis from that perspective, depicted below.

“It was great to achieve some verticality,” he remarked.

With numerous algorithms at play, one could assume that artificial intelligence forms part of May’s toolkit. Yet, up to now, it has only served an experimental purpose. May is collaborating with AI to assist in distinguishing individuals and objects, which he then creatively interprets. One challenge has been that AI, as research has indicated, does not consistently identify people reliably and can overlook individuals due to racial and other biases embedded in datasets.

“There is substantial effort in this domain focusing on what is termed fairness, accountability, and transparency,” observed Sanmay Das, a professor of computer science at George Mason University in Fairfax, VA. Das highlights the Gender Shades study associated with the Media Lab at the Massachusetts Institute of Technology, revealing that, in the study’s words, “commercial AI systems significantly misidentify the gender of women and individuals with darker skin tones.”

Das noted, “Much of this stems from biases in the underlying datasets they rely upon.”

May is acutely aware of these concerns as he begins to work with AI, a technology he describes as a “collaborator” due to the inclusion of datasets curated by humans.

Until now, pre-AI, “I’ve maintained control because I created the code myself,” May expressed. “I’m now transitioning into AI where I’m incorporating trained datasets that I didn’t develop. Thus, I’m inviting a collaborator into the process, which is a first for me. If there are inherent biases, they will manifest. I have no oversight over that.”

AI is a consideration that any practitioner in the emerging realm of algorithmic photography will need to reflect upon. Thus far, relatively few artists appear to be actively pursuing algorithmic photography—at least under that designation. British technology blogger Peter Krantz has documented his project in this sphere. Paris-based multimedia artist Sabina Covarrubias also engages in work she labels as algorithmic photography.

May will have the opportunity to explore bias and other facets of AI in his upcoming project: the AI Hokusai Arttech Research Project, an online initiative wherein invited artists such as May will utilize AI to generate artworks. The project is named after Japanese artist Katsushika Hokusai, renowned for his 19th-century innovations in printmaking.

Regardless of what materializes—or does not—from AI, May will persist in practicing the foundational principle of his art.

“The image can solely be crafted algorithmically,” he states. “Photography, as we comprehend it, can only gather light. It’s an accumulation of light caught by the sensor. Conversely, the algorithmic photo extracts specific data originating from the image.”

Could it be that the individual with the most data in the end not only prevails but is also the most innovative?

Mark Halper is a freelance journalist based near Bristol, England. He covers a broad spectrum from media magnates to subatomic particles.