junebug codes

Making immersive art installations with accessibility features seems like a given; if you're going to release something to the general public, why wouldn't you take into consideration different levels of ability or knowledge around art? It's surprising (or maybe not if you work in VR) that a lot of immersive media, including film, playable experiences, and physical installation doesn't always accommodate to this.

I was first introduced to the issue five years ago, when I attended my first academic conference: The International Community for Audio Display (ICAD). I was just an audio programming nerd working for my university as an Immersive Media Support Specialist, my first creative tech role. I learned a lot about the world of data sonification at the conference and its applications, but there was one particular thing that caught my full attention, audio-only playable experiences.

At the conference in Newcastle UK, a developer artist called Brandon (if I remember correctly) had a workshop where participants explored a variety of audio-based games and were mostly made, published, and played by blind people. And the way he talked about the hurdles and obstacles encountered by this community to just enjoy interactive media and storytelling motivated me to include this thinking in my work at North Carolina State University.

At the time I was a novice developer, and I hit challenges, seemingly insurmountable, building an accessible system for spatial audio in the browser. But after a few years of experience as a freelance developer, and I'm using funding from Arts Council England to build a screenreader accessible immersive web audio system, incorporating techniques and strategies I've learned from my recent education and my work experience.

As far as I know no one's tried this before. There accessibility in the html elements here https://discourse.threejs.org/t/how-did-they-add-html-elements-over-the-cylindrical-mesh/60437 when it comes to websites that contain 3D stuff baked into the site. This is for React-Three-Fiber, and it hybridizes the web app with scrollable elements. There's also a react three fiber A11y extension, but it forces to to write the accessibility as React components, which feels again like it places accessibility as this "extra layer" that developers have to choose to use.

While it's good this exists, I'm going for a deeper mode of interaction, closer to the original vision of WebVR.

In order to better understand this project, an important shift in thinking has to be made. We think of websites as many things, as infrastructure, as platforms, as public forums, as games, and as kind of media to interact. I believe public art installation could be one of these primary uses.

This mindset goes into every facet of development, from the tools to the works created. This is the main thing that a lot of developers will initially miss - they're used to building infrastructure so much that it wouldn't occur to them the creativity of the tools they use on a day-to-day.

Building on this foundation, I think there are more sociological implications to browser-based public art. There's two important concepts post-2020 that I'd like to highlight: hybridity and entanglement human-computer interaction (ECHI).

During the 2020 lockdown / pandemic era, there was a technological movement to move things online, and when lockdown ended, we lost a lot of that support, especially those who now had access to things they hadn't before, and thus the concept of hybridity emerged. How can create experiences accessible to both a in-person audience and an online audience simultaneously? How can these communities interact with each other in meaningful ways? One of the best examples I know of is the Audio Developer Conference in London, using live streaming, Gather, and other technologies like discord to both enhance the experience and broaden the audience to more people.

EHCI goes into academic territory. Conceptually, how are we connected (entangled) with our devices and other people, and how do our technologies (including communication methods) shape us? If you see a website as merely infrastructure, this still has implications, our infrastructures shape us dynamically. The way we interact with websites has turned us into screen-clickers and for some in the VR industry, headset-wearers.

So the question upon reflection is how can websites intended for public art installation be designed for hybridity and a harmonious relationship of human-machine?

Compiling this research and development into a kind of prototype, I'm interested specifically in how spatial audio can make the browser itself a more interesting and engaging form of public installation. Discounting the platform capitalism of the 2010's that brought us art through YouTube, Twitch, and other forms of social media, I think this kind of installation should push boundaries to move us from algorithmic feeds towards curated hyperlinks.