Return to processes: new tools for video synthesis

So, I’ve been actually quite busy this last month with different forms of online teaching and only occasionally had inspiration to continue with audio/visual experiments as part of the Saundaryalahari project. The creative spirit sometimes needs to collect itself after a time of innovation - also to harvest new ideas to take things to the next step. Mostly, I was just busy and a bit emotionally exhausted with all the tumult in the world bearing down…

Almost spontaneously, I’ve begun new processes with a new tool in my arsenal: a Gieskes 3TrinsRGB+1c video synthesizer. The 3Trins is a very creative approach on analogue video synthesis, and - perhaps on its surface - it might seem like another tool for creation of supportive visuals to accompany music. I was, however, mostly curious about this device because of its abilities to accept external control via audio and/or cv voltages and that it contained an audio output that resolves the video oscillators into sound - something that can be processed and/or recirculated back into the video generation, and thus offers potentially new perspectives on the reciprocal processes I am exploring:

The differences between the 3Trins and Pixivisor will largely be a more effective control toward the generation of visuals. The first experiments with the 3Trins largely have to do with getting accustomed to this kind of visual control - and it is i…

The differences between the 3Trins and Pixivisor will largely be a more effective control toward the generation of visuals. The first experiments with the 3Trins largely have to do with getting accustomed to this kind of visual control - and it is intreuging. On the first full day of experimentation, I was able to improvise this short film. The inputs were largely occupied by LFOs created by the Mutable Instruments Peaks - which, highly cycled enough, allow for the generation of triangle and square shapes from their waveforms. One additional oscillator was added to the audio output - otherwise it was entirely generated from a delayed output signal produced by the 3Trins.

Aside from aspects of sync and colour balance - also aspects that Pixivisor didn’t actively allow control, the inputs to each of the colour channels allow creative control by inputing VCOs. This is quite obvious, because VCOs can usually generate much higher frequencies to create layered visual effects on each channel. Furthermore, each channel can also be routed to one of the two internal LFOs and oscillate a little bit. You can create stacked oscillator scenarios.

One traditional way of creating video synthesis is to use a video camera and have the camera “feedback” the visual image created in real-time on a screen. This creates the familiar '“mirror-image” effect that we know from those carnival fun-houses, but it also is a significant method of adding to a reciprocal process. The image is fed back unto itself constantly, so that it generates visual patterns. By moving the camera a little in each direction, the oscillation mimics that of the width of a LFO/VCO fed into the colour channel. Actually, it is the most linear way of creating reciprocal processes - but the difference may be that modification of that input process - the “artistic” freedom element - is a bit trickier. I’ll keep exploring that though, and may use Pixivisor to supplement these processes on a different visual channel. The risk of that is probably that it will quickly become much too complicated. More to come…

Tuning experiments for understanding audio/visual synthesis 03.05.2020

The past few days have seen experiments that continue to inform my work in understanding how visuals can be created with audio synthesis. Ultimately, it will be useful to know what kind of sonic possibilities can alter the visuals and control them, but it is not my intention to merely control the visuals. By practicing difference setups and familiarising myself with pitches, intervals and rhythms that produce reliable images, I can create a palette that can be controlled in real time - hopefully that will sound good - but its not easy.

Pixivisor visual images generally produce a very intense fax-like sound that can be manipulated into interesting sound, but not easily. I’ve started these experiments by isolating various intervals with clean digital and analogue VCOs with only slight morphing. The top gif examples are from using a modular synth I’m using an oscilloscope to compare the waveform to the visual outcome. Essentially, it seems the complexity of the waveform (as opposed to a simple sine/sawtooth/triangle wave) creates contrast gradients, whereas the pitch determines horizontal frequency. Vertical frequency can be created with overtones and FM.

The more complex images (in the red and yellow gif below) are recreated using more complex digital synthesis algorithms from the OP-1 Digital engine, manipulating octaves (-21 detune cents made horizontal), synth perimeters and envelope. Watch the screen recording with sound, FFT and oscilloscope below.

ambient and beats electronic music experiments 25. - 29.04.2020

Recent days, I’ve been working with different palettes and ways of working with visuals. The experiments have not been so much about perfecting a reciprocal system, but to start from materials that sound good - almost pop music - and then seeing how much closer I can get to them after putting them through the visual system.

The first work I improvised on 20.04, and it turned out to be a pretty complete performance. The main part of the work involved two oscillators that were slightly modulated but could reliably stabilise the vertical bands. The percussion blurred the image enough to create sort of “pebbles” that would move from right to left. It was good fun.

The second work was an ambient work that I used to develop sense of palette. It was produced without using any visual reference information, but the steady tonality meant that I could experiment with horizontal consistency. I did manage to attempt to pitch shift the entire track with Melodyne to see if I could find a tonality that would stabilise the image completely, but I could not find it. Instead, I spent the time looking at different contrast settings and palettes that would fit the music. The last one I like the best - it looks more like sunrise over water in my opinion.

This evening's experiments 25.04.2020

These are this evenings experiments, routing audio directly through an aux bus via DP into Pixivisor. The pulsating sounds are created via a LPG from 2 oscillators. (one digital, one analogue) in the modular. Clouds is making the atmosphere.

Unity experiments 24.04.2020

I am spending this next few days working on experimentation with sending various gif files, hopefully those outputted from the more successful and pertinent pixivisor sessions, to Unity - eventually to be imported into the Teenage Engineering OP-Z for sequencing visuals. So far, it is a bit frustrating to match the various versions of Unity with the ones that will work properly with the OP-Z app software; it seems it will be easier to make compatible with the iOS versions of the app than the OSX versions. I have followed all the instructions to create the so-called sprite sheets for Synthpak’s GIF Looper template and even gotten so far as to have demonstrations work properly within Unity. Unfortunately, I’ve not managed to get them to work properly with the motion sequencer.

These experimentations, however, get me to thinking that perhaps I need to be thinking more about what I want to achieve with visuals - especially in regards to examining the reactive relationship between sound and image. I am sure that I don’t want to end up creating a visual system that merely responds to sound or plays “in synch” as this would be too literal of a relationship between sound and image.

For me, I am more interested to find a relationship between the reactive elements to both image and sound; something that even changes the meaning of these synethstetic relationships when they shift, interplay and “dance around” each other more than search for a literal, linear representation of both.

After the frustrating work last night struggling to get the Unity Video Pak to work correctly, I spent this afternoon (with what little time I have to compose and think at home) looking once again at the work of Alexander Zolotov aka Night Radio. He is the Russian electronic visual music artist that created Pixivisor, but he also has developed the Virtual ANS Synth and other visual music software I often use to demonstrate the Saundaryalahari concept and reciprocal audio visual synthesis ideas. Besides the coding work in Pixilang and Unity, his emphasis seems to be on the linear representation of sound. Phonopaper and Virtual ANS are excellent synths and work very well with visualising music and vice-versa, but they translate almost always literally, and in a straight line or something like that.

I have two main issues with this approach. The first is that it is often a static approach to visualise sound/image. I feel like the way we perceive time and image is not static - sometimes we pay attention to something and sometimes we lose focus or space out, but time almost always does not remain constant in our consciousness. If we are tripped-up on a sound or an interesting image, it may be milliseconds or seconds before we realise we are hanging on to the feeling of that perception, so that our stream of consciousness is uneven, like rafting down a river and getting stuck momentarily on the rocks. I think that this must be somehow accepted and measured in a sound/image system and the best way I can think of to do that is to make sure that the system is reciprocal, so that it is constantly producing movement in all directions. Even stasis is represented in this way. That’s interesting to me. The other issue I have with linear representations in an sound/image translation system is that it mostly always results in something that sounds similar, and - even if it is an advantage sometimes - is fairly easy to recreate, because the system that is tracking the incoming signal responds predictably. In nature, things are rarely always received the same (even if science wants us to believe things like the elements are predictable) and every input signal is based on a movement of an almost infinite amount of variables created by the multiplication of nature. Thats not a disadvantage to the creative system - it is actually an advantage - something that can be observed like a tool for how things always FEEL different even if they are the same. That’s important to me in a creative process.

I’ll have to have more of a think about the reciprocal system of the Saundaryalahari, though, and see what other technical limitations will come up as I continue to work with these systems. I had imagined I would work with an algorithmic aka Unity based sequence system as well as an analogue visualisation system like Pixivisor - maybe even overlay them together - but now that seems sort of silly. We will see what comes next…

The beginning of An explanation on how audio and visual creativity realms can become reciprocal

I have often been asked to explain what I am seeking when using this system and how it works. The reasons seem pretty clear to me what I am seeking, but of course, as it is research, I am not anticipating knowing now what the exact nature of the outcome will be. In any case, I will try explain a few thoughts about this system here and as needed, will continue to explain in subsequent blog posts.

Points:

  • Audio/visual synthesis using Pixivisor is part of the Saundaryalahari process, firstly, through its use of non-verbal/non-textural based interaction.

  • I am seeking a system in which I can get as close as possible to linking the auditory realms and the visual realms together in the creative process.

  • Qualitatively, I do not want to differentiate between theoretical and actual sound and image. In other words, I want to include distortion, phase, dissonance and accident in the process.

  • The fact that an audio file can be stored as an animated image file (gif.) and vice-versa, while simultaneously counting both formats (playable within software systems such as pixivisor) is poignant to this project.

  • It is not simply my purpose to “process” natural images (for example, by manipulating sound between two visual morphs of pixivisor, but to seek outcomes within the geometrical and visual structural elements of live pictures/images while creating unique sound/music. There is no difference in the processes in my opinion, and they can occur simultaneously and inform each other.

  • Human interaction with sound and image can occur simultaneously - like above - but moving image in particular can inform different aspects of performance in realtime than audio does to the ear. With an intuitive reciprocal system in place, creative improvisation has unique new ways to interact with itself.

  • Analogue, real-time processes are more important than algorithmic non-real-time systems and are better at formulating creative diversity. This is subjective.

More next time…

Saundaryalahari project in active phases

22.04.2020

Work on understanding how audio and visual systems can work together continues during the lockdown. Currently, I have set up many different iterations of the system to examine how to control manipulation of audio to create moving images in Pixivisor.

These are examples of gifs that are recorded entirely in Pixivisor, from electronic manipulation of sounds. The devices used were:

Modular synths VCOs to create amplitude, which mostly created vertical and horizontal movement. At the default frame-rate (64x64 pixels) of 10.77, stable images were reproduced at approximately the based pitch of F at about -24 between semitones (as recorded on an OP-1).

A Monomachine was able to generate a sequence that would mimic the frame rate and also produced a stable image when the outputs of the modular synth were run through a thru instrument machine at approx. 80.7bpm using 16th notes at a fairly steep envelope.

Notes: adding harmonies seemed to produce the most distortion. Octaves did not make must difference to the stability of the image, but 5ths and other intervals tended to distort the image on a second layer, often creating chaos.

The below are an example of the later experiments.


Cart (0)