So, we are trying now to develop a method to allow the Saundaryalahari system to be used online. After the performance in October, it was apparent that we needed to do quite some work on several aspects of the performance system in order to better reciprocate the processes involved.
Originally, we had planned to have S.3 in a new ensemble constellation and a few performances in Northern Italy in early 2021. Since we are still in uncertain circumstances and unable to travel, it makes sense to rehearse online - potentially also developing a system that can work over the internet.
The system is quite similar to the previous one, in that we can establish reactive audiovisual systems to perform with, but so far there is no avoiding a significant latency in processing on either end. I suppose the best and most sophisticated solution would be to develop a server to process externally from both computers - like a video game server - but the programming is extremely daunting - I won’t go there!
There is a latency in getting the signal from the performer in Italy, then the processes can be synced on my end, and the visual response will have an additional latency in getting there. Overall, it will be several hundred milliseconds at least, even with wired connections. This is perhaps an example where it is a helpful that the performer is a seasoned, highly-experienced professional. The performer adjusts to different playing latencies and adapts - much more effectively than any musical or technological system!
On the processing end, the signal I receive will be in sync for recording and archiving, at least within half of the latency intended by the performer. I’m using a routing system through DP that allows the Zoom call to be internally routed via Soundflower input and output to the visual system and through a monitoring system.
The next task, is to develop the reactivity of the visuals using the Structure coupled with the 3Trins - something we should have spent more time doing in rehearsal in Zurich in October 2020. Our rehearsal yesterday then came up with the following comments for developing reactivity in the system. We will meet weekly to develop the system with the following components to work on:
High and low frequency should be measured.
Timbre: the timbres of string instruments are quite variable and the frequency response of the body of the cello is irregular. Perhaps use a multi-band gate or spectrum map especially since as Saundaryalahari is quite melodic, there may not be some many timbre variations.
Perhaps we should measure noise.
Distortion (visual) could be a form of feedback.
Look around frequencies of 1000 - 3000hz; maybe put a gate on them to prevent decibel saturation.
Consider the centroid pitch and its pitch variability. Distortion can also change that variability -low pitches and high pitches move the centroid.
Develop the system to respond within a reasonable amplitude.
