Monday, October 27, 2008

Freeze a sound with Max/MSP/Jitter

Version FrancaiseIn April 2008, I was invited by composers Eric Chasalow and Maxwell Dulaney to give a 2-day seminar on spectral sound processing techniques at Brandeis University Music Department.

A topic the music students particularly enjoyed was the frozen sound, the audio equivalent of the cinematic "freeze frame shot". I taught the nuts and bold of the real-time stochastic spectral freeze technique (the stochastic component is aimed at breaking the ice - with the audience).

On this video, discover 5 variations on a Max/MSP/Jitter freeze tool:

Free downloads

frozen beauty

Syllabus

The program for just two 3-hour workshops was quite ambitious!

April 14th

  • Overview of the topic: “Spectral processing, or having fun with graphics and sound.”
    All students and faculty members are welcome (not only to the computer music specialists).
  • Spectral explorations 1
    Sonogram. Fast Fourier Transform (FFT). Recording spectral data into Jitter matrices. Frame to frame playback.
  • Introduction to Jitter
    Matrices, filters, expressions.
  • Spectral explorations 2
    Phase vocoders: Max/MSP version versus Max/MSP/Jitter version. Variable-speed playback (time stretching, time compression).

Waiting for next session

Assignment! Play with pictures, spectrograms, vocoders, [pfft~].

April 28th

  • Spectral explorations 3
    Real-time spectral processing with Max/MSP. Denoiser, amplitude compression, frequency shifting, pitch shifting.
  • Spectral explorations 4
    Back to our Max/MSP/Jitter phase vocoder. Blurry playback.
  • Spectral explorations 5
    Real-time spectral processing with Max/MSP and Jitter. Real-time Freeze.

Rose Picture "Frozen Beauty" CC License by freebird4

12 comments:

  1. That sounds awesome! I'll check that out. I'm putting on a concert at Longy on November 19, hope you can come!!

    ReplyDelete
  2. Thanks. Good to know about your concert! I'll stay tuned.

    ReplyDelete
  3. Jean-Francois, I am not yet an Max/Msp/Jitter owner, but intend to be very soon. Before I make the purchase, is it necessary to purchase the bundle that includes jitter as I am not doing any "video work". I noticed some of your patches include visuals and would like to use them. What would you recommend? A hunch tells me I should buy the full max/msp/jitter bundle... Thanks

    ReplyDelete
  4. Hello Christopher. Yes, as you guessed, most of the patches (Phase Vocoder and Freeze variations) require Jitter objects. Even the patches that don't include visuals use Jitter for the actual recording and playback of the sonogram data. If you are interested in spectral sound processing, you will surely benefit from matrix processing, thus from Jitter. All the best. Thanks for your question.

    ReplyDelete
  5. Wonderful. I'll be stopping back again to learn more. Much appreciated.

    ReplyDelete
  6. Jean,
    Congratulations for your exelent work! I was thinkig of a "freeze patch" for video. Do you think its possible to apply fft to video and obtain the same results?

    regards,
    Bruno

    ReplyDelete
  7. Thanks Bruno,
    You don't need FFT to freeze or time stretch video. If you use Jitter & Max, go through the Jitter tutorials, from the first ones. You won't need to go through all of them, just a couple, and you'll surely have ideas on how to freeze videos!

    ReplyDelete
  8. Jean,
    Thank you very much. I'll follow your directions. I hope you dont mind I have used your max msp externals to build a max for live patch: http://www.maxforlive.com/library/device.php?id=719
    (I'm looking fodward to create a video and audio freeze at the same time, jitter+your externals)

    Thanks for sharing your work!!

    ReplyDelete
  9. hello jean,

    did you thought about porting your fft-patches to the ftm-libary?
    iam about to acomplish the blurring effect in a non signaldriven way in ftm.

    i was wondering if it would be possible to do it by creating a matrix (eg. 1024 rows
    and 5 cols big) wich gets filled randomly with 0 and 1´s each 1024 samples – with only
    one 1 in each row. this matrix gets multiplied (each 1024 samples) by the frames to be blurred
    (2 matrices wich holds the magnitudes and the phases of the appropriate frames). the final step would then be adding the results of each row of the matrix, to get a propper fft-frame.

    this is not working for me now. i still hear the frameeffect, but it seems to mix the successive frames somehow… anyway.
    jean, do you think that this would work (at all)? any hints?

    thanks, joh

    ReplyDelete
    Replies
    1. Hi Joh,
      Thanks for your comments!
      I'm sure many of my "jitter-based" patches are portable to FTM. But I have not yet felt the need to use FTM.
      For FTM questions, I'm sure there is a forum or list discussion where people can help.

      If you only have the "first" batch of patches, feel free to download the "new" patches (from Fall 2011) : http://cycling74.com/toolbox/live-spectral-processing-patches-for-expo-74-nyc-2011/
      You might find a little more clarity in some of the patches.
      All best,

      Delete
  10. HI, I am deligthted with your freezes. Altough I find it takes at least 30 ms to pick up the sample. ANy recommendation about I/O vector size and signal vector Size? to make it faster?
    I am sampling an acoustic guitar. He is activating your patcher through a pedal via oxygen into the laptop via usb. I do not know whether the 30 ms delay is produced by the pedal smapling order or by the actual patcher. I hace tryed by hand direct on the patcher and it is also a 30 ms delay.
    I will really appreciate help.
    Your patcher is fantastic

    ReplyDelete
  11. Dear Adina Izarra, thank you for your comment, and for your question.
    When you use Fast Fourier Transform to switch between time domain and spectral domain, you get an intrinsic delay: for instance, a FFT size of 4096, means that you need 4096 samples to convert to spectral domain (when you enter the [pfft~] object in Max). With a sampling rate of 44100 Hz, it means 4096/44100 s = 92 ms.
    To shorten this delay linked to using the FFT, you can only use a higher sampling rate, or use a smaller FFT size. Both solutions have drawbacks, but you can experiment until you get a compromise working ok with your artistic goals.
    Since you cannot shorten this delay, it might be useless to use short I/O vector size or Signal vector size.
    Actually, I already found interesting to - use - the delay to my advantage. For instance, when you freeze a guitar sound, you might want to freeze it, but not from the attack, because of the high frequency content of the attack. Rather, you might find it interesting that the "freeze" starts with a short delay, thus producing a smoother sound. But of course, it depends on your goals.
    Another thing: yes, there might be some delay in your setup because of the midi transmission between the laptop and the computer, but I suspect the main delay you get comes from the FFT process itself.
    Hope that helps,
    All best,
    Jean-François.

    ReplyDelete