CS1315: Introduction to Media Computation

Chapter 8: Making Sounds by Combining Pieces We know that natural sounds are often the combination of multiple sounds. Adding waves in physics or math is hard. In computer science, its easy! Simply add the samples at the same index in the two waves: r srcSample in range(0, getLength(source)): destValue=getSampleValueAt(dest, srcSample)

srcValue=getSampleValueAt(source, srcSample) setSampleValueAt(source, srcSample, srcValue+destValue) a The first two are sine waves generated in Excel. The third is just the sum of the first two columns. a + b = c b

Uses for adding sounds We can mix sounds We even know how to change the volumes of the two sounds, even over time (e.g., fading in or fading out) We can create echoes We can add sine (or other) waves together to create kinds of instruments/sounds that do not physically exist, but which sound interesting and complex def addSoundInto(sound1, sound2):

for sampleNmr in range(0, getLength(sound1)): sample1 = getSampleValueAt(sound1, sampleNmr) sample2 = getSampleValueAt(sound2, sampleNmr) setSampleValueAt(sound2, sampleNmr, sample1 + sample2) Notice that this adds sound1 and sound2 by adding sound1 into sound2 >>> c4=makeSound(getMediaPath("bassoonc4.wav")) >>> e4=makeSound(getMediaPath("bassoone4.wav")) >>> g4=makeSound(getMediaPath("bassoong4.wav")) >>> addSoundInto(e4,c4) >>> play(c4)

>>> addSoundInto(g4,c4) >>> play(c4) def makeChord(sound1, sound2, sound3): for index in range(0, getLength(sound1)): s1Sample = getSampleValueAt(sound1, index) setSampleValueAt(sound1, index, s1Sample ) if index > 1000: s2Sample = getSampleValueAt(sound2, index - 1000) setSampleValueAt(sound1, index, s1Sample + s2Sample) if index > 2000: s3Sample = getSampleValueAt(sound3, index - 2000) setSampleValueAt(sound1, index, s1Sample + s2Sample + s3Sample)

Note that in -Add in sound2 after 1000 samples this version -Add in sound3 after 2000 samples were adding into sound1! def echo(sndFile, delay): s1 = makeSound(sndFile) s2 = makeSound(sndFile) for index in range(delay, getLength(s1)): echo = 0.6*getSampleValueAt(s2, index-delay) combo = getSampleValueAt(s1, index) + echo setSampleValueAt(s1, index, combo) play(s1)

return s1 This creates a delayed echo sound, multiplies it by 0.6 to make it fainter and then adds it into the original sound. Clicker: What is sndfile in the echo function? 1. Path to a sound file. 2. A sound that were going to make echoes from. 3. A base filename (like aah.wav) that were going to use with getMediaPath()

Top row is the samples of our sound. Were adding it to us, but delayed a few samples down, and multiplied to make it softer. Clicker: Could you go past the end of the sound? If youre adding two sounds together, one offset by a delay, couldnt you go past the end? 1.Absolutely you only want to do this with short sounds. 2.No, were only going to the end of the sound with the FOR loop.

3.Yes, so we make the target sound extra big to make space. def echoes(sndFile, delay, num): s1 = makeSound(sndFile) ends1 = getLength(s1) ends2 = ends1 + (delay * num) # convert samples secs = int(ends2/getSamplingRate(s1)) # to secs s2 = makeEmptySound(secs + 1) amp = 1.0 # make amplitude smaller for each echo for echoCount in range(0, num): amp = amp * 0.6

# amplitude 60% smaller each time for posns1 in range(0, ends1): posns2 = posns1 + (delay*echoCount) val1 = getSampleValueAt(s1, posns1)*amp val2 = getSampleValueAt(s2, posns2) setSampleValueAt(s2, posms2, val1 + val2) How sampling keyboards work They have a huge memory with recordings of lots of different instruments played at different notes When you press a key on the keyboard, the recording closest to the note you

just pressed is selected, and then the recording is shifted to exactly the note you requested. The shifting is a generalization of the half/double functions we saw earlier. Why +1 here? def double(source): len = getLength(source) / 2 + 1 target = makeEmptySound(len) targetIndex = 0 for sourceIndex in range(0, getLength( source), 2):

value = getSampleValueAt( source, sourceIndex) setSampleValueAt( target, targetIndex, value) targetIndex = targetIndex + 1 play(target) Heres the piece that does the doubling This is how a

sampling def half(source): synthesizer target = works! makeEmptySound(getLength(source) * 2) sourceIndex = 0 for targetIndex in range(0, getLength( target)): value = getSampleValueAt( source,Heres int(sourceIndex)) the setSampleValueAt( target, piece targetIndex, value)

that sourceIndex = sourceIndex + 0.5 does the Can we generalize shifting a sound into other frequencies? This way does NOT work: def shift(source, factor): target = makeEmptySound(getLength(source)) sourceIndex = 0 for targetIndex in range(0, getLength( target)): value = getSampleValueAt( source, int(sourceIndex)) setSampleValueAt( target, targetIndex, value)

sourceIndex = sourceIndex + factor play(target) work for shifting down, but not shifting up. Why? ello=pickAFile() rint hello s/guzdial/mediasources/hello.wav owerhello=shift(hello,0.75) igherhello=shift(hello,1.5) n't able to do what you wanted. rror java.lang.ArrayIndexOutOfBoundsException has o e check line 7 of /Users/guzdial/shift-broken.py def shift(source, factor): target = makeEmptySound(getLength(source))

sourceIndex = 0 for targetIndex in range(0, getLength( target)): value = getSampleValueAt( source, int(sourceIndex)) setSampleValueAt( target, targetIndex, value) sourceIndex = sourceIndex + factor if sourceIndex > getLength(source): sourceIndex = 0 play(target) For a desired frequency f we want a sampling interval like this:

How the original sound synthesizers worked What if we added pure sine waves? We can generate a sound that is just a single tone (see the book) We can then add them together (perhaps manipulating their volume) to create sounds that dont exist in nature Dont have to use just sine waves Waves that are square or triangular (seriously!) can be heard and have interesting dynamics We can add together waves of lots of types to create unique sounds that cant be created by physical instruments We call this additive synthesis

Additive synthesis as-is isnt used much anymore Sampling as an Algorithm Think about the similarities between: Halving the sounds frequency and scaling a picture larger. Doubling the sounds frequency and scaling a picture smaller. def half(filename): def copyBarbsFaceLarger(): source = makeSound(filename) # Set up the source and target pictures target = makeSound(filename) barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) sourceIndex = 1

canvasf = getMediaPath("7inX95in.jpg") for targetIndex in range(1, getLength( canvas = makePicture(canvasf) target)+1): # Now, do the actual copying setSampleValueAt( target, sourceX = 45 targetIndex, for targetX in range(100,100+((200-45)*2)): getSampleValueAt( source,sourceY = 25 int(sourceIndex))) for targetY in range(100,100+((200-25)*2)): sourceIndex = sourceIndex + 0.5color = getColor( getPixel(barb,int(sourceX),int(sourceY) play(target) setColor(getPixel(canvas,targetX,targetY)

return target sourceY = sourceY + 0.5 sourceX = sourceX + 0.5 show(barb) show(canvas) return canvas Our programs (functions) implement algorithms Algorithms are descriptions of behavior for solving a problem. A program (our Python functions) is an executable interpretations of algorithms.

The same algorithm can be implemented in many different languages. The same algorithm can be applied to many different data sets with similar results. Both of these functions implement a sampling algorithm Both of them do very similar things: Get an index to a source Get an index to a target For all the elements that we want to process:

Copy an element from the source at the integer value of the source index This is a to the target at the target index description Increment the source index by 1/2 of the Return the target when completed algorithm. Adding sine waves to make something completely new We saw earlier that complex sounds (like the sound of

your voice or a trumpet) can be seen as being a sum of sine waves. We can create complex sounds by summing sine waves. These are sounds made by mathematics, by invention, not based on anything in nature. Basic idea: Build a sine wave If we want a 440 Hz sound wave, then we need one

of these cycles every 1/440th of a second. We need to break this wave into the number of pieces in our sampling rate. Our Code def sineWave(freq ,amplitude ): # Get a blank sound mySound = getMediaPath(sec1silence.wav) buildSin = makeSound(mySound) # Set sound constant

sr = getSamplingRate(buildSin) # sampling rate interval = 1.0/ freq # Make sure its floating point samplesPerCycle = interval * sr # samples per cycle maxCycle = 2 * pi for pos in range (0, getLength(buildSin )): rawSample = sin((pos / samplesPerCycle) * maxCycle) sampleVal = int(amplitude*rawSample) setSampleValueAt(buildSin ,pos ,sampleVal) return buildSin Adding pure sine waves together >>> f440=sineWave (440 ,2000) >>> f880=sineWave (880 ,4000) >>> f1320=sineWave (1320 ,8000) >>> addSounds(f880 ,f440)

>>> addSounds(f1320 ,f440) >>> play(f440) >>> explore(f440) >>> just440=sineWave (440 ,2000) >>> play(just440) >>> explore(f440) Adding together 440Hz, 880Hz, and 1320Hz, with increasing

amplitudes. Comparing Comparing the waves Left, 440 Hz; Right, combined wave. In Explorer In the Spectrum view in MediaTools 440, 880, Making more complicated waves Using square waves,

# use float since interval is instead of sine fl point samplesPerCycle = interval * waves, can be a samplingRate richer sound # we need to switch every halfsquareWave(freq,amplitude): cycle Get a blank sound samplesPerHalfCycle = Sound = getMediaPath("sec1silence.wav") int(samplesPerCycle / 2) uare = makeSound(mySound) sampleVal = amplitude

Set music constants s = 1 mplingRate = getSamplingRate(square)i = 1 conds = 1 # play for 1 second for s in range (0, Build tools for this wave getLength(square)): seconds per cycle # if end of a half-cycle terval = 1.0 * seconds / freq if (i > samplesPerHalfCycle): # reverse the amplitude every half-cycle Building sounds with square waves

>>> sq440=squareWave(440,4000) >>> play(sq440) >>> sq880=squareWave(880,8000) >>> sq1320=squareWave(1320,10000) >>> writeSoundTo(sq440,"square440.wav") >>> addSounds(sq880,sq440) >>> addSounds(sq1320,sq440) >>> play(sq440) >>> writeSoundTo(sq440,"squarecombined440.wav" ) Basic square wave Summed square wave Sound synthesis techniques

Adding sine and square (and triangle) waves is additive sound synthesis. Most common modern synthesis technique is frequency modulation (FM) synthesis. Much richer sound. Just about any way you can imagine to fill a sound mathematically can lead to an interesting synthesis technique. Create random noise, then filter parts out: Subtractive synthesis

Adding envelopes Most real synthesizers today also allow you to manipulate envelopes An envelope is a definition of how quickly the aspects of the sound change over time For example, the rise in volume (attack), how the volume is sustained over time (sustain), how quickly the sound decays (decay): The ASD envelope Pianos tend to attack quickly, then

decay quickly (without pedals) Flutes tend to attack slowly and sustain as long as you want. Why write sound programs? Arent there audio tools that can do many of these things? Sure, and thats good enoughif thats good enough. If you just want to use a sound, then simply using tools to generate the noise/instrument/sound you want is fine.

Communicating process What if you want to tell someone else how you got that sound, so that they can replicate the process, or even modify the sound in some way, or make it better? You could write down all the steps in a sound application tool. Tedious, error prone. Or you could provide a program. A succinct, executable definition of a process. What is MP3?

MP3 files are files encoded according to the MPEG-3 standard. They are audio files, but they are compressed in special ways. They use a model of how we hear to get rid of some of the sound. If there is a soft sound at the same time as a loud sound, dont record the soft sound They use various compression techniques to

make the sound smaller. WAV files are compressed, but not as much, and dont use any smart models to make themselves smaller. What is MIDI? MIDI is a standard for encoding music, not sound. MIDI literally encodes For this instrument

(track), turn key #42 on then later For this instrument (track), turn key #31 off. The quality of the actual sound depends entirely on the synthesizerthe quality of the instrument generation (whether recorded or synthesized). MIDI files tend to be very, very small. Each MIDI instruction (Play key #42 track 7) is only about five bytes long. Not thousands of bytes long. Playing MIDI in JES

The function playNote allows you to play MIDI piano with JES. playNote takes three inputs: A note number Not a frequencyits literally the piano key number C in the first octave is 1, C# is 2, C in the fourth octave is 60, D in the fourth octave is 62. A duration in milliseconds (1/1000 of a second) An intensity (0-127)

Literally, how hard the key is pressed MIDI Example def song(): playNote(60,200,127) playNote(62,500,127) playNote(64,800,127) playNote(60,600,127) for i in range(1,2): playNote(64,120,127) playNote(65,120,127) playNote(67,60,127)

Recently Viewed Presentations

  • Whats on @ Burnage Sure Start January-March 2018

    Whats on @ Burnage Sure Start January-March 2018

    Burnage. Sure Start CC. 016 219 2932. WellComm Language Screen. Manchester Early Years . Speech and Language Therapy Team. Levsnhulme. Sure Start CC. 0161 219 2958. Didsbury Park . Sure Start CC. 0161 434 1601. Old Moat. Sure Start CC....
  • Questions for Consideration

    Questions for Consideration

    Questions for Consideration What is policy? How do politics affect policy? How does policy affect politics? How does policy achieve change?
  • Los Alamos National Laboratory DOE Review of the Heavy Ion ...

    Los Alamos National Laboratory DOE Review of the Heavy Ion ...

    Helvetica Arial Times New Roman Calibri Math A Symbol MS Pゴシック PMingLiU Default Design Los Alamos National Laboratory Heavy Ion Program PHENIX Phenix Muon and MVD Program Major Institutions Los Alamos Heavy Ion Program Current Personnel Slide 5 Slide 6...
  • Abstract Art & Photography 1 Objectives  Learn about

    Abstract Art & Photography 1 Objectives Learn about

    What might the artist be trying to say about former Prime Minister Tony Blair? * David Hockney, Mother 1, 1885 Pablo Picasso, Dora Maar Seated, 1941 Artist David Hockney used photographs to create photo montages which he has termed joiners....
  • Survey of Semantic Annotation Platforms

    Survey of Semantic Annotation Platforms

    Arial Times New Roman Wingdings Garamond Stream Bitmap Image Searching CiteSeer Metadata Using Nutch CiteSeer CiteSeer CiteSeer CiteSeer - phrase search CiteSeer - term search Goal Methodology - Nutch Methodology - Metadata Methodology Methodology XML to HTML Split Methodology -...
  • How is being a CPA similar to being a West Point Cadet?

    How is being a CPA similar to being a West Point Cadet?

    Principle of Due Care. Due care is the most onerous of the AICPA principles and is defined as an act or a course of action that is required of one by position, social custom, law, or religion, and often as...
  • CHinese Philosophies & Ethical Codes

    CHinese Philosophies & Ethical Codes

    Ms. Susan M. Pojer 551 - 479 B.C. Born in the feudal state of Liu. Became a teacher and editor of books. Li --> Rite, rules, ritual decorum (Binding force of an enduring stable society) Ren --> humaneness, benevolence, humanity...
  • CHARACTERISTICS OF THE AN/PRC-117F OR R/T-1796 Frequency Range

    CHARACTERISTICS OF THE AN/PRC-117F OR R/T-1796 Frequency Range

    ECCM (Frequency Hopping) • SINCGARS (16 kbps FSK Voice/Data and ESIP capabilities) • HAVE QUICK I/II (AM Voice and 16 kbps ASK Data) • Uses MX-18290, CYZ-10 DTD TRANSEC fill devices