OTHER TECH

So, we wanted to define a syntax describing how the audio objects exist in three-dimensional space and how they’re behaving. Some of them may be stationary and some may be moving, but, essentially, rather than mixing to specific speaker locations, you’re mixing into a three-dimensional space without regard for how many speakers might be used later. Once you’ve retained that information, you can use that to render to any arbitrary speaker configuration. You would be able to take information from the creative community that says, “We want this object here, that object there,” or, “We want reverb to come from here”—and adaptively map that to whatever playback resources are available based on what the renderer knows about the playback environment. So if you have two speakers in a TV, fine, we’ll do some virtual processing or some psychoacoustics using that information. I may decide, as a home theater enthusiast, that I want 33.3 channels. Well, because the mix is in space rather than a particular channel configuration, I can do that, and basically I’m able to achieve more resolution. Someone heard us talking about this and said, you know, the way you’re describing this, it’s as though speakers are like pixels. The more speakers you have, the more resolution you have, and the better ability you have to place things in space.

RS: So then, in essence, if you’re the guy mixing a movie, you end up with a three-dimensional sort of grid in front of you? And rather than thinking in terms of the actual speakers, you’re just programming, or placing things to move through this space?

AK: Yes. We’ve actually done a mix like that as a proof of concept. And it’s very interesting, because when you’re a mixer, you have to quickly forget about the idea that you’re mixing to a particular speaker or target channel and get used to the fact that you’re actually placing and creating a threedimensional audio experience, regardless of how many speakers there might be later or where they are. It’s almost like you’re putting a scrim over the speakers.

RS: OK, so what does that then mean for the listener if they walked into an MDA-equipped theater? What would they hear?

AK: Having this kind of information just opens up the door for development of all kinds of techniques for rendering a soundfield that would not even be considered if you’re just limited to [amplitude-only driven] speakers around the edges of the room. For example, when you were here, you heard a demonstration of our CircleCinema 3D depth-rendering technology. It creates this feeling of depth—objects moving toward you and away from you, receding from you. You can’t do that with regular surround—you have speakers behind you and in front of you, but there’s no speakers close up to your face. Now, with that demo, you only heard an approximation of where we think things should be. But with the full evolution to MDA, the creator can tell us, say, we want objects panned not just around a twodimensional plane, around the edges like you can do with traditional surround, but also around interior points, immersive points. Even with 7.1 today, the sound is still in a two-dimensional plane, and it still clings to the speakers and to the walls—you don’t get the feeling that you’re immersed in the soundfield. So, by applying different approaches, different technologies, there’s a lot of things that can be done to create a more immersive and involving sound experience.

Two-Speaker Surround
RS: I listened to the 2.1-channel demo of CircleCinema 3D and found it engaging in ways I’ve never heard from any traditional 5.1-channel system. How can a two-channel system possibly be better?

AK: Well, the basic concept is that the hearing system is a very sophisticated thing, and not limited to simple amplitude panning—not limited to location based on whether the sound is louder in one place and quieter in another. There are a lot of mechanisms at work. So why not use what we know about the hearing system to create this perception of three-dimensional space and emergence without a lot of boxes hanging around the room? That defines the whole theory of psychoacoustics: Let’s use the hearing system directly, rather than indirectly with speakers all over the place.

Now, it turns out that one of the techniques for projecting sound into space based on the auditory system is something called HRTF, or head-related transfer functions, where the frequency or spectral characteristics of a broadband audio signal, like speech or music, will vary depending on the angle relative to the ear canal. And that’s because of the structure of the head and the outer ear, and the shoulders—everything. And by understanding how that changes, we can take advantage of HRTF to create sounds in three-dimensional space, from a perception standpoint, that aren’t actually coming from speakers.

The next thing that you heard with CC3D was another psychoacoustic phenomenon that we kind of discovered last year about what sounds do when they come closer versus moving farther away. And we found that we were able to simulate something that normally can’t be done with traditional surround sound, which is proximity. Obviously, it happens to us every day in real life—if I walk closer to you, you can tell that my voice is coming from closer to you; if I walk away, you can tell I’m walking away. And again, that’s not just amplitude. So we’re taking advantage of what we learned there to create this feeling that things are being projected into space in the D axis, the depth axis.

So, when you have this two-speaker concept, this minimal-number-of-speakers concept, you can create a much more immersive soundfield because it’s matching playback to the human ear-brain system, to the perception system, in a much better way than you can when you’re just simulating surround by putting speakers around the room.

RS: Are you saying that in the ideal surround sound world, the only time you’d have more than a pair of speakers up front is when you were maybe just needing to fill the back part of a large auditorium or listening room?

AK: Almost. The other reason is to sometimes put something directly behind you, rather than having only this kind of immersive field of depth projection [from the front]. Most of the stuff that occurs when you’re watching a movie or TV is occurring in front of you, though occasionally there are things that kind of fly in from behind you.

So what we’re working on now is essentially a multichannel or 5.1 version of the twochannel system that you heard. You can have speakers back there, as long as you treat all the speakers with the same kind of technology to maintain this immersion.

But the main thing is, the potential for exploring these kinds of techniques is really unlimited. I mean, look, I think what you heard is pretty good—it impresses me and a lot of people. But there’s really no limit—we can go well beyond that, even. And maybe we will, or other people will.