TS-VAP · RESEARCH DEMOS

Speech Extraction Demos

01

Acoustic vs. MiniOmni VAP

Example 01

CONVERSATION VIDEO00:06
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. You got called up.
  2. I thought you said this wasn't going to happen for at least a year. (utterance clipped by this six-second window)

Partner

  1. Iraq.

Third-party

  1. Tell them that. You can go in, get a brand new phone, they'll hook you up with service even give you a free three months-- (utterance clipped by this six-second window)
WaveformsWaveform comparison on a shared time axis

Example 02

CONVERSATION VIDEO
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Conversation

  1. And that was the only thing I asked and then the conversation ended.
  2. Were you surprised about how her voice sounded or?

Third-party source

  1. You got to do two things in a life well under this construct societal construct you got to die you got to pay taxes maybe the traditional way of how we view death and grieving isn't what it has to be there can be rejoicing there's beauty there is transition there is power

ASR segments are approximate; speaker attribution is not verified.

WaveformsWaveforms on a shared time axis

Example 03

CONVERSATION VIDEO00:06
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. You know I would never let that happen. I wouldn't let that happen. (utterance clipped by this six-second window)

Partner

  1. No, no, really, if I can't pay to live somewhere, I'm going to end up living in my car. (utterance clipped by this six-second window)

Third-party

  1. It's funny it hasn't worked such wonders for you. (utterance clipped by this six-second window)
WaveformsWaveform comparison on a shared time axis

Example 04

CONVERSATION VIDEO
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Conversation

  1. And then I was just like, five minutes later she started crying and I was like, oh, nope, never mind. I hate this.
  2. Like I'm not mentally ready for this.
  3. Nobody's ever really ready for it.
  4. I feel like some people are ready for it. More ready. More ready, yeah.

Third-party source

  1. And I feel you're on this path.

ASR segments are approximate; speaker attribution is not verified.

WaveformsWaveforms on a shared time axis

Example 05

TARGET VIDEO00:12
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. …But it’s not totally blocking out the main disk of the sun.
  2. …In this ring of fire.

Added third voice

  1. Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel.
  2. Nor is Mr. Quilter’s manner less interesting…

Single-speaker source with an added third voice. The target reference is the original video soundtrack.

WaveformsTarget, added third voice, mixture and three model outputs on one time axis

Example 06

CONVERSATION VIDEO00:12
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. You have to tell me.
  2. It wouldn't work like this.

Partner

  1. I don't I don't know where to start.

Added third voice

  1. A laudable regard for the honor of the first proselyte has countenanced the belief, the hope, the wish, that the Ebionites, or at least the Nazarenes, were distinguished only by their obstinate perseverance in the practice of the Mosaic rites.
WaveformsWaveform comparison on a shared time axis

Example 07

CONVERSATION VIDEO00:09
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. No, well I go to them, but I do not sing.
  2. I haven't been to any here. The only ones I've been to were actually in Korea.

Partner

  1. What's your favorite spots?

Added third voice

  1. He translated at an early age, chiefly between eighteen forty-five and eighteen forty-nine, a great number of poems by the Italians contemporary with Dante or preceding him, and among other things he made a version of the whole Vita Nuova, prose and verse.
WaveformsTarget, partner, third-party, mixture and model outputs on one time axis

Example 08

CONVERSATION VIDEO00:09
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. Ironically, I don't think I even had a single sale that day, so I didn't even get paid that day.
  2. Yeah, I didn't make any money that day, and that was

Partner

  1. Oh, God. I can't.

Added third voice

  1. Grant was only a few miles away, but although commander in chief, he knew nothing of the hardest-fought battle of the Civil War until it was over.
WaveformsTarget, partner, third-party, mixture and model outputs on one time axis

Example 09

CONVERSATION VIDEO00:10
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. Is it a thick book, like pages plus, or do you think it's 500-ish?
  2. Oh, okay, okay. Oh, it's a textbook. Thank you for that information. I think we can

Partner

  1. I don't know. I think it's a textbook.

Added third voice

  1. The week following Christmas brought in a thaw, and by New Year's Day all the world about us was a broth of gray slush, and the guttered slope between the windmill and the barn was running black water.
WaveformsTarget, partner, third-party, mixture and model outputs on one time axis
02

Context configurations

History self-memory · Sync ASD · Pred MiniOmni VAP

Example 01

CONVERSATION VIDEO00:06
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. Elliot, worms don't pop. (utterance clipped by this six-second window)

Partner

  1. I don't mind what you do, see, you could paint your body green all over and dance naked through the Place Vendome and run off with every man in the world. And I shan't say a word as long as I know that you love me best. (utterance clipped by this six-second window)

Third-party

  1. Oh. Grow up Augie, the fish are not going to come. (utterance clipped by this six-second window)
WaveformsWaveform comparison on a shared time axis

Example 02

CONVERSATION VIDEO
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Conversation

  1. Come on.
  2. Never thought it would come back around on you.
  3. Aw man.
  4. One question.
  5. What I want to ask...

Third-party source

  1. And I think I always ask you like, does it hurt?

ASR segments are approximate; speaker attribution is not verified.

WaveformsWaveforms on a shared time axis

Example 03

CONVERSATION VIDEO
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Conversation

  1. Now. Yeah. What's, what's up there? He's in, um, familiar. Yeah. Okay. And where, where's God? He's somewhere up there.
  2. Yeah. Yeah. So God's a man? Yeah, he is. How did dad die? Why did he die? He had three left. Why did dad die? Do you know?

Third-party source

  1. The lawyer, you think that they're on your side, but after the proceedings, they went and had lunch.

ASR segments are approximate; speaker attribution is not verified.

WaveformsWaveforms on a shared time axis

Example 04

CONVERSATION VIDEO00:10
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

Yeah, well, absolutely. What we can -- let's just go ahead and just do a basic diagnostic of your system first off, okay. Can you go down to your start button?

Okay, And uh- go ahead and click on run.

Partner

Well, I- I can check that. Is there anything else we can do, too? Like, I'll go through and check the line throughout the house, but --

Uh huh.

Third-party

Russel Crowe wasn't forty until he got his big stuff.

WaveformsWaveforms on a shared time axis

Example 05

TARGET VIDEO00:16
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target · approximate ASR

  1. You know, it's inspiring because I think, like, for me, I'm all about, you know, my dad
  2. And I have had this saying where it's like, all you can do is try your best and all you
  3. Can do is be your best and that's all you can do.
  4. And so it's really inspiring.
  5. You know, I feel like that mindset is kind of the same.
  6. It's just like, be all you can be.

Added third voice

  1. Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel. Nor is Mr. Quilter’s manner less interesting than his matter.

Single-speaker interview with an added third voice. The original soundtrack is the target reference.

WaveformsTarget, third voice, mixture, baseline, History and Sync Pred comparison waveforms

Example 06

CONVERSATION VIDEO00:09
Target
Partner
PLAYINGMixture
Select audio
References
Model outputs
Transcripts

Target

  1. Maybe we can add swimming to it, since you and I are good at swimming.
  2. Okay, you don't want to have that. How about basketball?

Partner

  1. The swimming part, I don't want to add that.
  2. It's just going out of the lines of fitness and dance. (utterance clipped by this window)

Added third voice

  1. Her meeting with Letty was indescribably tender, and the days that followed were pretty equally divided between her and her brother in nursing the one and loving the other.
WaveformsTarget, partner, third-party, mixture and context model waveforms on one time axis