Segfault Crash

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Segfault Crash

Walter Hansen
SoX v14.2.0 CentOs 6 current i3 2g mem.

I have a script that processes batches of telephone voice files and it goes
along fine until one of them just rings with no pickup. Then it segfaults.

Here's the command that produces the issue:
sox work/8448364766.wav work/8448364766_sht.wav bandreject 0.5k 100h
bandreject 0.5k 200h bandreject 0.5k 300h highpass -1 1000 highpass -1 2000
silence 1 0.1 1% reverse trim 0.5 silence 1 0.1 1% reverse stat

That command is quite a mouthful. This script did work fine under a much
older version under RedHat something.



------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
On Mar 14 17:27:25, [hidden email] wrote:
> SoX v14.2.0 CentOs 6 current i3 2g mem.

14.2.0 was six years ago. The current version (14.4.1),
has seen some significant improvements.

> I have a script that processes batches of telephone voice files

Can you show us the script?
What is the purpose of this processing?

> and it goes along fine
> until one of them just rings with no pickup.

That should be a small enough file without any private info
- can you please put is somewhere for download?

> Then it segfaults.

And you are sure that it is this one file.
That is, the command below runs fine on all the other files,
and it segfaults on this one input file.

> Here's the command that produces the issue:
> sox work/8448364766.wav work/8448364766_sht.wav bandreject 0.5k 100h
> bandreject 0.5k 200h bandreject 0.5k 300h highpass -1 1000 highpass -1 2000
> silence 1 0.1 1% reverse trim 0.5 silence 1 0.1 1% reverse stat

That's a strange sox command indeed.

Is this what you are actually calling in the script?
i.e. this is what is written in the script, literally,
or is this composed in the script from variables?
That's why  want to see the script too.
Have you tried running this exact command on the file,
outside of the script?

> That command is quite a mouthful.

Why do you use three consecutive bandereject effects
with the same center frequency and different widths?
What exactly are you trying to archive by that?
(And why are you running bandreject at all?)

Then, why are you running two consecutive highpass filters
with different pass cutoff frequencies? That doesn't make
any sense. Also,  you are cutting off above the frequency
where you were doing the bandreject, so what was that for?

The "silence reverse seielce reverse", I assume,
is supposed to cut silence from the end. What exactly
is it meant to do? (Maybe these tricks were needed in 14.2.0)

        Jan


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Johnny Rosenberg
2014-03-15 9:47 GMT+01:00 Jan Stary <[hidden email]>:
On Mar 14 17:27:25, [hidden email] wrote:
> SoX v14.2.0 CentOs 6 current i3 2g mem.

14.2.0 was six years ago. The current version (14.4.1),
has seen some significant improvements.

> I have a script that processes batches of telephone voice files

Can you show us the script?
What is the purpose of this processing?

> and it goes along fine
> until one of them just rings with no pickup.

That should be a small enough file without any private info
- can you please put is somewhere for download?

> Then it segfaults.

And you are sure that it is this one file.
That is, the command below runs fine on all the other files,
and it segfaults on this one input file.

> Here's the command that produces the issue:
> sox work/8448364766.wav work/8448364766_sht.wav bandreject 0.5k 100h
> bandreject 0.5k 200h bandreject 0.5k 300h highpass -1 1000 highpass -1 2000
> silence 1 0.1 1% reverse trim 0.5 silence 1 0.1 1% reverse stat

That's a strange sox command indeed.

Is this what you are actually calling in the script?
i.e. this is what is written in the script, literally,
or is this composed in the script from variables?
That's why  want to see the script too.
Have you tried running this exact command on the file,
outside of the script?

> That command is quite a mouthful.

Why do you use three consecutive bandereject effects
with the same center frequency and different widths?
What exactly are you trying to archive by that?
(And why are you running bandreject at all?)

Then, why are you running two consecutive highpass filters
with different pass cutoff frequencies? That doesn't make
any sense. Also,  you are cutting off above the frequency
where you were doing the bandreject, so what was that for?

The "silence reverse seielce reverse", I assume,
is supposed to cut silence from the end. What exactly
is it meant to do? (Maybe these tricks were needed in 14.2.0)

Sounds strange to me too, but still, is SoX meant to crash when the user does something strange? Wouldn't an error or warning message be better…?

I agree however, that the OP should try a newer version and maybe re-think his command…


Johnny Rosenberg
 

        Jan


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

keghn
In reply to this post by Walter Hansen
Hi,

The average sum of a signed repeating wave should be nearly equal to zero, like your for your phone ring.
And you take take a sample of it every like around 1/10 of a second. Or something else depending on your
style of ring tone?
Maybe you could use the non silence and the above to trigger your recording.

Are you recording on an android or land line?

best of luck,

      keghn

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
On Mar 15 16:17:49, [hidden email] wrote:
> The average sum of a signed repeating wave should be nearly equal to zero, like your for your phone ring.
> And you take take a sample of it every like around 1/10 of a second. Or something else depending on your
> style of ring tone?
> Maybe you could use the non silence and the above to trigger your recording.
> Are you recording on an android or land line?

I fail to see how "average sum of a wave"
or indeed any of the above has anthing to do
with the OP's original question.


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
In reply to this post by Jan Stary
Sorry for the late response. Just back to work Monday. Responses below.

-----Original Message-----
From: Jan Stary
Sent: Saturday, March 15, 2014 1:47 AM
To: [hidden email]
Subject: Re: [SoX-users] Segfault Crash

On Mar 14 17:27:25, [hidden email] wrote:
> SoX v14.2.0 CentOs 6 current i3 2g mem.
>>14.2.0 was six years ago. The current version (14.4.1),
>>has seen some significant improvements.

Thought that might be the case. I guess I'll try removing the yum version
and installing from source. It worked fine under the really ancient version
of sox which I think was from something like RedHat 8. That machine (P3-800)
crashed and I built a new one and installed the latest CentOS.

> I have a script that processes batches of telephone voice files
>>Can you show us the script?
>>What is the purpose of this processing?

Looking at it I don't think posting the full code would add anything.

We're a small phone company and we have a caller that checks the messages on
our numbers. We had a guy figure out a way to have the calling program check
the message against a sample. It's rather simplistic. It simply strips the
recording down to the base message eliminating the ringing and any following
dead air. It then compares the length of the the message and deems it a
match if it's pretty close. In practice it works with less than a 1% margin
of error.

This is the sox filter he uses to pare the message down to it's base. I
haven't fully broken it down to see what magic it does.

>That should be a small enough file without any private info
>- can you please put is somewhere for download?

I put some files up on a server:
http://64.142.127.158/sox/

message.wav - a normal full recording
message_sht.wav - after being pared down by the sox command
message_stat.txt - the text file that sox can output with data on the sound
file
ringonly.wav - this is one of the files that caused the segfault when the
sox command was executed

They actually are phone number file names, but I changed it so they'd be
easier to read.

>And you are sure that it is this one file.
>That is, the command below runs fine on all the other files,
>and it segfaults on this one input file.

I tried it manually and got the same thing. Watching the script run it goes
through hundreds of files in short order and then produces an error. I see
it happen and then I also have the output that matches. It's not the end of
the world but is pretty annoying. The segfault generates an email message,
if it didn't do that I'd probably ignore it, but no doubt these messages
will end up building up as I'll forget to go in and prune them.

>> Here's the command that produces the issue:
>> sox work/8448364766.wav work/8448364766_sht.wav bandreject 0.5k 100h
>> bandreject 0.5k 200h bandreject 0.5k 300h highpass -1 1000 highpass -1
>> 2000
>> silence 1 0.1 1% reverse trim 0.5 silence 1 0.1 1% reverse stat
>
>That's a strange sox command indeed.
>
>Is this what you are actually calling in the script?
>i.e. this is what is written in the script, literally,
>or is this composed in the script from variables?
>That's why  want to see the script too.
>Have you tried running this exact command on the file,
>outside of the script?

Yes, that's the command:
    sox -V1 ${in_wav_file} ${out_wav_file} \
               bandreject 0.5k 100h \
               bandreject 0.5k 200h \
               bandreject 0.5k 300h \
               highpass -1 1000 highpass -1 2000 \
               silence 1 0.1 1% \
               reverse trim 0.5 \
               silence 1 0.1 1% \
               reverse stat >& $stat_file

I added the -V1 to the script as the new version of sox seems to default to
verbose mode. One of the first things I did was try it outside the script
and yes it did the exact same segfault.

>> That command is quite a mouthful.
>Why do you use three consecutive bandereject effects
>with the same center frequency and different widths?
>What exactly are you trying to archive by that?
>(And why are you running bandreject at all?)
>
>Then, why are you running two consecutive highpass filters
>with different pass cutoff frequencies? That doesn't make
>any sense. Also,  you are cutting off above the frequency
>where you were doing the bandreject, so what was that for?
>
>The "silence reverse seielce reverse", I assume,
>is supposed to cut silence from the end. What exactly
>is it meant to do? (Maybe these tricks were needed in 14.2.0)

I think all this is answered above. I haven't decoded what it's doing bit by
bit but it's taking a full call and reducing it to just the message. The
ring only recordings cause it to segfault where I'd expect it to just give a
blank output file or at least come back with an error message that I could
deal with. Instead it just crashes.


>Jan




------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
In reply to this post by keghn
I don’t really understand this. The caller program calls a number using a voicemodem on a landline. It records around 30 seconds which generally incudes a couple rings, some sort of message (or somebody saying Hello?), and then some dead air. The sox command pares it down to just the message. It worked fine with the very old version of sox (ten years old or so), but with the current CentOS yum version it segfaults.
 
It sounds like doing a source install is a good idea.
 
Sent: Saturday, March 15, 2014 1:17 PM
Subject: Re: [SoX-users] Segfault Crash
 
Hi,

The average sum of a signed repeating wave should be nearly equal to zero, like your for your phone ring.
And you take take a sample of it every like around 1/10 of a second. Or something else depending on your
style of ring tone?
Maybe you could use the non silence and the above to trigger your recording.

Are you recording on an android or land line?

best of luck,

      keghn


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech


_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
In reply to this post by Jan Stary
I uninstalled the default version and did a source install of 14.4.1 and
voila it's now running smoothly. Running the command manually on one of the
problem ring only files produces an output that says something like "this is
probably a text file not sound" and lists the length as 0 which is just
fine.



------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
urg. Now it's never heard of a PVF file even though it's never had an issue
converting them before.


-----Original Message-----
From: Walter Hansen
Sent: Monday, March 17, 2014 3:15 PM
To: [hidden email]
Subject: Re: [SoX-users] Segfault Crash

I uninstalled the default version and did a source install of 14.4.1 and
voila it's now running smoothly. Running the command manually on one of the
problem ring only files produces an output that says something like "this is
probably a text file not sound" and lists the length as 0 which is just
fine.



------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users 


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
In reply to this post by Walter Hansen
On Mar 17 11:21:21, [hidden email] wrote:
> This is the sox filter he uses to pare the message down to it's base.
> I haven't fully broken it down to see what magic it does.

That particular command seems to be a misguided way
to strip the initial dial tones and the final silence,
see comments below.


> >That should be a small enough file without any private info
> >- can you please put is somewhere for download?
>
> I put some files up on a server:
> http://64.142.127.158/sox/
>
> message.wav - a normal full recording
> message_sht.wav - after being pared down by the sox command
> message_stat.txt - the text file that sox can output with data on the sound
> file
> ringonly.wav - this is one of the files that caused the segfault when the
> sox command was executed

You call it 'ringonly', but it is almost identical to the
message.wav, a normal message, where the dial tones are followed
by the lady's announcement about the number being available;
it's not the ring only. Are you sure that this is the counterexample
you intended?

> I tried it manually and got the same thing.
>     sox -V1 ${in_wav_file} ${out_wav_file} \
>                bandreject 0.5k 100h \
>                bandreject 0.5k 200h \
>                bandreject 0.5k 300h \
>                highpass -1 1000 highpass -1 2000 \
>                silence 1 0.1 1% \
>                reverse trim 0.5 \
>                silence 1 0.1 1% \
>                reverse stat >& $stat_file

SoX 14.4.1 runs that exact command just fine on ringonly.wav.
No segfault, the actual message gets isolated.

> I added the -V1 to the script as the new version of sox seems to default to
> verbose mode.

The default is -V2, which is errors and warning.
See the manpage.

> >> That command is quite a mouthful.
> >Why do you use three consecutive bandereject effects
> >with the same center frequency and different widths?
> >What exactly are you trying to archive by that?
> >(And why are you running bandreject at all?)
> >
> >Then, why are you running two consecutive highpass filters
> >with different pass cutoff frequencies? That doesn't make
> >any sense. Also,  you are cutting off above the frequency
> >where you were doing the bandreject, so what was that for?
> >
> >The "silence reverse seielce reverse", I assume,
> >is supposed to cut silence from the end. What exactly
> >is it meant to do? (Maybe these tricks were needed in 14.2.0)
>
> I think all this is answered above.
> I haven't decoded what it's doing bit by bit
> but it's taking a full call and reducing it to just the message.

No, it's not answered at all.
If the whole point of that effects chain is to filter out
the initial ringing (or silence it enough so that it
does not interfere with the following 'silence' effect),
and cut the tail silence from the recording,
there is a much simpler way to do that.

What you have there now seems as leftovers of a trial-and-error
that eventually led to those constants being used,
but the particular effect chain makes no sense really.

First of all, the dial tone is well defined:
http://en.wikipedia.org/wiki/Dual-tone_multi-frequency#Special_tone_frequencies
Indeed, try "play -n -c 1 -r 8000 synth 1.5 trap 440 sin 480 gain -6",
that's the ringing I hear in your recordings.
Run "sox message.wav -n spectrogram"
and look at the resulting spectrogram.png
- see those two short lines at the beginning?

So to filter that out, you want to filter out frequencies 440 and 480
(and probably some narrow band around them as well).

Instead, your sox command rejects 500Hz and a 100Hz band around it;
and then it rejects 500Hz and a 200Hz band around it,
making the previous bandreject irrelevant;
and then it rejects 500Hz and a 300Hz band around it;
making both the previous irrelevant. Makes no sense. Also, try
play message.wav bandreject 500 100 bandreject 500 200 bandreject 500 300
- it silences those ring tones a bit, but not entirely.
Where does the '500' come from, really? Trial and error I say,
and not knowing that it's actually 440 and 480.

Next, your sox xommand does "highpass -1 1000",
making the whole dance around 500Hz irrelevant.
And THEN, it does "highpass -1 2000", rendering
ALL of the previous irrelevant.

Instead of all that, you could get a better result with e.g.
"play message.wav sinc 600"; try it yourself on message.wav

The silence effect that follows can probably be simplified too:
you don't need the "silence reverse silence reverse",
you can cut silence from both the beginning and the end
in one run; see the description of the silence effect
in the manpage.

> The ring only recordings cause it to segfault
> where I'd expect it to just give a blank output file

Now I am pretty sure that the ringonly.wav is not what you meant
- it's not 'ring only'.


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
In reply to this post by Walter Hansen
On Mar 17 15:15:44, [hidden email] wrote:
> I uninstalled the default version and did a source install of 14.4.1 and
> voila it's now running smoothly. Running the command manually on one of the
> problem ring only files produces an output that says something like "this is
> probably a text file not sound" and lists the length as 0 which is just
> fine.

Just to be sure: this is your script's output, not sox's, right?
What is the result of running _just_ the sox command over that file
(with -V, and without the final 'stat')?


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
In reply to this post by Walter Hansen
On Mar 17 11:21:21, [hidden email] wrote:
> We're a small phone company and we have a caller that checks the messages on
> our numbers. We had a guy figure out a way to have the calling program check
> the message against a sample. It's rather simplistic. It simply strips the
> recording down to the base message eliminating the ringing and any following
> dead air. It then compares the length of the the message and deems it a
                             ^^^^^^^^^^
> match if it's pretty close.

>     sox -V1 ${in_wav_file} ${out_wav_file} \
>                bandreject 0.5k 100h \
>                bandreject 0.5k 200h \
>                bandreject 0.5k 300h \
>                highpass -1 1000 highpass -1 2000 \
>                silence 1 0.1 1% \
>                reverse trim 0.5 \
>                silence 1 0.1 1% \
>                reverse stat >& $stat_file

The '>&' puzzles me but maybe that's how your shell does redirection.
In my shell, I would say '2> statfile'.

But if you only use the _length_ to compare it,
you don't even need the 'stat' effect, just
soxi -D $out_wav_file


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
In reply to this post by Walter Hansen
On Mar 17 15:15:44, [hidden email] wrote:
> I uninstalled the default version and did a source install of 14.4.1 and
> voila it's now running smoothly. Running the command manually on one of the
> problem ring only files produces an output that says something like "this is
> probably a text file not sound" and lists the length as 0 which is just
> fine.

Can you please post that exact file somewhere?
I will try to run both 14.4.1 and 14.2.0 over it.


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
In reply to this post by Walter Hansen
On Mar 17 16:45:01, [hidden email] wrote:
> urg. Now it's never heard of a PVF file even though it's never had an issue
> converting them before.

What you have now is a different installation of SoX.
How _exactly_ did you install it? What did ./configure said?
What supporting libraries has it found (mp3, ogg, pvf, ...)?


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
In reply to this post by Jan Stary
Interesting. The ">& filename" seems to be an antiquated way to say
">filename 2>&1" and is equivalent to "&>filename". Looks like it's obtuse
on purpose. Again I didn't write it. Thanks for the soxi command. I'm
thinking of pulling the whole thing straight into the perl script from which
it is run. The guy uses two languages, shell script and a math language to
do something that's really simple. He probably considers his code to be
elegant; but I think it's downright cryptic, not that I've done any serious
shell scripting in years.

-----Original Message-----
From: Jan Stary
Sent: Monday, March 17, 2014 5:30 PM
To: [hidden email]
Subject: Re: [SoX-users] Segfault Crash

On Mar 17 11:21:21, [hidden email] wrote:
> We're a small phone company and we have a caller that checks the messages
> on
> our numbers. We had a guy figure out a way to have the calling program
> check
> the message against a sample. It's rather simplistic. It simply strips the
> recording down to the base message eliminating the ringing and any
> following
> dead air. It then compares the length of the the message and deems it a
                             ^^^^^^^^^^
> match if it's pretty close.

>     sox -V1 ${in_wav_file} ${out_wav_file} \
>                bandreject 0.5k 100h \
>                bandreject 0.5k 200h \
>                bandreject 0.5k 300h \
>                highpass -1 1000 highpass -1 2000 \
>                silence 1 0.1 1% \
>                reverse trim 0.5 \
>                silence 1 0.1 1% \
>                reverse stat >& $stat_file

The '>&' puzzles me but maybe that's how your shell does redirection.
In my shell, I would say '2> statfile'.

But if you only use the _length_ to compare it,
you don't even need the 'stat' effect, just
soxi -D $out_wav_file


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users 


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
In reply to this post by Jan Stary
I put it up at the same location. It's two files.
cmp_pvfs.sh
cmp_lengths.bc


-----Original Message-----
From: Jan Stary
Sent: Monday, March 17, 2014 5:36 PM
To: [hidden email]
Subject: Re: [SoX-users] Segfault Crash

On Mar 17 15:15:44, [hidden email] wrote:
> I uninstalled the default version and did a source install of 14.4.1 and
> voila it's now running smoothly. Running the command manually on one of
> the
> problem ring only files produces an output that says something like "this
> is
> probably a text file not sound" and lists the length as 0 which is just
> fine.

Can you please post that exact file somewhere?
I will try to run both 14.4.1 and 14.2.0 over it.


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users 


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
In reply to this post by Jan Stary
This message got buried in my spam filter for some reason.

This sounds lovely. My boss outsourced this project to a contractor because
he thinks the contractor is cheap and skilled. Sounds like he did the
absolute minimal job. I think we paid him for weeks of labor on this. But
then on the other hand it does work.

I'll try to play around with some of the conversions you've suggested today.
His actual code runs from a shell script going to a math script called bc
when all of this could be handled in a single perl program easily. It also
generates a ton of extraneous output and doesn't clean up after itself. All
it's doing is stripping down the vocal file and then making a length
comparison; it shouldn't be an issue to put this all in the main perl script
in two or three lines of code. I like perl for it's ability to do just about
anything at the same level as a shell script.

-----Original Message-----
From: Jan Stary
Sent: Monday, March 17, 2014 5:23 PM
To: [hidden email]
Subject: Re: [SoX-users] Segfault Crash

On Mar 17 11:21:21, [hidden email] wrote:
> This is the sox filter he uses to pare the message down to it's base.
> I haven't fully broken it down to see what magic it does.

That particular command seems to be a misguided way
to strip the initial dial tones and the final silence,
see comments below.


> >That should be a small enough file without any private info
> >- can you please put is somewhere for download?
>
> I put some files up on a server:
> http://64.142.127.158/sox/
>
> message.wav - a normal full recording
> message_sht.wav - after being pared down by the sox command
> message_stat.txt - the text file that sox can output with data on the
> sound
> file
> ringonly.wav - this is one of the files that caused the segfault when the
> sox command was executed

You call it 'ringonly', but it is almost identical to the
message.wav, a normal message, where the dial tones are followed
by the lady's announcement about the number being available;
it's not the ring only. Are you sure that this is the counterexample
you intended?

> I tried it manually and got the same thing.
>     sox -V1 ${in_wav_file} ${out_wav_file} \
>                bandreject 0.5k 100h \
>                bandreject 0.5k 200h \
>                bandreject 0.5k 300h \
>                highpass -1 1000 highpass -1 2000 \
>                silence 1 0.1 1% \
>                reverse trim 0.5 \
>                silence 1 0.1 1% \
>                reverse stat >& $stat_file

SoX 14.4.1 runs that exact command just fine on ringonly.wav.
No segfault, the actual message gets isolated.

> I added the -V1 to the script as the new version of sox seems to default
> to
> verbose mode.

The default is -V2, which is errors and warning.
See the manpage.

> >> That command is quite a mouthful.
> >Why do you use three consecutive bandereject effects
> >with the same center frequency and different widths?
> >What exactly are you trying to archive by that?
> >(And why are you running bandreject at all?)
> >
> >Then, why are you running two consecutive highpass filters
> >with different pass cutoff frequencies? That doesn't make
> >any sense. Also,  you are cutting off above the frequency
> >where you were doing the bandreject, so what was that for?
> >
> >The "silence reverse seielce reverse", I assume,
> >is supposed to cut silence from the end. What exactly
> >is it meant to do? (Maybe these tricks were needed in 14.2.0)
>
> I think all this is answered above.
> I haven't decoded what it's doing bit by bit
> but it's taking a full call and reducing it to just the message.

No, it's not answered at all.
If the whole point of that effects chain is to filter out
the initial ringing (or silence it enough so that it
does not interfere with the following 'silence' effect),
and cut the tail silence from the recording,
there is a much simpler way to do that.

What you have there now seems as leftovers of a trial-and-error
that eventually led to those constants being used,
but the particular effect chain makes no sense really.

First of all, the dial tone is well defined:
http://en.wikipedia.org/wiki/Dual-tone_multi-frequency#Special_tone_frequencies
Indeed, try "play -n -c 1 -r 8000 synth 1.5 trap 440 sin 480 gain -6",
that's the ringing I hear in your recordings.
Run "sox message.wav -n spectrogram"
and look at the resulting spectrogram.png
- see those two short lines at the beginning?

So to filter that out, you want to filter out frequencies 440 and 480
(and probably some narrow band around them as well).

Instead, your sox command rejects 500Hz and a 100Hz band around it;
and then it rejects 500Hz and a 200Hz band around it,
making the previous bandreject irrelevant;
and then it rejects 500Hz and a 300Hz band around it;
making both the previous irrelevant. Makes no sense. Also, try
play message.wav bandreject 500 100 bandreject 500 200 bandreject 500 300
- it silences those ring tones a bit, but not entirely.
Where does the '500' come from, really? Trial and error I say,
and not knowing that it's actually 440 and 480.

Next, your sox xommand does "highpass -1 1000",
making the whole dance around 500Hz irrelevant.
And THEN, it does "highpass -1 2000", rendering
ALL of the previous irrelevant.

Instead of all that, you could get a better result with e.g.
"play message.wav sinc 600"; try it yourself on message.wav

The silence effect that follows can probably be simplified too:
you don't need the "silence reverse silence reverse",
you can cut silence from both the beginning and the end
in one run; see the description of the silence effect
in the manpage.

> The ring only recordings cause it to segfault
> where I'd expect it to just give a blank output file

Now I am pretty sure that the ringonly.wav is not what you meant
- it's not 'ring only'.


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users 


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
On Mar 19 09:23:32, [hidden email] wrote:

> This sounds lovely. My boss outsourced this project to a contractor because
> he thinks the contractor is cheap and skilled. Sounds like he did the
> absolute minimal job. I think we paid him for weeks of labor on this. But
> then on the other hand it does work.
>
> I'll try to play around with some of the conversions you've suggested today.
> His actual code runs from a shell script going to a math script called bc
> when all of this could be handled in a single perl program easily. It also
> generates a ton of extraneous output and doesn't clean up after itself. All
> it's doing is stripping down the vocal file and then making a length
> comparison; it shouldn't be an issue to put this all in the main perl script
> in two or three lines of code. I like perl for it's ability to do just about
> anything at the same level as a shell script.

Well, I like shell for its ability to do almost anything
at the same level as Perl :-) Not that I am a shell guru,
but the script you sent sure is a coding attrocity ...


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Walter Hansen
I put the code al inline in my perl program. I tried it with this:

`sox -V0 $dir/calls/$number.pvf $dir/calls/$number.2.pvf sinc 600 silence 1
0.1 1% reverse silence 1 0.1 1% reverse`;

and the lengths did not match up very well at all. I played around with the
values a bit, but the lengths just didn't match up as well. So I shrugged
and put this in:

`sox -V0 $dir/calls/$number.pvf $dir/calls/$number.2.pvf bandreject 0.5k
100h bandreject 0.5k 200h bandreject 0.5k 300h highpass -1 1000 highpass -1
2000 silence 1 0.1 1% reverse trim 0.5 silence 1 0.1 1% reverse`;

and suddenly they matched up just like before. I'm just pulling the length
from soxi -D.

However now I'm just pulling the sample length once and using it to check
the recordings which is much more efficient.

I still have problems putting my head around this code, copied from the bc
math language into perl. I know what he's trying to do, but it seems it
could be done in a way which is simpler and easier to read.

my $mindifmult = 12;
print "MinDifMult: $mindifmult\n";
my $sum = $sampleLength + $recordingLength;
print "Sum: $sum\n";
my $avg = $sum * 0.5;
print "Avg: $avg\n";
my $dif = abs($sampleLength - $recordingLength);
print "Dif: $dif\n";
my $difmult;
if ($dif > 0) {$difmult = $avg/$dif;} else {$difmult = 1000;}
print "DifMult: $difmult\n";
if ($difmult > $mindifmult) {
    $output = "same";
  } else {
    $output = "different"; }




-----Original Message-----
From: Jan Stary
Sent: Wednesday, March 19, 2014 3:05 PM
To: [hidden email]
Subject: Re: [SoX-users] Segfault Crash

On Mar 19 09:23:32, [hidden email] wrote:

> This sounds lovely. My boss outsourced this project to a contractor
> because
> he thinks the contractor is cheap and skilled. Sounds like he did the
> absolute minimal job. I think we paid him for weeks of labor on this. But
> then on the other hand it does work.
>
> I'll try to play around with some of the conversions you've suggested
> today.
> His actual code runs from a shell script going to a math script called bc
> when all of this could be handled in a single perl program easily. It also
> generates a ton of extraneous output and doesn't clean up after itself.
> All
> it's doing is stripping down the vocal file and then making a length
> comparison; it shouldn't be an issue to put this all in the main perl
> script
> in two or three lines of code. I like perl for it's ability to do just
> about
> anything at the same level as a shell script.

Well, I like shell for its ability to do almost anything
at the same level as Perl :-) Not that I am a shell guru,
but the script you sent sure is a coding attrocity ...


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users 


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
Reply | Threaded
Open this post in threaded view
|

Re: Segfault Crash

Jan Stary
On Mar 19 15:36:50, [hidden email] wrote:

> I put the code al inline in my perl program. I tried it with this:
>
> `sox -V0 $dir/calls/$number.pvf $dir/calls/$number.2.pvf sinc 600 silence 1
> 0.1 1% reverse silence 1 0.1 1% reverse`;
>
> and the lengths did not match up very well at all. I played around with the
> values a bit, but the lengths just didn't match up as well. So I shrugged
> and put this in:
>
> `sox -V0 $dir/calls/$number.pvf $dir/calls/$number.2.pvf bandreject 0.5k
> 100h bandreject 0.5k 200h bandreject 0.5k 300h highpass -1 1000 highpass -1
> 2000 silence 1 0.1 1% reverse trim 0.5 silence 1 0.1 1% reverse`;
>
> and suddenly they matched up just like before. I'm just pulling the length
> from soxi -D.
>
> However now I'm just pulling the sample length once and using it to check
> the recordings which is much more efficient.

Well, that's what I was asking about: why would you try to match
every recording against every other recording? I am not sure
I understand the big picture completely: these numbers you are calling
are phone numbers that your company administers, right? So you know,
in advance, whether a certain number is assigned to a user (caller),
or whether it will respond with some prerecorded message, and which one
-- because it's you whi set that up so, right? So I still don't see
clearly why you are doing this ...

But let's keep this SoX-related. On that one message.wav I could test on,
my command does what I understood is the point, namely ctrip the 00:00:30
recording to the actual message. I am not saying these are _the_right_
numbers to use - my whole point is that what you do can be done more
easily, and in fact, calling the sequence of bandrejects does not
really make much sense.

What happens whan you e.g. replace the sequence of bandrejects
with just the final "bandreject 500 300" ?


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Sox-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sox-users
12