[SOLVED] Differences in Perl Scripts

All installation and configuration problems and questions

Moderators: gerski, enjay, williamconley, Op3r, Staydog, gardo, mflorell, MJCoate, mcargile, Kumba, Michael_N

[SOLVED] Differences in Perl Scripts

Postby AlSam » Fri May 15, 2020 11:36 am

I am interested in backing up recordings and am not sure which script I should use. The two that may help are AST_CRON_audio_3_ftp_FTPSSL.pl and AST_CRON_audio_4_ftp2_FTPSSL.pl but I don't know the difference in their functionality. Can anyone tell me what they are? Also, as I was reading the first few lines of each script, I found that one is a STEP-3 and the other a STEP-4 program. Must there be a STEP-3 program being run before running a STEP-4 program?
Last edited by AlSam on Fri Aug 28, 2020 2:21 pm, edited 1 time in total.
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby AlSam » Tue May 19, 2020 7:14 am

..or maybe is there documentation available on them (besides the comments at the beginning of them)?
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Tue May 19, 2020 9:54 am

Yes, a level-4 CRON-audio script can only run after the level-3 script has run, this is because of where the recordings are located. We created the level-4 scripts so that clients wanting to send recordings to 2 separate archive servers could do so in an automated way.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Differences in Perl Scripts

Postby AlSam » Tue May 19, 2020 10:07 am

Thanks for the response. Do these scripts move recordings or just copy them?
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby williamconley » Tue May 19, 2020 10:40 am

AlSam wrote:Thanks for the response. Do these scripts move recordings or just copy them?


Vicidial Recording Management Clarified
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20018
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: Differences in Perl Scripts

Postby AlSam » Tue May 19, 2020 12:41 pm

Thanks for the info, williamconley. I reviewed it and am a bit confused by the first two lines in point 4. The first bullet states that the mp3 file is moved from one folder to another but the following bullet states that it is now redundant. How is it redundant if it has been moved and not just copied?
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby AlSam » Tue May 19, 2020 12:50 pm

Also, one of the points references the astguiclient.conf. Is this meant to have the server directly deposit recordings to the server configured in the file? Does the perl script depend on the configurations in the astguiclient.conf file?
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby williamconley » Tue May 19, 2020 12:56 pm

AlSam wrote:Thanks for the info, williamconley. I reviewed it and am a bit confused by the first two lines in point 4. The first bullet states that the mp3 file is moved from one folder to another but the following bullet states that it is now redundant. How is it redundant if it has been moved and not just copied?

Because it was moved from one place to another on the Dialer. But it was first Copied to the archive server. Thus the local file is now the redundant copy. This is signified by moving it to the "you can delete me, because I'm redundant" FTP folder.

Once the upload to the archive server has completed, there is Zero need for the copy on the dialer. However: If you don't have a mirror or other copy of the audio files on the archive server, now you are left with the danger of the Archive server's HD dying the losing ALL your recordings now that they are conveniently all collected from all dialers to a single location. That single location is now prone to death of HD. You can either set up a mirror of that drive ... or use FTP2 to copy all the files AGAIN to another server. That's what FTP2 is for. If you don't use FTP2, then the files in FTP can be deleted any time you like. But if you DO use FTP2 to create your redundant backup of the archive server, then you can delete the files in FTP2 any time you like instead, but don't delete anything from FTP as those are still "in transit" and have not yet landed in the FTP2 server.

Described it like ... three times. I hope it sticks. lol 8-)

AlSam wrote:Also, one of the points references the astguiclient.conf. Is this meant to have the server directly deposit recordings to the server configured in the file? Does the perl script depend on the configurations in the astguiclient.conf file?

/usr/share/astguiclient/AST_CRON_audio_3_ftp.pl Uses credentials from /etc/astguiclient.conf, yes. AST_CRON_audio_4_ftp2.pl does not.

Updated Wiki. 8-)
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20018
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: Differences in Perl Scripts

Postby AlSam » Tue May 19, 2020 1:37 pm

Thanks. That was very helpful.
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby AlSam » Tue Jun 09, 2020 1:36 pm

Tried running
Code: Select all
/usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl --help

which resulted in
Code: Select all
Can't locate Net/FTPSSL.pm in @INC (you may need to install the Net::FTPSSL module) (@INC contains: /usr/lib/perl5/site_perl/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.18.2 /usr/lib/perl5/vendor_perl/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.18.2 /usr/lib/perl5/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/5.18.2 /usr/lib/perl5/site_perl .) at /usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl line 392.
BEGIN failed--compilation aborted at /usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl line 392.

Took its suggestion (you may need to install the Net::FTPSSL module) and ran
Code: Select all
perl -MCPAN -e 'shell'

then
Code: Select all
install Net::FTPSSL

which resulted in
Code: Select all
....
Writing Makefile for Net::FTPSSL
Writing MYMETA.yml and MYMETA.json
Can't exec "make": No such file or directory at /usr/lib/perl5/5.18.2/CPAN/Distribution.pm line 2084.
  CLEACH/Net-FTPSSL-0.42.tar.gz
  make -- NOT OK
'YAML' not installed, will not store persistent state
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible
Failed during this command:
 CLEACH/Net-FTPSSL-0.42.tar.gz                : make NO

It references the make package. Do I download it from the OpenSUSE repositories or does it come from CPAN? I see it also states YAML not installed. Again, is this something that should be downloaded from OpenSUSE repositories or CPAN? Installation of the make package via OpenSUSE repository causes the addition of the site-config package. Will this addition have a negative impact on the server's proper operation?

Thanks
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Tue Jun 09, 2020 8:40 pm

On OpenSuSE, if you can install it through 'yast' then that should go much faster. The error you are getting is because you don't have the 'make' package installed, so cpan can't install anything. I'd suggest just trying to install the perl module though yast first.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Differences in Perl Scripts

Postby AlSam » Wed Jun 10, 2020 9:12 am

Thanks for the quick response. I was able to install the required module. Is there a newer version (current is 180616-2248) of this script (AST_CRON_audio_3_ftp_FTPSSL.pl) which includes the --ftp-validate option referenced under the Suggestions section in the Vicidial Recording Management Clarified document? Also, is there any other reference material regarding the options available for the script? I am not clear on a few of them like debug, debugX, test, transfer-limit, list-limit, and max-files.
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Wed Jun 10, 2020 9:32 am

I think we updated that script in February, so I'd recommend updating to the latest svn/trunk revision.

As for documentation, all there is is right in the script, usually in the comments at the top.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Differences in Perl Scripts

Postby williamconley » Thu Jun 11, 2020 11:09 am

Just because it's a good idea ... went ahead and moved this topic to "Support" instead of "General Discussion".
Vicidial Installation and Repair, plus Hosting and Colocation
Newest Product: Vicidial Agent Only Beep - Beta
http://www.PoundTeam.com # 352-269-0000 # +44(203) 769-2294
williamconley
 
Posts: 20018
Joined: Wed Oct 31, 2007 4:17 pm
Location: Davenport, FL (By Disney!)

Re: Differences in Perl Scripts

Postby AlSam » Tue Jun 30, 2020 7:04 am

How does it determine which recordings it will process each time it runs? I find that my FTP server has recordings from yesterday yet there are still recordings from over a year ago on the Vicidial server.
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Tue Jun 30, 2020 8:49 am

Each script will only look in one specific folder for recordings to process. The folder it looks in depends on the script and the option flags.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Differences in Perl Scripts

Postby AlSam » Tue Jun 30, 2020 8:56 am

I may not have provided enough detail in my question. I am using the --MP3 parameter and it, in fact, is processesing recordings from said folder. However, I don't understand the order in which recordings are selected to be processed from within the MP3 folder.
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Tue Jun 30, 2020 6:01 pm

The script uses the perl "readdir" function which really just calls the OS directory list function. It is not sorted by the script in any way, it uses whatever operating system order that the file list is supplied to it in. This might be the last modification date of the file or the creation date usually.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Differences in Perl Scripts

Postby AlSam » Thu Jul 02, 2020 6:47 am

OK. I'll look into it. Thanks.
I noticed that the --max-files switch does nothing. I reviewed the script and found the only time it is referenced is in the help section. I am running the 200205-1716 version of the script.
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby AlSam » Mon Jul 13, 2020 2:09 pm

Thanks for all the help. I was able to setup the transfer of files successfully. I am going to be transferring recordings from a different Vicidial platform which has multiple Asterisk servers. Is it advisable to dump all recordings from the different Asterisk servers into a single directory within the storage server? What are the chances of having recordings with the same name in more than one Asterisk server? Are there any best practices that should be followed in this circumstance?
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Tue Jul 14, 2020 6:26 am

Due to filesystem limitations on the destination archive server, we always recommend using the dated directories that the Vicidial FTP perl scripts use.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Differences in Perl Scripts

Postby AlSam » Tue Jul 14, 2020 7:10 am

I may not have explained myself properly. What I meant is that if it would be advisable to have the following structure or not.
Image
Or if recordings should go into the date folders regardless of the Asterisk server from whence they came.
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Tue Jul 14, 2020 7:42 am

We usually recommend putting each cluster's recordings in the same directory structure. So if your Astersk1, 2, 3... are all on the same VICIdial cluster, we would recommend sending them all to the same set of dated directories.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida

Re: Differences in Perl Scripts

Postby AlSam » Tue Jul 14, 2020 7:45 am

So there's no chance there would be two different recordings with the same name on different Asterisk servers (same cluster) which might end up having one overwrite the other?
ViciBox v.7.0.4-170113 | VERSION: 2.14-697a BUILD: 190121-2019 | Asterisk 11.25.1-vici | All-In-One | No additional software
AlSam
 
Posts: 125
Joined: Thu Sep 02, 2010 1:17 pm

Re: Differences in Perl Scripts

Postby mflorell » Tue Jul 14, 2020 9:15 am

I don't know how you are currently naming your recordings. If you use something like "FULLDATE_CUSTPHONE_AGENT" then yes there is no chance that recordings will have duplicate names.
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida


Return to Support

Who is online

Users browsing this forum: No registered users and 79 guests