Fastest way to upload multiple lead files?

Any and all non-support discussions

Moderators: gerski, enjay, williamconley, Op3r, Staydog, gardo, mflorell, MJCoate, mcargile, Kumba, Michael_N

Fastest way to upload multiple lead files?

Postby xodiacx » Wed Jul 06, 2016 1:46 am

What's the best way to upload multiple lead files? Each lead file is save as text tab delimited and contains 25,000 data each. And the process is "check for duplicates by phone in all campaign lists".

Vicidial v 7.0.1 VERSION: 2.12-541a BUILD: 160306-1053 asterisk 11
xodiacx
 
Posts: 29
Joined: Thu Sep 05, 2013 3:38 pm

Re: Fastest way to upload multiple lead files?

Postby mflorell » Wed Jul 06, 2016 7:37 am

The fastest way is to set up an FTP account and run the command-line lead loader on your dialer from SSH.


# /usr/share/astguiclient/VICIDIAL_IN_new_leads_file.pl --help
allowed run time options:
[--quiet] = quiet
[--test] = test
[--version] = version
[--forcegmt] = forces gmt value of column after comments column
[--debug] = debug output
[--format=standard] = ability to define a format, standard is default, formats allowed shown in examples
[--forcelistid=1234] = overrides the listID given in the file with the 1234
[--forcelistfilename] = overrides the listID using last number in filename: (XYZ_1234.txt = list ID 1234)
[--forcephonecode=44] = overrides the phone_code given in the lead with the 44
[--file-prefix-filter=WXYZ] = will only process lead files that begin with the characters you define
[--new-list-for-each-file] = creates a new list for each file loaded, listID = YYYYMMDDX where X is incremented
[--new-listid-prefix=X] = prefix for listID when creating new lists, must be only numbers, and 4 or less digits
[--new-listname-prefix=X] = prefix for list name when creating new lists, will be followed by filename
[--new-list-campaign=X] = campaign that the new list will be assigned to
[--new-list-active=X] = Y or N, if list is to be set as active when created, default N
[--new-list-reset-times=X] = reset times(4 digits each dash separated) default is blank
[--new-list-tz-setting=X] = COUNTRY_AND_AREA_CODE|POSTAL_CODE|NANPA_PREFIX|OWNER_TIME_ZONE_CODE default COUNTRY_AND_AREA_CODE
[--USACAN-prefix-check] = check for the 4th digit 2-9, USA and Canada validation
[--USACAN-areacode-check] = check for the valid phone code 1 areacodes, USA and Canada validation
[--NANPA-ac-prefix-check] = check for the valid areacode and prefix from the NANPA prefix database, if loaded
[--duplicate-check] = checks for the same phone number in the same list id before inserting lead
[--duplicate-campaign-check] = checks for the same phone number in the same campaign before inserting lead
[--duplicate-system-check] = checks for the same phone number in the entire system before inserting lead
[--duplicate-tap-list-check] = checks for the same title/alt-number in the same list ID before inserting lead
[--duplicate-tap-system-check] = checks for the same title/alt-number in the entire system before inserting lead
[--duplicate-tnm-delete] = checks for same phone and title in same list then deletes original if title is different
[--dob-mmddyyy] = checks for slashes in the date of birth field and formats from MM/DD/YYYY to MySQL
[--dob-ddmmyyy] = checks for slashes in the date of birth field and formats from DD/MM/YYYY to MySQL
[--postal-code-gmt] = checks for the time zone based on the postal code given where available
[--time-zone-code-gmt] = checks for the time zone based on the owner field time zone code given where available
[--nanpa-gmt] = checks for the time zone based on the NANPA prefix database, if loaded
[--ftp-pull] = grabs lead files from a remote FTP server, uses REPORTS FTP login information
[--ftp-dir=leads_in] = remote FTP server directory to grab files from, should have a DONE sub-directory
[--email-list=test.com:test2.com] = send email results for each file to these addresses
[--email-sender=vicidial] = sender for the email results
[-h] = this help screen


This script takes in lead files in the following order when they are placed in the /usr/share/astguiclient/LEADS_IN directory to be imported into the vicidial_list table (examples):

standard:
vendor_lead_code|source_id|list_id|phone_code|phone_number|title|first_name|middle|last_name|address1|address2|address3|city|state|province|postal_code|country|gender|date_of_birth|alt_phone|email|security_phrase|COMMENTS|called_count|status|entry_date|multi-alt-entries
3857822|31022|105|01144|1625551212|MRS|B||BURTON|249 MUNDON ROAD|MALDON|ESSEX||||CM9 6PW|UK||||||COMMENTS|2|B|2007-08-09 00:00:00|7275551212_1_work!7275551213_61_sister house!7275551214_44_neighbor

stdrankowner:
vendor_lead_code|source_id|list_id|phone_code|phone_number|title|first_name|middle|last_name|address1|address2|address3|city|state|province|postal_code|country|gender|date_of_birth|alt_phone|email|security_phrase|COMMENTS|called_count|status|entry_date|rank|owner|multi-alt-entries
3857822|31022|105|01144|1625551212|MRS|B||BURTON|249 MUNDON ROAD|MALDON|ESSEX||||CM9 6PW|UK||||||COMMENTS|2|B|2007-08-09 00:00:00|99|6666|7275551212_1_work!7275551213_61_sister house!7275551214_44_neighbor

minicsv:
address1,city,name,phone_number,state,postal_code
"105 Fifth St","Steinhatchee","Frank Smith","3525556601","FL","32359"

minicsv02:
name,address1,city,state,postal_code,phone_number
"Adam Smith","123 Fourth St","Englishtown","NS","B0C1H0","902-555-1212"

fixed254:
fixed width(without the quotes)
"9185551212ROSE SMITHS 155 TIGER MOUNTAIN RD. RR 1 BOX 107 HENRYETTA OK74437-941DEMG 226555 0 "

sctab08:
STATE NAME PHONE FAX ADDRESS ADDRESS2 CITY ZIP
AL Tom Wilson 205 5551212 1689 15th Ave BESSEMER 35020

sccsv11:
STATE,ID,NAME,ADDRESS,ADDRESS2,CITY,COUNTY,PHONE,FAX,ZIP,COMMENTS
AL,16700,Fairfax Retirement Center (Crowder House),1324 25th St,,FAIRFAX,JEFFERSON,(205) 555-0508,,35733,Comments

dccsv10:
VENDOR_ID,FIRST_NAME,LAST_NAME,PHONE_1,PHONE_2,PHONE_3,PHONE_4,PHONE_5,PHONE_6,PHONE_7
"100998","ANGELA ","SMITH ","3145551212","3145551213","3145551214","0","3145551215","3145551216","0",

pipe30tz:
TEST01||09292011|1|5125554727||Mike||Frank|||||||||||||||||C|2145559922|TESTSURVEY|TESTSURVEY|111
vote17csv:
Voter ID,County Code,Name Last,Gender,Race,Birth Date,Party Affiliation,Precinct,Congressional District,House District,Senate District,County Commission District,School Board District,Age Bracket,_2010_GEN,_2012_PRI_A_Date,Phone_10digit
"110306742","DAD","Da Silva Ramos","F","5","06/12/1950","DEM","100","23","0","0","0","0","4","A","","0033559433"

t2r13csv:
first_name,last_name,address3,comments1,province,phone_number,address1,city,state,postal_code,comments2,comments3,comments4
Bob,Smith,"Bob, Inc.",lead,president,3125551212,1234 main st,Chicago,IL,60987,good,1235,T2R

twotab:
UniqueID,PhoneNumber
110306742,3125556666

dccsv43, dccsvref51, dccsv52 and dccsvref52:
---format too confusing to list in the help screen---
mflorell
Site Admin
 
Posts: 18335
Joined: Wed Jun 07, 2006 2:45 pm
Location: Florida


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 44 guests