Calls dropping like flies

Gawd, I dunno what happened today. Maybe I got up on a wrong foot or something, but I've been seeing 20% dropped calls today. This is despite constantly seeing agents that are ready. I would literally watch the timeonVDAD screen and see DROPPED number jump up by 5-6 even though this whole time there were agents available to receive the call.
My system is: Slackware 12.0
Asterisk 1.12.24
AGC 2.0.4 (which was given out at the recent boot camp)
I followed scratch install.
Average load is usually below 1. Haven't seen it go above 2 since a I moved the database to another server a few days ago.
I looked at the output of ASTVDauto and noticed something that might not look good:
2008-05-03 00:20:41|US 207.90.4.159: agents: 7 dial_level: 2.5|
2008-05-03 00:20:41|US 207.90.4.159: Calls to place: 1 (34 - 33) 34 |
2008-05-03 00:20:41|CAMPAIGN DIFFERENTIAL: 34.7 9.05 (9.05 - 0)|
2008-05-03 00:20:41|LOCAL TRUNK SHORTAGE: 0|0 (34 - 40)|
2008-05-03 00:20:41|US 207.90.4.159: CALLING|
|UPDATE vicidial_hopper set status='QUEUE', user='VDAD_207.90.4.159' where campaign_id='US' and status='READY' order by priority desc,hopper_id LIMIT 1|
hopper rows updated to QUEUE: |1|
|SELECT lead_id,alt_dial FROM vicidial_hopper where campaign_id='US' and status='QUEUE' and user='VDAD_207.90.4.159' order by priority desc,hopper_id LIMIT 1|
|UPDATE vicidial_hopper set status='INCALL' where lead_id='922717'|
hopper row updated to INCALL: |1|922717|
2008-05-03 00:20:41|| number call dialed|US|V0503002041000922717|INSERT INTO vicidial_manager values('','','2008-05-03 00:20:41','NEW','N','207.90.4.159','','Originate','V0503002041000922717','Exten: 8368','Context: default','Channel: Local/15106582972@default','Priority: 1','Callerid: "V0503002041000922717" <0000000000>','Timeout: 20000','','','','')|-7.00|NONE||
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:41|| lagged call vla agent PAUSED 0E0|20080503002011|20080503002031|20080503002041||
2008-05-03 00:20:41|| lagged call vac agent DELETED 0E0|2008-05-03 00:18:41||
2008-05-03 00:20:41|| lastcalldate UPDATED 1|'80209'||
2008-05-03 00:20:41|| logindate UPDATED 1|'US'||
loop counter: |5699623|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:43|LIVE AGENTS LOGGED IN: 31 ACTIVE CALLS: 34|
2008-05-03 00:20:43|OLD TRUNK SHORTS CLEARED: 9 |'','US'||
2008-05-03 00:20:43|US 207.90.4.159: agents: 4 dial_level: 2.5|
2008-05-03 00:20:43|US 207.90.4.159: Calls to place: -4 (30 - 34) 34 |
2008-05-03 00:20:43|CAMPAIGN DIFFERENTIAL: 34.65 8.75 (8.75 - 0)|
2008-05-03 00:20:43|LOCAL TRUNK SHORTAGE: 0|0 (30 - 40)|
2008-05-03 00:20:43|US 207.90.4.159: CALLING|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:43|| dead call vac deleted|165679|869967|V0503002021000869967|1209788441|1|||
2008-05-03 00:20:43|| dead NA call added to log 1209788421.1428230|869967|3237178970|SENT|NA|1||
2008-05-03 00:20:43|| dead call vac lead marked NA|869967|3237178970|SENT||
2008-05-03 00:20:43|| dead call vla agent PAUSED 0E0|869967|3237178970|SENT||
2008-05-03 00:20:43|| lagged call vla agent PAUSED 0E0|20080503002013|20080503002033|20080503002043||
2008-05-03 00:20:43|| lagged call vac agent DELETED 0E0|2008-05-03 00:18:43||
2008-05-03 00:20:43|| logindate UPDATED 1|'US'||
loop counter: |5699622|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:46|LIVE AGENTS LOGGED IN: 32 ACTIVE CALLS: 31|
2008-05-03 00:20:46|OLD TRUNK SHORTS CLEARED: 9 |'','US'||
2008-05-03 00:20:46|US 207.90.4.159: agents: 4 dial_level: 2.5|
2008-05-03 00:20:46|US 207.90.4.159: Calls to place: -1 (30 - 31) 31 |
2008-05-03 00:20:46|CAMPAIGN DIFFERENTIAL: 34.6 8.5 (8.5 - 0)|
2008-05-03 00:20:46|LOCAL TRUNK SHORTAGE: 0|0 (30 - 40)|
2008-05-03 00:20:46|US 207.90.4.159: CALLING|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:46|| lagged call vla agent PAUSED 0E0|20080503002016|20080503002036|20080503002046||
2008-05-03 00:20:46|| lagged call vac agent DELETED 0E0|2008-05-03 00:18:46||
2008-05-03 00:20:46|| logindate UPDATED 1|'US'||
I would appreciate any pointers on what I can do to fix it.
-TJ
My system is: Slackware 12.0
Asterisk 1.12.24
AGC 2.0.4 (which was given out at the recent boot camp)
I followed scratch install.
Average load is usually below 1. Haven't seen it go above 2 since a I moved the database to another server a few days ago.
I looked at the output of ASTVDauto and noticed something that might not look good:
2008-05-03 00:20:41|US 207.90.4.159: agents: 7 dial_level: 2.5|
2008-05-03 00:20:41|US 207.90.4.159: Calls to place: 1 (34 - 33) 34 |
2008-05-03 00:20:41|CAMPAIGN DIFFERENTIAL: 34.7 9.05 (9.05 - 0)|
2008-05-03 00:20:41|LOCAL TRUNK SHORTAGE: 0|0 (34 - 40)|
2008-05-03 00:20:41|US 207.90.4.159: CALLING|
|UPDATE vicidial_hopper set status='QUEUE', user='VDAD_207.90.4.159' where campaign_id='US' and status='READY' order by priority desc,hopper_id LIMIT 1|
hopper rows updated to QUEUE: |1|
|SELECT lead_id,alt_dial FROM vicidial_hopper where campaign_id='US' and status='QUEUE' and user='VDAD_207.90.4.159' order by priority desc,hopper_id LIMIT 1|
|UPDATE vicidial_hopper set status='INCALL' where lead_id='922717'|
hopper row updated to INCALL: |1|922717|
2008-05-03 00:20:41|| number call dialed|US|V0503002041000922717|INSERT INTO vicidial_manager values('','','2008-05-03 00:20:41','NEW','N','207.90.4.159','','Originate','V0503002041000922717','Exten: 8368','Context: default','Channel: Local/15106582972@default','Priority: 1','Callerid: "V0503002041000922717" <0000000000>','Timeout: 20000','','','','')|-7.00|NONE||
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:41|| lagged call vla agent PAUSED 0E0|20080503002011|20080503002031|20080503002041||
2008-05-03 00:20:41|| lagged call vac agent DELETED 0E0|2008-05-03 00:18:41||
2008-05-03 00:20:41|| lastcalldate UPDATED 1|'80209'||
2008-05-03 00:20:41|| logindate UPDATED 1|'US'||
loop counter: |5699623|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:43|LIVE AGENTS LOGGED IN: 31 ACTIVE CALLS: 34|
2008-05-03 00:20:43|OLD TRUNK SHORTS CLEARED: 9 |'','US'||
2008-05-03 00:20:43|US 207.90.4.159: agents: 4 dial_level: 2.5|
2008-05-03 00:20:43|US 207.90.4.159: Calls to place: -4 (30 - 34) 34 |
2008-05-03 00:20:43|CAMPAIGN DIFFERENTIAL: 34.65 8.75 (8.75 - 0)|
2008-05-03 00:20:43|LOCAL TRUNK SHORTAGE: 0|0 (30 - 40)|
2008-05-03 00:20:43|US 207.90.4.159: CALLING|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:43|| dead call vac deleted|165679|869967|V0503002021000869967|1209788441|1|||
2008-05-03 00:20:43|| dead NA call added to log 1209788421.1428230|869967|3237178970|SENT|NA|1||
2008-05-03 00:20:43|| dead call vac lead marked NA|869967|3237178970|SENT||
2008-05-03 00:20:43|| dead call vla agent PAUSED 0E0|869967|3237178970|SENT||
2008-05-03 00:20:43|| lagged call vla agent PAUSED 0E0|20080503002013|20080503002033|20080503002043||
2008-05-03 00:20:43|| lagged call vac agent DELETED 0E0|2008-05-03 00:18:43||
2008-05-03 00:20:43|| logindate UPDATED 1|'US'||
loop counter: |5699622|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:46|LIVE AGENTS LOGGED IN: 32 ACTIVE CALLS: 31|
2008-05-03 00:20:46|OLD TRUNK SHORTS CLEARED: 9 |'','US'||
2008-05-03 00:20:46|US 207.90.4.159: agents: 4 dial_level: 2.5|
2008-05-03 00:20:46|US 207.90.4.159: Calls to place: -1 (30 - 31) 31 |
2008-05-03 00:20:46|CAMPAIGN DIFFERENTIAL: 34.6 8.5 (8.5 - 0)|
2008-05-03 00:20:46|LOCAL TRUNK SHORTAGE: 0|0 (30 - 40)|
2008-05-03 00:20:46|US 207.90.4.159: CALLING|
TIME DEBUG: -5|-4|1| GMT: 04:20
2008-05-03 00:20:46|| lagged call vla agent PAUSED 0E0|20080503002016|20080503002036|20080503002046||
2008-05-03 00:20:46|| lagged call vac agent DELETED 0E0|2008-05-03 00:18:46||
2008-05-03 00:20:46|| logindate UPDATED 1|'US'||
I would appreciate any pointers on what I can do to fix it.
-TJ