There's always been the intention of doing a documented end to end deployment of the Cisco UC Stack in a lab, and the end of year break seems like a good enough time than any.
Concept is to cover the installation of a standard office UC backend, including integration into a Microsoft Environment (AD / Exchange etc).
The UC Environment will include the following systems:
- CallManager 10.5
- Unity Connection 10.5
- IM and Presence 10.5
- UCCX 10.6
- VCS Expressway for MRA.
Our Microsoft Environment will consist of the following:
- Windows 2012 R2 Domain Controller.
- Exchange 2010 for EMail.
Other services we'll deploy:
- A Linux box for UC Backups
- A CDR Collection System based on Kibana and Elasticsearch.
I'll be using trial / demo licenses for all the above systems where necessary.
Lets begin...
Saturday, 26 December 2015
Sunday, 11 August 2013
Syslog not logging events though the server is receiving syslog packets
Recent issue with a new centos syslog-ng box which was configured as a syslog destination for a number of devices, but we weren't seeing logs being generated.
- Re-ran syslog, still no traffic
# Drop packets that look like they are spoofed. Default = 1.
# 0 = don't check;
# 1 = drop packets that as sourced at a directly connected interface
# but were input from another interface;
# 2 = drop any packets that look spoofed.
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.bond1.rp_filter = 0
Things we tested
- Checked syslog-ng.conf - all good.
- Sent test syslogs via a command line syslog generator - all good.
- Ran tcpdump on the interface receiving syslog via 'tcpdump -A -i bond1 udp port 514' and could see traffic for syslog successfully.- Re-ran syslog, still no traffic
Solution
- Issue was resolved after adding the following to /etc/sysctl.conf and running sysctl -p.# Drop packets that look like they are spoofed. Default = 1.
# 0 = don't check;
# 1 = drop packets that as sourced at a directly connected interface
# but were input from another interface;
# 2 = drop any packets that look spoofed.
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.bond1.rp_filter = 0
Tuesday, 30 April 2013
tac_plus install on Ubuntu - Fixing the crypt error message during build
I'm going to add a couple of posts later on in regards to the process of installing tac_plus on Ubuntu but I thought I'd pre-empt the full post with a quick fix if you're trying to install and hit the following error:
authen.cpp: In member function 'int tauthen::changepass(int)':
authen.cpp:1057:52: error: 'crypt' was not declared in this scope
I initially added #define <crypt.h>' to authen.cpp but was finding that the passwords being generated by the function wasn't matching what the tac_plus web front end was generating.
Ended up tracing the fix back to adding #include <unistd.h> to the define section in tac_plus.h.
Testing following a successful build showed passwords being generated successfully by both the tac_plus program and the php web front end.
Hope this shortcuts the build process for people.
authen.cpp: In member function 'int tauthen::changepass(int)':
authen.cpp:1057:52: error: 'crypt' was not declared in this scope
I initially added #define <crypt.h>' to authen.cpp but was finding that the passwords being generated by the function wasn't matching what the tac_plus web front end was generating.
Ended up tracing the fix back to adding #include <unistd.h> to the define section in tac_plus.h.
Testing following a successful build showed passwords being generated successfully by both the tac_plus program and the php web front end.
Hope this shortcuts the build process for people.
Sunday, 21 April 2013
tac_plus Deep-Dive #1 - Command Authorization and db tables
First in a few posts I hope to get up to here covering tac_plus (using the version from networkforums.com) which I've been looking into over the weekend. All the work I've been doing so far has been in relation to cisco devices
Primary interest has been how we get command authorization working on our network equipment. Having someone 'reload' a core device in the middle of the day is only something we want to happen once...
So as a quick overview of how some of the values in the database come together:
Primary interest has been how we get command authorization working on our network equipment. Having someone 'reload' a core device in the middle of the day is only something we want to happen once...
So as a quick overview of how some of the values in the database come together:
- The column 'cmd_dflt' in the user table marks whether an unlisted command will be permitted or denied if it doesn't exist explicitly in the node table. 0 is the default value and means non explicit commands will be denied. Set to 1 and commands not explicitly included will be allowed.
- The node table contains the command sets applied to users/groups.
- Entries with the 'service' column set to 52 are returned to the device on exec (eg once logged in). It's here you can return the priviledge level. (value1 would be 'priv-lvl' and value is '1' or '15'.
- Entries in relation to command authorization will have a service value of 56. Type value set to 57 means permitted, type value set to 58 is denied.
Friday, 16 November 2012
Tracing an Outbound UDP Flood
Had a client running on our virtual machine infrastructure who we saw starting to generate really high outbound traffic. This in turn would affect other infrastructure as it attempted to cope with 1Gb of outbound traffic.
After some basic analysis we determined the outbound traffic was UDP, originating from the clients server and heading back to the internet (in this case a host in Brazil). After doing an initial packet capture from the host we were seeing that the UDP Traffic data was all 'X' characters. Google searches for anything that may generate this kind of traffic was inconslusive, What I did find however in searching for outbound UDP flooding was a blog entry that discussed the use of a PHP script (http://scott.cm/outbound-dos/) which pointed me in the direction of looking for a script kiddie script.
In going through the packet capture again, I found that we'd actually (luckily) captured the initial http request from the external party who was kicking off the flood (it was luck becuase I was originally only kicking off the capture when I saw the flooding begin on our real time graphing).
What we saw:
http://www.host.sample/403.php?act=phptools&host=x.x.x.x&time=460&port=80
Given that x.x.x.x was then the host that we were flooding UDP out to, kind of a coincidence to see the ip was in the request.
Taking a look on the server then yielded the 403.php file. It was pretty simple, and ultimately did the following:
$fp = fsockopen('udp://'.$host, $rand, $errno, $errstr, 5);
If you google this string you'll find it around in a number of scripts. Additional to 403.php we found 404.php which appears to be a gzencoded shell app which once running on the server allows you to do a number of items. Additional to this was 'a.php' which looks like it was a vulnerability scanner.
Ultimately we cleaned off the scripts, alerted the client of the issue and moved on with real work.
After some basic analysis we determined the outbound traffic was UDP, originating from the clients server and heading back to the internet (in this case a host in Brazil). After doing an initial packet capture from the host we were seeing that the UDP Traffic data was all 'X' characters. Google searches for anything that may generate this kind of traffic was inconslusive, What I did find however in searching for outbound UDP flooding was a blog entry that discussed the use of a PHP script (http://scott.cm/outbound-dos/) which pointed me in the direction of looking for a script kiddie script.
In going through the packet capture again, I found that we'd actually (luckily) captured the initial http request from the external party who was kicking off the flood (it was luck becuase I was originally only kicking off the capture when I saw the flooding begin on our real time graphing).
What we saw:
http://www.host.sample/403.php?act=phptools&host=x.x.x.x&time=460&port=80
Given that x.x.x.x was then the host that we were flooding UDP out to, kind of a coincidence to see the ip was in the request.
Taking a look on the server then yielded the 403.php file. It was pretty simple, and ultimately did the following:
$fp = fsockopen('udp://'.$host, $rand, $errno, $errstr, 5);
If you google this string you'll find it around in a number of scripts. Additional to 403.php we found 404.php which appears to be a gzencoded shell app which once running on the server allows you to do a number of items. Additional to this was 'a.php' which looks like it was a vulnerability scanner.
Ultimately we cleaned off the scripts, alerted the client of the issue and moved on with real work.
Sunday, 14 October 2012
CUCM SQL Query for Users set to reset passwords
The following query run from the SSH CLI will give you a list of users whose accounts are set to enforce a password change:
run sql ccm select u.userid, c.credmustchange from enduser as u inner join credential as c on c.fkenduser = u.pkid where tkcredential = 3 and c.credmustchange = 't'
run sql ccm select u.userid, c.credmustchange from enduser as u inner join credential as c on c.fkenduser = u.pkid where tkcredential = 3 and c.credmustchange = 't'
Tuesday, 9 October 2012
Debugging VoIP on Cisco Routers
Debugging VoIP on Cisco Routers
Just a basic collection of debugs that may be useful to run when troubleshooting VoIP issues on Cisco Routers.As per all debugs, enabling debugs on routers can cause CPU overload causing the unit to become unresponsive. Enable debugs carefully on busy systems.
General Dialpeer / Call Flow
These debugs will give you a call flow including pattern matching on dialpeer's and which was the ingress and egress dial peers:- debug voip dialpeer inout
- debug voip ccapi inout
H323 Useful Debugs
For a deeper look at H323 setups and call flows:- debug cch323 h225
- debug h245 asn1
- debug cch323 session
- debug cch323 error
SIP useful debugs
- debug ccsip messages
- debug ccsip ?
Other Notes
- If you're running a newer version of IOS make sure that your Voice service config is allowing traffic through from your remote ip's. Saw an issue where H323 wasn't showing any debug traffic when a call was being sent through. Issue was an with voice service not accepting connections because the remote ip wasn't in the list of allowed IP's.
- Dialpeer matching logic useful doc: http://www.cisco.com/en/US/docs/ios/12_3/vvf_c/voice_troubleshooting/old/vts_appa.html
- An awesome debug doc from cisco: https://supportforums.cisco.com/docs/DOC-16310
Sunday, 9 September 2012
Opsview 64bit Agent on Ubuntu 12.04 (Precise)
Had some real issues finding the agent and getting it easily installed on a new install of Ubuntu 12.04 (Precise) I'd stood up as a backup target for UC Backups.
Steps:
Steps:
- Edit /etc/apt/sources.list and add 'deb http://downloads.opsview.com/opsview-core/latest/apt precise main' at the end of the file.
- run 'apt-get update'.
- Run 'apt-get install opsview-agent'.
- Should be good to go.
Wednesday, 27 June 2012
CUCM SQL Queries for users with IPCC Extension and Primary Extension asigned
Run the following in a CLI session on your CUCM Server to generate a list of users with IPCC extensions assigned in their user profile:
run sql select e.userid, n.dnorpattern from enduser as e, numplan as n, endusernumplanmap as x where (e.pkid = x.fkenduser and x.fknumplan = n.pkid and tkdnusage=2)
Run the following query for users with primary extensions assigned:
run sql select e.userid, n.dnorpattern from enduser as e, numplan as n, endusernumplanmap as x where (e.pkid = x.fkenduser and x.fknumplan = n.pkid and tkdnusage=1)
run sql select e.userid, n.dnorpattern from enduser as e, numplan as n, endusernumplanmap as x where (e.pkid = x.fkenduser and x.fknumplan = n.pkid and tkdnusage=2)
Run the following query for users with primary extensions assigned:
run sql select e.userid, n.dnorpattern from enduser as e, numplan as n, endusernumplanmap as x where (e.pkid = x.fkenduser and x.fknumplan = n.pkid and tkdnusage=1)
Wednesday, 6 June 2012
Getting Cisco Phone Designer working on x64 Windows 7
Have to give credit to the guys on the cisco forums for this tip. Cisco Phone designer doesn't support x64 Windows 7. To get it to run:
- Open up command prompt as administrator and type this command:
- C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\Ldr64.exe SetWoW
- Now you should be able to run apps that use only the .NET 32 bit CLR
- To revert back to the default 64 bit framework run
- C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\Ldr64.exe Set64
Issues with SCCP Phone Registration through a Juniper Netscreen
Deployed an NG5-GT out to a remote site to get a phone up and running behind it via ipsec VPN. Tested everything locally before deployment and phone registration was all good. Once shipped to site, the phone wouldn't register. We'd see attempts to register in CUCM logs but we'd see it marked as 'rejected' and log analysis would show a Transient Connection and Registration Failure.
Ended up switching off the ALG for SCCP (Security > ALG) and the phone registered fine.
Just in case anyone ever has the same issue.
Ended up switching off the ALG for SCCP (Security > ALG) and the phone registered fine.
Just in case anyone ever has the same issue.
Tuesday, 5 June 2012
To login and logout of Extension Mobility via a URL:
https://x.x.x.x:8443/emapp/EMAppServlet?device=SEP6400F1111111&&userid=userid&seq=PIN
https://x.x.x.x:8443/emapp/EMAppServlet?device=SEP6400F1111111&doLogout=true
For when your provisioning app hits the craps and you need to do it remotely :).
https://x.x.x.x:8443/emapp/EMAppServlet?device=SEP6400F1111111&&userid=userid&seq=PIN
https://x.x.x.x:8443/emapp/EMAppServlet?device=SEP6400F1111111&doLogout=true
For when your provisioning app hits the craps and you need to do it remotely :).
Monday, 4 June 2012
9971 Phones need at least one DN assigned to register
Yet another knocking head against the wall issue. We've been running phones without extensions and EM enabled so that users could log in to get their correct Lines. Came to put a 9971 into production and the phone wouldn't register. Status message on the screen was simply 'Phone not registered'.
Went through all the debugging steps. Status messages on the phone showed no obvious issues. Trust list was being updated successfully. TFTP load was happening correctly. Uploaded new firmware to the cluster and the phones pulled it down correctly. Basic network traces showed the phone booting, doing 3 dns requests, pulling config down via HTTP from the TFTP server (a few requests) and then heading off to the NTP server. We never saw any SIP requestes (port 5060).
The only thing of real note was this line in the status logs:
7301 NOT 13:25:22.214160 CVM-cfg.line1NotConfigured: [thread=configmgr MQThread][class=cip.cfg.j] sipMinCfgCheck():Line 1 not configured..
After spending a couple of hours on the phone with TAC, managed to narrow the issue down to the phone having no DN assigned. Assigned a DN and the phone registers fine.
If you hit this issue and are using EM, don't forget to update the EM logout profile on the phone, it takes precedence over the physical phone profile. If you only update the physical device record and not the Device Profile it won't register.
Hope this saves someone some time.
Went through all the debugging steps. Status messages on the phone showed no obvious issues. Trust list was being updated successfully. TFTP load was happening correctly. Uploaded new firmware to the cluster and the phones pulled it down correctly. Basic network traces showed the phone booting, doing 3 dns requests, pulling config down via HTTP from the TFTP server (a few requests) and then heading off to the NTP server. We never saw any SIP requestes (port 5060).
The only thing of real note was this line in the status logs:
7301 NOT 13:25:22.214160 CVM-cfg.line1NotConfigured: [thread=configmgr MQThread][class=cip.cfg.j] sipMinCfgCheck():Line 1 not configured..
After spending a couple of hours on the phone with TAC, managed to narrow the issue down to the phone having no DN assigned. Assigned a DN and the phone registers fine.
If you hit this issue and are using EM, don't forget to update the EM logout profile on the phone, it takes precedence over the physical phone profile. If you only update the physical device record and not the Device Profile it won't register.
Hope this saves someone some time.
Tuesday, 29 May 2012
Ending a Call via URL with CUCM Webdialer
Had a customer who was wanting to integrate their custom CRM application with CUCM. Easiest way to do this was via the Webdialer. Their current setup was to use a modem placed in front of an analogue phone which was then controlled by the CRM app via a dialer string.
To use the webdialer process we can fire off the the URL 'https://x.x.x.x:8443/webdialer/Webdialer?destination=0123456789' from within the application to cause a call to kick off. (Note that you need to have logged the end user into webdialer at least once previously so that settings and login details are saved).
Next request was to be able to hang the call up via a button in the app so that the call center agent never has to physically touch the phone (or softphone) to hang up calls.
After mucking around with Fiddler, seems that you can use the following url string to accomplish the hangup: 'https://x.x.x.x:8443/webdialer/Webdialer?cmd=doEndCall'. This will hang up the call (assuming there is one in progress) on the device managed by the webdialer setup. If no call is in progress then nothing untoward happens.
Now to get the CRM app updated with this code and we'll see how it goes.
To use the webdialer process we can fire off the the URL 'https://x.x.x.x:8443/webdialer/Webdialer?destination=0123456789' from within the application to cause a call to kick off. (Note that you need to have logged the end user into webdialer at least once previously so that settings and login details are saved).
Next request was to be able to hang the call up via a button in the app so that the call center agent never has to physically touch the phone (or softphone) to hang up calls.
After mucking around with Fiddler, seems that you can use the following url string to accomplish the hangup: 'https://x.x.x.x:8443/webdialer/Webdialer?cmd=doEndCall'. This will hang up the call (assuming there is one in progress) on the device managed by the webdialer setup. If no call is in progress then nothing untoward happens.
Now to get the CRM app updated with this code and we'll see how it goes.
Tuesday, 23 August 2011
Troubleshooting Last=Phone-Reg-Rej messages after IOS Upgrade
Hit an issue when I was going through an upgrade of the IOS on a 2951 router running CME for a client. Completed the upgrade to a version of IOS with CME 8.6 (was previously running CME 8.0) and I found that after phones had upgraded (took a few hours after all the phones kicked the absolute out of the tftp service on the router), I was seeing the following in the Phone debug displays along the lines of:
12:05:36a 24: Name=SEP90000000473C Load= SCCP45.9-1-1SR1S Last=Phone-Reg-Rej
I couldn't initially see what was going wrong. I ended up running a 'debug ephone register' and it was returning the fact that the max number of ephones was reached. Checking the 'telephony-service' settings showed 'max ephones 165'.
What ended up being the issue was that I needed to accept the license on the system. I had to do a couple of things, first was run the command 'license accept end user agreement'. This didn't actually fix up the telephony service, I had to go into 'telephony-service' and execute 'max ephone 164'. This change then prompted for the agreement of the EULA. Once completed, the phones all registered.
Also saw issues with the cme-srst license but I'll detail that in a different post.
12:05:36a 24: Name=SEP90000000473C Load= SCCP45.9-1-1SR1S Last=Phone-Reg-Rej
I couldn't initially see what was going wrong. I ended up running a 'debug ephone register' and it was returning the fact that the max number of ephones was reached. Checking the 'telephony-service' settings showed 'max ephones 165'.
What ended up being the issue was that I needed to accept the license on the system. I had to do a couple of things, first was run the command 'license accept end user agreement'. This didn't actually fix up the telephony service, I had to go into 'telephony-service' and execute 'max ephone 164'. This change then prompted for the agreement of the EULA. Once completed, the phones all registered.
Also saw issues with the cme-srst license but I'll detail that in a different post.
Subscribe to:
Posts (Atom)