Paul's Internet Landfill/ 2015/ Provisioning Cisco 7941 Phones in FusionPBX using TFTP

Provisioning Cisco 7941 Phones in FusionPBX using TFTP

For a popular project that has been around for a decade, good documentation on FreeSWITCH in general and FusionPBX in particular seems pretty sparse. I have been evaluating FusionPBX for a project at work, and it has taken me hours and hours to figure out how to do basic things. In order to get phone provisioning working, I have resorted to source-diving, which is a pretty good indication that the topic is bloggable. (Mind you, source-diving in FusionPBX is not that painful. The code is structured well, and for all its limitations PHP is easy to understand and easy to tinker with on the fly.)

FusionPBX comes with the ability to provision VoIP handsets: specifying their extension info, names, and so on. Being able to do this via the web interface is convenient: FusionPBX will populate your configuration files with usernames and passwords for your extensions, so you don't have to type (and retype) them manually. That means you are more likely to use complicated passwords, which I wholeheartedly support. It also means you don't have to remember to copy and paste (and then fiddle with) config files.

In this example I was trying to configure some donated Cisco 7941 phones to work with FusionPBX. Unlike the similarly-named Cisco 7940 phones, these models are based on Java and use XML-based configuration files, and the config files are structured differently between the two models. The 7941 phones require a TFTP server to get their firmware files and configuration.

I got really confused about how the TFTP server and FusionPBX work together to provision phones, so here is an overview:

Here are some brief notes betraying all the places I got stuck. For posterity, I am working with:

I will assume you have FusionPBX installed and working with a softphone already, to the extent you can make and receive calls using an external VoIP provider.

Setting up the TFTP Server

First, aptitude install tftpd-hpa . I am pretty sure FusionPBX wants the TFTP server and FusionPBX to be running on the same server, although you might be able to use nfs mounts or sshfs or something to fool FusionPBX into thinking it is writing on a local filesystem.

In /etc/default/tftpd-hpa, set the following:

TFTP_USERNAME="www-data"

My TFTP root is /srv/tftp , so I gave ownership of this folder to the web server account, which is scary but necessary if FusionPBX is going to populate the TFTP folder with config files. You might want to take away user write permission from your firmware files.

chown -R www-data /srv/tftp

One annoying aspect of phone provisioning is that all phones want their firmware and configuration files right in the TFTP root, which gets messy if you have many phone models to configure. There is no good solution to this, but a coworker suggested putting each phone's configuration in a folder, and then using symlinks to the root (thanks Devin!):

cd /srv/tftp
mkdir cisco-7941
cp /tmp/cisco-firmware/* /srv/tftp/cisco-7941
ln -s cisco-7941/* .

In my DHCP server (dnsmasq on pfSense in this case) I set Option 150 to point to the IP address of the TFTP (and FusionPBX) server. The type of the record is "IP address" and the content is the bare IP.

Other phones use different options. Avaya uses Option 176 (which is a string, and has a different format). Most other TFTP-enabled phones use option 66. Cisco phones will fall back to this default TFTP option if you do not specify Option 150, but in my case I wanted to avoid this.

Getting Cisco 7941 Firmware

I was able to get the Cisco SIP firmware for this phone without paying money, but I had to sign up for a Cisco username and password.

There is good documentation about the format of the config files here: http://www.voip-info.org/wiki/view/Standalone+Cisco+7941/7961+without+a+local+PBX

Put the firmware files into your TFTP server root. To test whether things are working, it is helpful to manually create a config file for the phone. I found I needed to specify the following to get the phone working:

<?xml version="1.0" ?>
<device>
      <deviceProtocol>SIP</deviceProtocol>
      <sshUserId>root</sshUserId>
      <sshPassword>gjhe237ydh</sshPassword>
      <devicePool>
            <dateTimeSetting>
                  <dateTemplate>Y-M-D</dateTemplate>
                  <timeZone>Eastern Standard/Daylight Time</timeZone>
                  <ntps>
                        <ntp>
                              <name>192.168.100.1</name>
                              <ntpMode>Unicast</ntpMode>
                        </ntp>
                  </ntps>
            </dateTimeSetting>
            <callManagerGroup>
                  <members>
                        <member priority="0">
                              <callManager>
                                    <processNodeName>nb-freeswitch</processNodeName>
                                    <ports>
                                          <sipPort>5060</sipPort>
                                    </ports>
                              </callManager>
                        </member>
                  </members>
            </callManagerGroup>
      </devicePool>
      <sipProfile>
            <sipProxies>
                  <registerWithProxy>true</registerWithProxy>
            </sipProxies>
            <preferredCodec>g711u</preferredCodec>
            <phoneLabel>123_cisco</phoneLabel>
            <sipLines>
                  <line button="1">
                        <featureID>9</featureID>
                        <featureLabel>123_cisco</featureLabel>
                        <proxy>nb-freeswitch</proxy>
                        <port>5060</port>
                        <authName>123_cisco</authName>
                        <name>123_cisco</name>
                        <authPassword>supersecurepassword1</authPassword>
                        <messageWaitingLampPolicy>3</messageWaitingLampPolicy>
                        <messagesNumber>123_cisco</messagesNumber>
                  </line>
            </sipLines>
            <dialTemplate>dialplan-cisco-7941.xml</dialTemplate>
      </sipProfile>
      <loadInformation>SIP41.8-5-4S</loadInformation>
      <networkLocale>Canada</networkLocale>
      <networkLocaleInfo>
            <name>Canada</name>
            <version>5.0(2)a</version>
      </networkLocaleInfo>
      <directoryURL></directoryURL>
      <servicesURL></servicesURL>
</device>

It mayt be worth explaining the messagesNumber setting: this is the number you dial to access voicemail when you press the "Messages" button on the phone. You could specify *97 here, but then you are prompted for both the extension username and password; if you dial the extension directly you are only prompted for a password.

Call the file SEP<macaddr>.cnf.xml, where <macaddr> is the MAC address of the phone, specified in all caps with no spaces: eg SEP00123456789012.cnf.xml . You will want some kind of dialplan file as well. Mine is called dialplan-ciso-7941.xml, and it looks like this:

<DIALTEMPLATE>
<TEMPLATE MATCH="011*" Timeout="5" User="Phone"/> <!-- Local operator-->
<TEMPLATE MATCH="6.." Timeout="0" User="Phone"/> <!-- 4 digits intra-office -->
<TEMPLATE MATCH=".11" Timeout="0" User="Phone"/> <!-- Service numbers -->
<TEMPLATE MATCH="1.........." Timeout="0" User="Phone"/> <!-- Long Distance -->
<TEMPLATE MATCH="519......." Timeout="0" User="Phone"/> <!-- 10 digits -->
<TEMPLATE MATCH="226......." Timeout="0" User="Phone"/> <!-- 10 digits -->
<TEMPLATE MATCH="*"  Timeout="2"/>             <!-- Anything else -->
</DIALTEMPLATE>

I have not fussed with this dialplan too much. "226" and "519" are local area codes here, so I added those two lines.

If you are lucky, then at this point you should be able to factory reset the phone and get it uploading the new firmware and configuration file.

To factory reset the phone:

The phone should indicate that it is downloading the firmware files, and it will reboot a bunch of times.

If you are not lucky then the phone won't find the TFTP server, or it won't be able to get the files. Assuming the IP address given to the Cisco phone by DHCP is 192.168.100.101, the following invocation (run on the FusionPBX server) can be helpful:

tcpdump -n host 192.168.100.101 and port 69

You will also want to make sure that your TFTP server is working and listening:

netstat -anp --inet | grep tftpd

and using the tftp client from a different computer to download a test file are both helpful here.

At this point you should have a working phone, but FusionPBX does not know about the phone, and you have to manually make new SEP<macaddr>.cnf.xml files for each phone.

Registering TFTP in FusionPBX

To tell FusionPBX about the TFTP server, you have to specify some variables. In Advanced -> Default Settings you need to specify the Switch -> Provision -> Directory field (which is in the Switch section, not the Provision section). Set this to the directory on your server (in my case /srv/tftp ).

Apparently in FusionPBX there used to be a bunch of variables to specify different TFTP and FTP and HTTP folders. Those have all been replaced by the single Directory field, so I guess that all provisioning files have to go in the same place regardless of what file transfer protocol you use.

Next, in the Provision -> Enabled field you need to enable the value and set the value to true.

Usually FusionPBX is pretty good about making configuration changes take effect when you save them from the web interface, but in my case I found the changes did not take effect until I rebooted.

There are other variables in the Provision section that look relevant (such as restricting the subnets that can provision phones using CIDR notation) but I have not played with any of them yet.

Creating Templates and Associating Lines

I found that this part was really poorly documented!

Different phones come with different template files. FusionPBX uses these template files to populate your TFTP root with config files for each individual handset.

The Cisco 7941 does not come with templates out of the box, so I made my own. You can do this from the web interface (Advanced -> Provision Editor) but I found it easier to SSH into the server and edit config files directly. On my installation, template files live here:

/var/www/fusionpbx/resources/templates/provision/

This folder contains subfolders for each manufacturer FusionPBX knows about. In the cisco subfolder I made a sub-subfolder called 7941. In the 7941 folder I added two files:

Make sure that www-data can read and write these files so that you can edit them from the web interface later.

In the SEP{mac}.cnf.xml file you replace the hard coded values from your testing file above with variables. Here is a file that is working for me:

<?xml version="1.0" ?>
<device>
      <deviceProtocol>SIP</deviceProtocol>
      <sshUserId>root</sshUserId>
      <sshPassword>gjhe237ydh</sshPassword>
      <devicePool>
            <dateTimeSetting>
                  <dateTemplate>Y-M-D</dateTemplate>
                  <timeZone>Eastern Standard/Daylight Time</timeZone>
                  <ntps>
                        <ntp>
                              <name>192.168.100.1</name>
                              <ntpMode>Unicast</ntpMode>
                        </ntp>
                  </ntps>
            </dateTimeSetting>
            <callManagerGroup>
                  <members>
                        <member priority="0">
                              <callManager>
                                    <processNodeName>{$domain_name}</processNodeName>
                                    <ports>
                                          <sipPort>5060</sipPort>
                                    </ports>
                              </callManager>
                        </member>
                  </members>
            </callManagerGroup>
      </devicePool>
      <sipProfile>
            <sipProxies>
                  <registerWithProxy>true</registerWithProxy>
            </sipProxies>
            <preferredCodec>g711u</preferredCodec>
            <phoneLabel>{$display_name_1}</phoneLabel>
            <sipLines>
                  <line button="1">
                        <featureID>9</featureID>
                        <featureLabel>{$display_name_1}</featureLabel>
                        <proxy>{$server_address_1}</proxy>
                        <port>5060</port>
                        <authName>{$auth_id_1}</authName>
                        <name>{$auth_id_1}</name>
                        <authPassword>{$user_password_1}</authPassword>
                        <messageWaitingLampPolicy>3</messageWaitingLampPolicy>
                        <messagesNumber>{$auth_id_1}</messagesNumber>
                  </line>
            </sipLines>
            <dialTemplate>dialplan-cisco-7941.xml</dialTemplate>
      </sipProfile>
      <loadInformation>SIP41.8-5-4S</loadInformation>
      <networkLocale>Canada</networkLocale>
      <networkLocaleInfo>
            <name>Canada</name>
            <version>5.0(2)a</version>
      </networkLocaleInfo>
      <directoryURL></directoryURL>
      <servicesURL></servicesURL>
</device>

Where do these variable names ({$auth_id_1}, {$display_name_1}, etc) come from? They are set in the render() method of the following PHP file:

/var/www/fusionpbx/apps/provision/resource/classes/provision.php

[CHECK AGAIN]

the _1 at the end of some of the variables requires explanation. In the next section you associate line information for each device with a FusionPBX extension. (I find the use of the term "Line" confusing; in Norstar parlance this is more like an "Answer DN".) Each line you add comes with an index, which does not need to match the button mappings on the phone. That index determines the number at the end of the variables.

In my case, I wanted Line button 1 to correspond to Line 1 of the device, so the mapping was easy. If you wanted to program the second line with an additional extension that (for some reason) you specified as Line 5 in the interface, the part of the template specifying lines would look something like:

                  <line button="2">
                        <featureID>9</featureID>
                        <featureLabel>{$display_name_5}</featureLabel>
                        <proxy>{$server_address_5}</proxy>
                        <port>5060</port>
                        <authName>{$auth_id_5}</authName>
                        <name>{$auth_id_5}</name>
                        <authPassword>{$user_password_5}</authPassword>
                        <messageWaitingLampPolicy>3</messageWaitingLampPolicy>
                        <messagesNumber>{$auth_id_5}</messagesNumber>
                  </line>

From looking at the source, here are different variables that can be specified for the line values, assuming this is the first line:

(I am pretty sure that some of the values I am using in this template are incorrect, in the sense that I am putting the wrong value in the wrong place. I am also not bothering to specify ports, although I should.)

There are also a bunch of values that can be used for key mappings. Again, assuming key 1, we have:

but as of this writing I have not tried configuring these yet.

Other variables that are not associated with lines or keys (and thus do not have digits appended) are:

At this point FusionPBX now has template files, but will not actually be populating the TFTP folder with anything.

Provisioning Phones via the Web Interface

To test this functionality, it is helpful to have at least two handsets and two extensions to play with.

First, set up some extensions for these phones. You may want to make sure they work with your softphone.

Next, set up Accounts -> Devices for the Cisco phones. I find that specifying a minimal amount of information for each phone is useful: I just specify the MAC address a description, and the phone template (which should show up in the list as cisco/7941)

Now things get tricky, and I am not sure whether my workflow is right. I go back to the Extensions section of the interface, and in the Device Provisioning section I then choose the device from the list of MAC addresses.

Then I have to go back to the Device section and make sure that the line index is set correctly (to 1 in my case, because I have _1 appended to my variables in the template). A bunch of the other provisioning information (Server Address, Auth ID, Password, and maybe some others) should be prefilled for you.

When you save these changes, you should find that /srv/tftp is populated with one .cnf.xml file for each phone, with the phone's MAC address specified in the name. Now if you boot your phones (a factory reset should not be necessary) they should be provisioned with your FusionPBX extensions.

One bug: I found that the first time I saved the file the .cnf.xml filename was specified with lowercase letters, which is wrong for Cisco phones. But if I saved the page a second time the correct filename gets generated.

If this is not working check the generated .cnf.xml files and make sure that all of them have been populated with the variable values you want. I found this process fiddly: sometimes one of my test files would be populated and the other would not. Sometimes this was because I inadvertently added multiple extensions to a phone, and gave them all the same line index.

I think templates for ALL phones get generated when you save changes to ANY device. So you do not want to fiddle with the generated files manually.

At this point you will want to figure out how to configure keys, directories, voicemail lights, and other specific features. As of this writing, I have no idea how to do this, but getting basic phone functionality working was a big step that I thought was worth documenting. I also want to provision some different models of phones (in particular some Avaya 4610SW handsets we were donated).