Paul's Internet Landfill/ 2016/ Provisioning a SPA8000 ATA Using FreePBX OSS Endpoint Manager

Provisioning a SPA8000 ATA Using FreePBX OSS Endpoint Manager

The Linksys/Cisco (and Sipura?) SPA8000 is an eight-port VoIP ATA: it allows you to register up to eight analog telephones to a VoIP provider. They are the cheapest ATA I know of that provides eight FXS ports, and we use them extensively to provide VoIP "lines" to our Norstar system.

Recently, we set up a SPA8000 to provide regular analog telephones using Asterisk with FreePBX. Provisioning this box automatically was a trial that required diving into the FreePBX source code, so I am documenting what I learned in public.

The SPA8000 is weird because it is essentially four two-port SPA2102 ATAs hooked together. Lines one and two come from the first ATA. The other six lines come from three ATAs networked "behind" the first one. They use NAT to traverse the first ATA and connect to the VoIP provider.

Most VoIP providers provide SIP traffic via port 5060. This works for Line 1 (and maybe Line 2?) but the other lines cannot use this port. Instead, by default the ATA gives them other port numbers: 5060 for line 1, 5061 for line 2, 5160 for line 3, 5161 for line 4, 5260 for line 5 and so on.

The FreePBX web interface comes with a largely-unsupported provisioning system called the "OSS Endpoint Manager". (Sangoma, the company which produces FreePBX, has its own commercial provisioning system, which might explain why they are not that enthusiastic about supporting OSS Endpoint Manager.) The job of the Endpoint Manager is to "provision" phones by taking extension and registration information already stored in Asterisk/FreePBX and make it available to VoIP handsets and ATAs. When these handsets and ATAs boot, they pick up the generated configuration files and use them to configure themselves appropriately.

To do this, the configuration manager takes template files and populates them with relevant registration information. It then drops these template files into a TFTP folder for the handsets/ATAs to pick up.

Supposedly, the SPA8000 is supported by OSS Endpoint manager. Unfortunately, it does not work right, because the existing template sets the server port for every line to the same value: 5060. This might work for lines 1 and 2, but lines 3-8 won't be able to register with this template. My job was to fix the template so that all the lines would register.

(My other option was to forget about the OSS Endpoint Manager entirely and manually enter registration information into the SPA8000. This would have worked, but it makes working with the FreePBX interface messy in other ways, because FreePBX won't know that the extensions associated with the SPA8000 are already registered to a device.)

As of this writing, the version of FreePBX is 12.0.76.2 . The version of Endpoint Manager is 2.11.13 . The version of Asterisk is 11.13.1 . Who knows what the future will bring?

Here is what little I learned.

Templates and Variables

The primary template for the SPA8000 (and a bunch of other Linksys devices) are in two files $model.cfg and spa$mac.xml . On my installation, I found these templates in the following folder: /var/www/html/_ep_phone_modules/endpoint/cisco/linksysata/ .

Supposedly the $model.cfg is only used for initial configuration, and so is not that helpful to us. spa$mac.xml is the important file.

In spa$mac.xml you will find some stanzas with line settings. Here is the loop indicator and the first stanza:

{line_loop}

<!-- Line Registration Information -->
<Display_Name_{$line}_ ua="na">{$displayname}</Display_Name_{$line}_>
<User_ID_{$line}_ ua="na">{$username}</User_ID_{$line}_>
<Password_{$line}_ ua="na">{$secret}</Password_{$line}_>
<Dial_Plan_{$line}_ ua="na">{$dial_plan}</Dial_Plan_{$line}_>
<SIP_Port_{$line}_ group="Ext_{$line}/SIP_Settings">{$server_port}</SIP_Port_{$line}_>
<Register_Expires_{$line}_ group="Ext_{$line}/Proxy_and_Registration">{$server_expires|3600}</Register_Expires_{$line}_>
<Ans_Call_Without_Reg_{$line}_ ua="na">{$answer_call_without_reg|No}</Ans_Call_Without_Reg_{$line}_>

There are clearly variables within curly braces, and the one we are interested in looks like {$server_port}. If we can change this per line then we are golden.

Not so fast, cowboy. While the endpoint manager does provide mechanisms for setting variables, in this case they are irrelevant. In /var/www/html/endpointman/includes/functions.inc there is the following code snippet:

//Set Variables according to the template_data files included. We can include 
// different template.xml files within family_data.xml 
// also one can create template_data_custom.xml which will get included or 
// template_data_<model_name>_custom.xml which will also get included
//line 'global' will set variables that aren't line dependant


$provisioner_lib->settings = $new_template_data;

//Loop through Lines!
$li = 0;
foreach ($phone_info['line'] as $line) {
    $line_options = is_array($line_ops[$line['line']]) ? $line_ops[$line['line']] : array();
    $line_statics = array('line' => $line['line'], 'username' => $line['ext'], 
        'authname' => $line['ext'], 'secret' => $line['secret'], 'displayname' => $line['description'], 
        'server_host' => $this->global_cfg['srvip'], 'server_port' => '5060', 
        'user_extension' => $line['user_extension']);

    $provisioner_lib->settings['line'][$li] = array_merge($line_options, $line_statics);
    $li++;
}

Most of the variables (eg username, secret, displayname) are specified in some kind of $line array. Meanwhile, server_port is hardcoded to be 5060. I spent a bunch of time trying to work around this by specifying $server_port in a file, with no success.

To make a long story short, I decided to hack around the problem by coming up with a new variable, $line_sip_port. This variable should have a different value for each line. The comments suggest that there are line-dependent variables, and the array_merge call suggests that we can (somehow) merge new variables into the configuration if we can get them into $line_options.

To use this variable, I changed the server_port line in spa$mac.xml from:

<SIP_Port_{$line}_ group="Ext_{$line}/SIP_Settings">{$server_port}</SIP_Port_{$line}_>

to

<SIP_Port_{$line}_ group="Ext_{$line}/SIP_Settings">{$line_sip_port|5060}</SIP_Port_{$line}_>

The |5060 indicates that if $line_sip_port does not exist then the default value "5060" should be used for the port.

Specifying Variables

How do you create new variables? How do you specify them so they are different for each line? This is where things get really messy, and although I got something working I do not understand the mechanisms very well.

First, variables. In the file /var/www/html/_ep_phone_modules/endpoint/cisco/linksysata/family_data.json there is a file which specifies options for each phone model in the family. The entry for the SPA8000 appears to be:

{
    "model": "SPA8000",
    "lines": "8",
    "id": "7",
    "template_data": [
        "template_data.json",
        "keys.json"
    ]
},

If we look at template_data.json we see a complicated JSON file that seems to specify variables. These variables apply globally. For example, here is $enable_webserver:

{
  "variable":"$enable_webserver",
  "default_value":"Yes",
  "description":"Enable Webserver",
  "type":"radio",
  "data":[
    {
      "text":"Yes",
      "value":"Yes"
    },
    {
      "text":"No",
      "value":"No"
    }
  ]
},

There is a different JSON file several levels up, in /var/www/html/_ep_phone_modules/endpoint/global_template_data.json . It contains intriguing stanzas with per-line options. Here is an excerpt:

{
    "description": "Line Options",
    "type": "loop_line_options",
    "data": {
        "item": [
       {
            "variable": "$line_enabled",
            "default_value": false,
            "description": "Enable Line {$count}",
            "type": "checkbox"
        },
        {
            "variable": "$displayname",
            "default_value": "",
            "description": "Display Name",
            "type": "input"
        },
        {
            "variable": "$username",
            "default_value": "",
            "description": "Username Name",
            "type": "input"
        },
     }

}

$displayname and $username are defined per line, so this is promising. The magic seems to be the line

"type": "loop_line_options",

and the variable {$count} .

Thus, to add new variables, I made a new JSON file using the "loop_line_options" to populate the server ports. I called this file spa8000_lines.json and modified family_data.json to include it:

{
    "model": "SPA8000",
    "lines": "8",
    "id": "7",
    "template_data": [
        "template_data.json",
        "spa8000_lines.json",
        "keys.json"
    ]
},

That was the easy part. The hard part was actually crafting the JSON file. I have no idea how the provided files are crafted (I expect using some automated process), but I ended up hacking together a hand-coded file. One of the hardest parts was figuring the proper nesting levels. Here is the file I came up with:

{
  "template_data":{
    "category":[ {
        "name":"lines-level-0",
        "subcategory":[ {
            "name":"Line Settings Level 1",
            "item":[ {
                 "type":"loop_line_options",
                 "description":"Line Ports Level 4",
                 "data":{
                     "item":[ {
                        "variable":"$line_sip_port",
                        "default_value":"506{$count}",
                        "description":"Line {$count} SIP Port",
                        "type":"input"
                     }]

                 }

             }]
         } ]
     } ]
  }
}

The default_value entry is a terrible hack. It appends the line number {$count} to the string 506 to produce the ports 5061 to 5068 for the eight lines. This will break dramatically for ATAs with greater than nine lines, but I could not figure out a good way of doing the math properly.

It is not strictly necessarily to prepopulate the default_value with different values for each line. This JSON snippet makes a new screen in the endpoint manager,
and it is possible to set these values manually for each line. That is error-prone, however, so I prefer my terrible hack.

Line Loop Options and Lines

In the process of figuring out the JSON files, I learned another tidbit of documentation which is worth documenting. The magic for the SPA8000 ports was the following line:

"type": "line_loop_options"
There is another type available: "loop". Here is an example, taken
from
/var/www/html/_ep_phone_modules/endpoint/polycom/spom/linekeys.json

{ "description":"Line Keys", "type":"loop", "loop_start":"1", "loop_end":"12", "data":{ "item":[ { "variable":"$lineops_linekeys", "default_value":"1", "description":"LineKeys For Line {$count}", "type":"input" }, { "type":"break" } ] } }

This type requires you to specify "loop_start" and "loop_end" elements as well. At first I thought this was the secret to setting server ports, but it did not work, because the loop variable {$count} was not specific to a line.

I do not think that there are other secret types, based on functions.inc. There are case statements to parse loop and line_loop_options (and lineloop, which I think is the same thing?) in that file, but there is nothing else.