This guide will address establishing a remote connection with a mobile operator, create a data collecting RapidSMS application, and everything in between – specifically using the Kannel SMS gateway.
I realize this guide will only cater to a very specific group of technicians seeking to use the following technologies:
-IPSec VPN, as their means of communicating with a mobile operator (SMS Center).
-Kannel, as the SMS Gateway which talks to the SMS Center, relaying messages via HTTP requests.
-RapidSMS, as their choice of data-collection and communication tool.
This guide is aimed at demystifying any confusion that may occur in an otherwise obscure and arcane world of SMS. I will use my very own examples and clarify version numbers to show what works. I am using the Debian 6.0 “Squeeze” release.
VPN
We will begin by establishing a Virtual Private Network as the very first step – without it, no data would be able to get in or out from the SMS Center.
At this point, you should have already made arrangements with at least one mobile operator to establish a VPN with their SMS Center. Additionally, they should have provided you with a list of parameters necessary for the connection, these include authentication methods, source IP addresses, ports, keys, and much more.
This guide will be focusing on establishing an IPSec VPN – solely because of the specific parameters given by my mobile operator.
Below is a copy of the sheet of parameters given to me by the local mobile operator post-agreement, use it to cross-reference with the configurations in the next few steps:
To establish security associations with the host (SMS Center), we will use racoon, an ISAKMP daemon – as it compliments the IKE and IPSec parameters given to me.
I am using racoon version: 1:0.7.3-12
$ sudo apt-get install racoon
During setup, I recommend using direct configuration mode.
Along with racoon, the package also contains control tools and kernel SPD & SAD manipulation tools which we will discuss later.
We can now begin configuring racoon – by default, the configuration file is located at /etc/racoon/racoon.conf
Racoon configuration is heavily documented with countless examples. Below is a list of websites you can refer to configure your file based on the parameters given to you:
–http://linux.die.net/man/5/racoon.conf
–http://www.kame.net/newsletter/20001119/
–http://netbsd.gw.com/cgi-bin/man-cgi?racoon.conf
–http://www.netbsd.org/docs/network/ipsec/rasvpn.html#client_conf
This is my racoon.conf:
log info;
path pre_shared_key "/etc/racoon/psk.key";
listen {
adminsock "/var/run/racoon/racoon.sock" "root" "operator" 0660;
isakmp 192.168.1.60 [500];
}
remote 11.111.111.1 {
exchange_mode main;
nat_traversal off;
ike_frag on;
mode_cfg off;
passive off;
script "/etc/racoon/phase1-up.sh" phase1_up;
script "/etc/racoon/phase1-down.sh" phase1_down;
peers_identifier address "33.33.33.33";
my_identifier address "22.222.22.22";
proposal_check obey;
proposal {
encryption_algorithm 3des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group 2;
}
}
sainfo address 22.22.22.22 any subnet 33.33.33.33/32 any {
lifetime time 2 hour;
pfs_group 2;
encryption_algorithm 3des;
authentication_algorithm hmac_md5;
compression_algorithm deflate;
}
Before running racoon, we need to update our Security Association Database (SAD) entries as well as Security Policy Database (SPD) entries in the kernel. The setkey utility, included in the recently installed package, can take care of this. Setkey deserves an entire thread for itself, so I won’t go into much detail – but there are several sites you can refer in order to properly take advantage of it:
–http://www.freebsd.org/cgi/man.cgi?query=setkey
–https://help.ubuntu.com/community/IPSecHowTo
–http://netbsd.gw.com/cgi-bin/man-cgi?setkey+8+NetBSD-current
For clearance, here is what my setkey config file looks like:
flush; spdflush; spdadd 22.222.22.22 33.33.33.33/32 any -P out ipsec esp/tunnel/192.168.1.60-11.111.111.1/require; spdadd 33.33.33.33/32 82.114.84.33 any -P in ipsec esp/tunnel/11.111.111.1-192.168.1.60/require;
And I can run it like so:
$ setkey -f /etc/racoon/setkey.conf
After setkey, we can ran safely run racoon:
$ racoon
Note: You can specify the racoon config file location (other than the default) using the -f argument. Also, I recommend running racoon in the foreground (-F), for easier & faster logging, as you are bound to run into some initial errors. If you are certain that everything is in order and you are still experiencing issues, you may want to consider modifying your firewall:
–https://help.ubuntu.com/community/IptablesHowTo
$ racoon -f /root/Desktop/racoon.conf -F -dd
Finally, using the racoon control tool, racoonctl, we can connect to the VPN.
$ racoonctl vpn-connect 11.111.111.1
Don’t forget to add the new route:
$ route add -net 33.33.33.33/32 eth2
And lastly, ping the server to double-check if everything is up and running:
$ ping 33.33.33.33
In conclusion, to establish a VPN connection with an operator, the following are required:
1. An agreement with the operator to establish a VPN, along with parameters.
2. A properly configured VPN client.
(In my case)
1. Install racoon.
2. Update security associations (setkey).
3. Configure racoon.
4. Run racoon.
5. Run racoon control tool (VPN connect).
6. Add to routing table.
Note: Should you experience any faulty configuration and need to restart racoon, make sure to kill any previously running processes:
$ killall -9 racoon $ killall -9 racoonctl
Kannel
After the VPN has been established, we can now begin installing Kannel. Kannel acts as a gateway between the connected SMS Center and you – essentially sending and receiving SMS’.
I am using Kannel version: 1.4.3-2
$ sudo apt-get install kannel
By default, the Kannel config file is located at /etc/kannel/kannel.conf
The following can be used to properly configure Kannel, but I will specifically outline the noteworthy Kannel ‘groups’.
–http://www.kannel.org/download/1.1.6/gateway-1.1.6/doc/kannel.conf
–http://www.kannel.org/download/1.4.0/userguide-1.4.0/userguide.html
–http://www.3deography.net/en/articles-topmenu-50/40-setting-up-a-sms-gateway-with-linux-and-kannel
–http://mobiforge.com/developing/story/sending-sms-with-smpp-kannel-and-java
Please read the variables carefully and know when they are necessary.
The first group I want to mention is smsc. This group must include all the data needed to connect that SMS center. Note: In addition to the parameters given to me by the mobile operator, a username and password were also given for their SMS Center. This is my smsc group configuration:
group=smsc smsc=smpp smsc-id=operatorX interface-version=34 host=33.33.33.33 port=2775 receive-port=2775 system-id="foo" smsc-password=bar system-type=default transceiver-mode=1
The next group is core. This group is absolutely necessary for proper functionality as it contains the most general configuration for bearerbox. Note that the admin-port and admin-password may be arbitrary as they are only used to execute various administration commands.
group = core admin-port = 13000 smsbox-port = 13002 admin-password = foobar log-file = "log" log-level = 2
Core also handles what machines can and can’t send SMS via HTTP.
The smsbox is also a very noteworthy group. Essentially, this group handles outgoing sms – this is what mine looks like:
group = smsbox bearerbox-host = 127.0.0.1 sendsms-port = 13014
And this is the group that defines an account which can be used for the SMS-push-via-HTTP interface:
group = sendsms-user username = user password = pass
What this part of the configuration translates to is this URL:
http://127.0.0.1:13014/cgi-bin/sendsms?username=user&password=pass&from=7777&to=12345678&text=sometext
With this URL, you are sending “sometext” to mobile # 12345678 from 7777 via user (Note that in some countries, such as the U.S., the from parameter is only limited to the phone # assigned to you by the operator and cannot be altered.
Finally, the last group that’s of importance is sms-service. This group is in charge of handling incoming SMS.
group = sms-service keyword = default max-messages = 0 get-url = "http://127.0.0.1:8000/router/receive?backend=%i&sender=%p&message=%b" concatenation = true assume-plain-text = true accept-x-kannel-headers = true omit-empty = true
The main idea here is the get-url. This simply calls the specified URL whenever a message is received from the SMS Center. i.e.: If mobile # 12345678 sends “hello” to 7777 (your assigned #), the URL, “http://127.0.0.1:8000/router/receive?backend=operatorX&sender=12345678&message=hello” will be called automatically.
Important Points:
- get-url :: The url can include a list of parameters, which are parsed before the url is fetched. This calls the URL and GET’s the content and replies to the sender with that content.
- post-url :: As above, but request is done as POST, not GET.
“127.0.0.1:8000/router/receive” is simply a RapidSMS httprouter instance that we will create in the next step that handles incoming SMS.
This concludes setting up Kannel. In summary:
1. Install Kannel.
2. Configure group: core – general bearerbox configuration and allow/deny IP.
3. Configure group: smsc – handles connection with the SMS Center.
4. Configure group: smsbox & sendsms-user – handles outgoing SMS.
5. Configure group: sms-service – handles incoming SMS by calling URL.
Now we simply run bearerbox first and then smsbox. Bearerbox handles the connections with the SMS Center. Smsbox provides an http service to send the SMS through the bearerbox.
Navigate to your kannel.conf file and:
$ cd /etc/kannel $ bearerbox
Open up another terminal and run smsbox while bearerbox is running in the foreground for best monitoring.
$ cd /etc/kannel $ smsbox
If everything is in order, you should receive an “Accepted for delivery” message after calling a URL similar to this:
http://127.0.0.1:13014/cgi-bin/sendsms?username=user&password=pass&from=7777&to=12345678&text=sometext
And the recipient should also have received the message.
If not, kill previous bearerbox & smsbox processes and reconfigure…
RapidSMS
We will now begin installing RapidSMS and create an application that will record first and last names sent via SMS into a MySQL database.
It is important to pay attention to what versions I am installing from this point on as there are countless backwards compatibility issues involving Python, RapidSMS, Django, and other required packages. The versions herein have proven to be complementary of each other in my case…
Let’s first setup a virtual environment running Python 2.7.
$ sudo apt-get install python2.7 $ sudo apt-get install python-virtualenv python-pip $ virtualenv --no-site-packages --python=python2.7 ~/rapidsms_environment $ source ~/rapidsms_environment
Next, install Django 1.3 – I have found 1.3 to be the most stable when working with RapidSMS:
$ pip install django==1.3
Then we can install RapidSMS along with rapidsms-httprouter, which allows for the RapidSMS ‘routing’ process to be done in the HTTP thread. Note that solely installing rapidsms-httprouter 0.5.1 will also install the correct version of RapidSMS, 0.9.6a.
$ pip install rapidsms-httprouter
Once again, be sure to work in your virtual environment, then setup a RapidSMS project.
$ source ~/rapidsms_environment $ cd /root/Desktop/ $ rapidsms-admin.py startproject rapidsms_project $ cd /root/Desktop/rapidsms_project
Now we can begin to configure RapidSMS to work with Kannel. Open settings.py from your project’s directory and append to the ‘INSTALLED_BACKENDS’ like so:
INSTALLED_BACKENDS = {
"operatorX": {
"ENGINE": "rapidsms.backends.kannel",
"host": "127.0.0.1",
"port": 8000,
"sendsms_url": "http://127.0.0.1:13014/cgi-bin/sendsms",
"sendsms_params": {"from": "7777",
"username": "user",
"password": "pass",
"coding": 0,
"charset": "ascii",
}
},
"message_tester": {
"ENGINE": "rapidsms.backends.bucket",
}
}
For a more in-depth explanation of RapidSMS backend configuration, please refer the following:
After the ‘INSTALLED_BACKENDS’ segment is complete, we will now add rapidsms_httprouter to the ‘INSTALLED_APPS’ segment:
INSTALLED_APPS = [ "rapidsms", .. .. "rapidsms_httprouter" ]
Following this, we must append the following into urls.py:
urlpatterns = patterns('',
..
..
('', include('rapidsms_httprouter.urls'))
)
This tells the RapidSMS project to allow the native httprouter app to handle URL schemes, specifically the URL that will be called by Kannel whenever incoming messages arrive. Httprouter will then route to other apps to handle that URL. We will now move on to build a custom RapidSMS app that will simply register a person into our MySQL database and send a response SMS letting them know that they are successfully registered.
Since we will be using MySQL, we must configure it with our RapidSMS project. In settings.py append the following to ‘DATABASES’:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.mysql",
"OPTIONS" :{
"read_default_file": "/root/Desktop/rapidsms_project/sql.cnf",
}
}
}
The ‘read_default_file’ will contain your MySQL credentials. In “/root/Desktop/rapidsms_project”, I have the following:
[client] database = rapidsms user = root password = pass123 default-character-set = utf8
Syncdb will synchronize and update the table structures – both RapidSMS’ and its corresponding apps.
$ python manage.py syncdb
Before we move on to the next step, let us check if we are able to get RapidSMS up and running standalone. Navigate to your project’s directory and start the router and server simultaneously in different terminals. The router is the main component of RapidSMS – its process will start all the configured backends and apps. The server is simply the webserver.
$ cd /root/Desktop/rapidsms_project $ python manage.py runserver $ python manage.py runrouter
If there you experience no errors from the last three commands, we are ready to create our app.
RapidSMS Application
We can now create a RapidSMS app that receives an SMS, parses it, inputs the information into a database, then responds to the sender with a confirmation message. For more in-depth descriptions and alternative apps refer: https://github.com/rapidsms/rapidsms-documentation/blob/master/source/tutorial.rst
$ cd /root/Desktop/rapidsms_project $ python manage.py startapp my_application
This will create the foundation for a new application.
Before we begin writing the app, lets configure it to work with the main RapidSMS project. Open up ‘settings.py’ again add your app to ‘INSTALLED_APPS’:
"rapidsms", .. "rapidsms_httprouter", .. "my_application"
Additionally, while we are in settings.py, append the following group. This group will be used by rapidsms_httprouter for SMS handling:
SMS_APPS = [
"rapidsms_project.my_application",
]
Finally, in order for you app to reply to messages, rapidsms_httprouter needs the URL scheme – add this to settings.py:
ROUTER_URL = "http://localhost:13014/cgi-bin/sendsms?from=7777&username=user&password=pass&text=%(text)s&to=%(recipient)s&smsc=%(backend)s"
Keep in mind that we have already configured Kannel to call the proper URL to integrate with rapidsms_httprouter.
Let’s create the app now. Again, this app will be simply storing the first and last name of anyone who sends a message in the following format:
register first_name last_name
So, we have to create a table that will store this information. models.py – “Your Django models are defined in this file. We’ll be using this file to define the model structure for persisting data in the database.” models.py will be taking care of the database structure.
In models.py:
from django.db import models
# Create your models here.
class Birth(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
Now re-sync the database:
$ python manage.py syncdb
Navigate to your app’s directory and open ‘app.py’ – This file is where your application logic is stored, as described here.
Below is my version of the app.
#!/usr/bin/env python
# vim: ai ts=4 sts=4 et sw=4
import rapidsms
import re
from rapidsms.apps.base import AppBase
#database model
from models import *
class App(AppBase):
#regular expression operations: "register string string"
pattern = re.compile(r'^register\s+(\w+)\s+(\w+)', re.IGNORECASE)
def handle(self, message):
#everything but "register"
response = self.pattern.findall(message.text)
if response:
entry = response[0]
# Persist entry in the database
My_application(
first_name=entry[0],
last_name=entry[1]).save()
message.respond("Thank you %s, you are registered!" % entry[0])
else:
message.respond('Please enter: register first_name last_name.')
Save your app and start the server. While closely monitoring all terminals, try and send a message from a cellphone or via URL. If it failed to respond after you sent the message, check all terminals and see where it went wrong, otherwise congratulations!
Below is a list of possible problems that may arise and their solutions:
- After calling a URL and you receive: Queued for later delivery: Problem with bearerbox or smsbox – If you have configured Kannel properly, kill both processes and retry and check to make sure VPN tunnel is up.
- After calling a URL and you receive: Accepted for delivery but you did not receive the message: Check the URL encoding, some international numbers require ‘+’ before the number. ‘+’ is encoded: ‘%2B’, append it before the ‘to’ parameter.
- Receive message: “Could not fetch content, sorry.” Check if Kannel sms-service group is properly configured or if your app is properly parsing then responding a request and the response is shorter than 140 characters.
- Receive message: “Request failed.” Check if Kannel sms-service group is properly configured or if your app is properly parsing then responding a request.
- Else: google
In conclusion, we have established a VPN with a mobile-operator’s SMS Center, configured Kannel to send and receive messages to and from the SMS Center as well as RapidSMS, have setup a RapidSMS project with an app, and installed rapidsms_httprouter to route messages to and from the RapidSMS app in the HTTP thread.
Feel free to ask any questions.
Altin Ukshini 