- Connector types
- Connector modes
- Static vs non-static
- Connector states
- Message types
- Address rewriting
- Inheritance and virtual connectors
- Limiting connector queue sizes
- Retry schemes
- Sample configurations
Connectors provide the interface between Enterprise Messaging Gateway and other systems and applications. For example, one connector can connect to an operator’s SMSC and another connector can accept incoming messages via HTTP. Each connector can be available in one or more instances. The core functionality of EMG is to route messages from one connector to another.
When testing EMG or running benchmarks it can be useful to route message from an outgoing EMG connector to an incoming connector in the same server. We call this short-circuiting connectors.
A connector can be one of two types: Incoming or outgoing. An incoming connector accepts incoming connections and an outgoing connector initiates connections. Please note however that whether a connector is incoming or outgoing does not tell anything about the direction of flow of the messages on the connector. Messages can be received on an outgoing connection, for example when connecting to a Nokia SMSC.
Both incoming and outgoing connectors can exist in 0 or more instances. However, normally an outgoing connector only needs one instance while an incoming connector needs several instances in order to be able to serve several simultaneous connections.
Connectors can be defined to be transmit-only, receive-only or both transmit and receive. This is not the same as incoming vs outgoing. For example when connecting to a SMSC an outgoing connector can poll for messages, which will then be received.
The connector mode is specified using the MODE keyword. By default connectors can both transmit and receive messages. Incoming messages on a transmit-only connector will be rejected.
In SMPP 3.3 however, sessions will be used either for transmit or receive depending on the bind operation used. This means that in order to be able to both send and receive messages at least two connectors must be defined.
Static vs non-static
An outgoing connector can be defined as static using the STATIC keyword. This indicates that the connector should connect to the remote entity immediately even if there are no messages to send. The connector will still respect any idle timeout specified. If IDLETIMEOUT is set to a non-zero value the connector will disconnect after being idle the specified number of seconds and the immediately reconnect since it is configured as static.
Usually you would use IDLETIMEOUT=90 and set KEEPALIVE to a slightly lower value, for example 60 (seconds). This will keep the connection open by sending keepalive operations periodically to ensure the connection up.
A connector can be in exactly one of the following states at a specific time:
|IDLE||Connector is just created or has been dynamically disconnected. Static connectors should never be in this state during normal operation.|
|CONNECTED||Connector is connected to remote endpoint but has not been authenticated.|
|BOUND||Connector is authenticated and allowed to transmit and receive messages.|
|TERMINATING||Connector will perform a graceful shutdown as soon as possible.|
|DEAD||Connector is shut down.|
|ERROR||Connector has failed a repeated number of times and is awaiting a new retry period.|
On top of the above a connector can be put “on hold”, which means messages are accepted in the queue but no messages are transmitted. A message can be put on hold using the emgclient utility or another implementation using the MGP protocol.
Starting with EMG 6 it is also possible to “force close” a connector either using emgclient or via MGP API. The connector can either be bounced (forced to close but then released again to allow for immedia reconnect) or forced to close with this state persisted in which case the “force close” state will survive a server restart.
A connector is available in 0 or more instances. In order for the connector to be active and be able to send or receive messages it must be available in at least one instance.
Outgoing connectors usually only needs to be available in one instance. From EMGs point of view one instance is enough to handle hundreds of messages per second. However depending on delays imposed by the remote entity using more than one instance MAY increase performance.
Incoming connectors must be available in more than one instance in order to allow for several simultaneous connections. When an incoming request has been processed and the remote entity logs out and drops the connection the instance can be re-used. However, in some cases if the connection fails due to a network error for example it may take a while before this is detected and the session is cleared and available for use again. In this case it would be an advantage to have more instances than the expected number of simultaneous connections. If two incoming connections are expected maybe 5 instances would be suitable.
The default message type in EMG is SMS. This for example means that when a message is received on an SMTP connector it is automatically parsed and converted to a format that suits SMS and all attachments that are not text are discarded. In order to preserve the formatting for an incoming e-mail message it is therefore necessary to set the message type to e-mail by using DEFAULT_MSGTYPE=EMAIL. Then the conversion will not take place until the message is sent out over a connector of another message type. The message types in EMG are as shown below.
|SMS||1||Default message type|
|6||E-mail message formatted according to RFC 822 or RFC 1521 (MIME)|
|MMS||7||Multimedia message (MMS)|
|WAPPUSH||9||WAP push message|
In order to translate between different character sets, being able to send and receive specific symbols etc it may be necessary to apply mappings to the message data.
Mappings in EMG are defined per connector using the MAPPING keyword. The keyword specifies a filename which contains the mapping or translation table. EMG handles one-to-one, one-to-many, many-to-one and many-to-many.mappings and both text and binary data can be processed.
The format of a mapping file is the following (fields are tab-separated), left-hand column specifies a source data looked for and right-hand column the replacement:
# This is a comment in a sample mapping file. # First we define a mapping applied when a message is received # Both character string and hex codes can be used # First line translates all "a"s into "b"s IN < "a" "b" "xyz" "zyx" 03,"a" "a",03,02 > # Then a mapping which will be applied to outgoing messages First line will have no effect (as long as ascii char 0x41 is "A" OUT < "a",41,"b" "aAb" 01,02,03 20,20 >
Mappings are processed top to bottom and the following two examples will generate different result if applied to the string “aabacc”.
# Example 1 (will generate "ddbdcc") IN < "a" "d" "aa" "xx" > # Example 2 (will generate "xxbdcc") IN < "aa" "xx" "a" "d" >
Incoming (IN) mappings are applied when an incoming message is parsed before it is logged and routed. Therefore the EMG server only sees the translated message data.
Outgoing (OUT) mappings are applied to the message just before they are sent and leave EMG. Therefore the translated message data is never seen in the EMG server.
Characters can be removed by using an empty replacement, “”.
Mappings are only supported for 7-bit text messages.
Both source and destination addresses can be rewritten by EMG after a message has been received or before a message is sent. This is particularly useful to make sure that addresses comply with the format requirements of the receiving entity.
There are four keywords used for destination address rewriting applied in the order they appear below:
There are also four corresponding keywords for source address rewriting:
These keywords handle outgoing messages via a specific connector while the corresponding keywords ending with “_IN” handle incoming messages, as in REPLACE_PREFIX_IN and REGEXP_SOURCEADDR_IN.
REPLACEPREFIX was introduced in EMG 2.5 and can do all that the REMOVE/REQUIRE-keywords can do and more. It takes as argument one or more comma-separated pattern/replacement pairs where each matched (prefix) pattern is replaced by the corresponding replacement. An empty pattern can be used to add a prefix and an empty replacement to remove a prefix. All pairs are applied sequentially in the order they appear.
This example could be used in Sweden where we want to handle addresses received in the following formats:
|+46123456||International format, + as prefix|
|46123456||International format, no prefix|
|0046123456||International format, 00 as prefix|
When messages are sent out the receiving SMSC expected format is usually an international format without any prefix. The above REPLACEPREFIX statement would perform the wanted rewrite. Once again, operations are performed in sequence from left to right.
First any prefix (`+’ or `00′) is removed if present, then if a national number was supplied the leading 0 is replaced by the country code (46). If instead a +-character was required for the connection only a small modification would be needed:
This example performs the same conversion as the earlier and in addition a “+” is added as the final step.
Address masquerading can be used to hide the real sender of a mobile originated (MO) message before forwarding the message to a third-party. This may be required by some operators in order to get permission to handle MO traffic.
The masquerading is performed using a simple but efficient obfuscation of the address and the algorithm used is reversible and unique for each address. An address with 1-9 digits will be translated into a 10-digit address. Addresses with 10-18 digits will be translated into a 20-digit address.
When using the masquerade keyword on a connector it will be applied to all messages sent and received over that connector.
See the connector keyword MASQUERADE for more information.
Source Address Translation (SAT)
Source Address Translation (SAT) is the procedure of replacing the source address of a message with another address and is most commonly used for implementing bidirectional e-mail to SMS services.
For an e-mail to SMS service an incoming e-mail will obviously have an e-mail address as source (sender) address and before forwarding the message to a mobile phone as SMS it is needed to replace the e-mail address with a valid GSM number. If the user should then be able to reply to the message by simply hitting the reply button the GSM number much be chosen so that the reply message can be routed back to EMG via the SMSC.
Further if the combination of source and destination address can be made unique for each message a specific mobile phone user receives it will be possible to map a specific reply to a message back to the original message sent to the user. We call this threaded messages.
All this functionality can be implemented using SAT pools in EMG where a SAT pool is a pool of source addresses from which EMG chooses an address for each message going out to a mobile phone via EMG.
For the connector where the e-mail is to be received the connector keyword SATPOOL_CREATE_IN is used to indicate that SAT entries should be created when messages are received. On the connector where a reply SMS arrives the SATPOOL_LOOKUP_IN keyword is used in order to perform the lookup for the previously created SAT entry and to rewrite the address back to the original e-mail address.
The SATPOOL_CREATE and SATPOOL_LOOKUP keywords works the same way but operates on messages being sent out via the connector where they are used.
Sample configuration (parts of configuration omitted):
SATPOOL sat1 < ADDRESSRANGE=10010-10020 THREADED > CONNECTOR smtp-in1 < PROTOCOL=SMTP ... SATPOOL_CREATE_IN=sat1 # Keep source e-mail address intact REGEXP_SOURCEADDR_IN= ROUTE=smsc ... > CONNECTOR smsc < PROTOCOL=SMPP ... SATPOOL_LOOKUP_IN=sat1 ROUTE=smtp-out1 ... > CONNECTOR smtp-out1 < PROTOCOL=SMTP INSTANCES=10 ADDRESS=#MX DOMAIN=example.com MAPPING=mappings/gsm-iso8859-out.map DEFAULT_MSGTYPE=EMAIL >
Inheritance and virtual connectors
When having a large configuration with many connectors there will be many connectors with similar configurations. In order to minimize configuration and make it more readable it is possible to let connectors inherit attributes from other connectors and also defining virtual connectors which are only used for inheritance and never instantiated themselves.
# Parent for all outgoing SMSC connectors CONNECTOR smsc-template < TYPE=OUTGOING INSTANCES=1 LOGMESSAGE=160 LOGPDU VIRTUAL > CONNECTOR smsc1 < INHERIT=smsc-template ADDRESS=10.0.0.1:5000 PROTOCOL=SMPP USER=user1 PASSWORD=secret > CONNECTOR smsc2 < INHERIT=smsc-template ADDRESS=10.0.0.2:5000 PROTOCOL=UCP USER=user1 PASSWORD=secret >
A connector can only inherit from one other connector but it is possible to have “inheritance chains” where connectors inherit in multiple levels. If a keyword is specified on both a parent connector as well as the child connector the keyword on the child connector is used.
Limiting connector queue sizes
By using the general keyword MAXTOTALQUEUESIZE it is possible to limit the total number of messages in the connector queues server-wide. This is useful when EMG is one of several components through which messages will pass and it is preferable not to let queues build in EMG. When the queue size limit has been reached EMG will reject further incoming messages using a, protocol specific, temporary error code, indicating that the client should check back later for a new try.
It is possible to use the IGNOREMAXTOTALQUEUESIZE keyword for a connector in order for it to accept messages even after the limit has been reached. This is useful for testing purposes as well as for priority connectors.
MAXTOTALQUEUESIZE does not impose an exact limit. It may be exceeded by a smaller amount of messages depending on current server conditions.
In EMG 5.2 MAXTOTALQUEUESIZE_SOFT was introduced providing a way to throttle messages softly by imposing a delay of 0.1 seconds when receiving a message and the specified queue size has been exceeded.
It is possible to define custom retry schemes through connector keyword RETRYSCHEME.
EMG distinguishes between connector and message errors. If an error occurs for a static connector and the connector is disconnected it will automically try to reconnect and if a number of reconnects fail it will go ERROR temporarily and then another set of reconnects will be performed.
For message errors the default is to terminate the message with status “failed” if an error is received in response to the send operation (for example SMPP operation “submit_sm”). This is not always the case, especially if the error code indicates a temporary error. One such error is SMPP error code ESME_RTHROTTLED (0x58) which indicates messages more frequently than allowed.
Sample retry scheme
Sample retry scheme entry to handle ESME_RTHROTTLED:
#Type (C=Connector, M=Message) and command (*=any) # Error code # Retrytime (applies to type C only) # Connects (applies to type C only) # Maxsleep (applies to type C only) # Hold delay (in seconds) # Flags (1:good message, 2:bad domain, 4:bad connector) M* 0x58 0 0 0 5 1
The above would cause a message, sent over the connector with any message operation, that is rejected with an error code of 0x58 to be requeued and put on hold for 5 seconds before next send attempt. The flag indicates the message is “good”, ie that the error is not related to the specific message.
Sample connector configurations.
Incoming MGP supporting up to 3 connections
CONNECTOR mgp-in1 < TYPE=INCOMING PROTOCOL=MGP ADDRESS=localhost:7185 # Up to 3 connections INSTANCES=3 # File where we find authentication info for users USERS=mgp-users >
Incoming SMPP supporting up to 10 connections
CONNECTOR smpp-in1 < TYPE=INCOMING PROTOCOL=SMPP ADDRESS=localhost:9000 # Up to 10 connections INSTANCES=10 # Remove prefixes and convert national swedish numbers # to international format REPLACEPREFIX_IN=+/,00/,0/46 REPLACEPREFIX_SOURCEADDR_IN=+/,00/,0/46 # File where we find authentication info for users USERS=smpp-users >
Incoming CIMD2 supporting 1 connection
CONNECTOR cimd2-in1 < TYPE=INCOMING PROTOCOL=CIMD2 ADDRESS=localhost:9000 # One connection only INSTANCES=1 # File where we find authentication info for users USERS=cimd2-users # Force messages to be routed to connector ebe1 ROUTE=ebe1 >
CONNECTOR smsc1-smpp < TYPE=OUTGOING PROTOCOL=SMPP # Connector to server 10.0.0.1 port 2775 ADDRESS=10.0.0.1:2775 # One instance is enough INSTANCES=1 # Username/password to use when authenticating USERNAME=smppuser PASSWORD=secret # Try to connect three times before considered connection failed MAXFAILEDCONNECTS=3 # When connection failed sleep for 5 minutes before trying again MAXFAILEDSLEEP=300 # If destination address does not start with "00", add it REQUIREPREFIX=00 # IDLETIMEOUT defaults to 30 seconds >
Outgoing UCP using authentication via operation 60
CONNECTOR smsc1-ucp < TYPE=OUTGOING PROTOCOL=UCP # Connector to server 10.0.0.1 port 5000 ADDRESS=10.0.0.1:5000 # One instance is enough INSTANCES=1 # Username/password to use when authenticating USERNAME=ucpuser PASSWORD=secret # Type of number, short number alias (used by auth operation 60) AUTHTON=6 # Number plan id, private (used by auth operation 60) AUTHNPI=5 # Do not timeout when idle IDLETIMEOUT=0 # Incoming messages should be routed to connector smpp-in1 ROUTE=smpp-in1 >
Outgoing UCP via modem
CONNECTOR smsc2-ucp < TYPE=OUTGOING PROTOCOL=UCP # Phone number to operator's modem pool ADDRESS=01122334456 # Modem is connected to /dev/ttyS0 (Linux?) MODEM=ttyS0 # Init string to send before dialing INIT_STRING=AT&F&K3 # One instance is all a tty can handle INSTANCES=1 # Wait 2 seconds after connect WAITDELAY=20 # Wait 2 seconds after message operation is sent MSGDELAY=20 # Operator only allows us to send 2 messages per session OPS_MAXPERSESSION=2 # If message does not contain a source address use this DEFAULT_SOURCEADDR=7654321 >
CONNECTOR http1 < TYPE=OUTGOING PROTOCOL=HTTP # We use full URL for HTTP ADDRESS=http://www.myhost.com/cgi-bin/handlemessage.sh # One instance is enough INSTANCES=1 # Timeout after 5 seconds idle IDLETIMEOUT=5 >
CONNECTOR ebe1 < TYPE=OUTGOING PROTOCOL=EBE # Program or script to execute ADDRESS=/usr/local/bin/ebe-handlesms.sh # One instance is enough INSTANCES=1 >
CONNECTOR gsm1 < # GSM connectors are always outgoing TYPE=OUTGOING PROTOCOL=GSM # TTY to which GSM device is attached MODEM=cua/a # Only one instance is allowed INSTANCES=1 # Poll for incoming messages every 30 seconds POLLRECEIVE=30 # No SCA in PDU, needed for some GSM devices #GSMNOSCA # Memory storage is used (Ericsson devices) #GSMSTORE=ME # Required for setting dest address of received messages FORCE_DESTADDR_IN=4670123123 >