Setting up a dial-plan for dictations in Asterisk

Setting up a dictation system in FreePBX is a snap ... IF you're familiar with Asterisk and FreePBX.
I have spent 3 days toying around with it and finally setup an Asterisk based system with no prior experience.

First though, let me give out due kudos to the guys over at FreePBX.org for putting out a great product. Also check out their SipStation.com site for getting SIP trunking service. Since the creators of FreePBX are closely affiliated with SipStation.com it's no wonder that setting up your SipStation.com SIP account details into your FreePBX server is drop-dead-easy. Thank you guys!

So why was setting up an Asterisk dictation system so time consuming?

• Well for one thing, there is a lot of documentation to go through if you are a complete noob like me. I am a firm believer that one should read the documentation and ask questions if you're still stuck. The best book I have found to date for Asterisk is Practical Asterisk 1.4 and 1.6
• One of the opening quotes of that book states in part "...how much fun tinkering could be" which I whole-heartedly agree with. If you're noob like am, you're going to want to play with your FreePBX system for a few days get a feel of all it can do.
• For questions that you may post on various forums, dont always expect an accurate; fast or courteous reply, but dont let that discourage you from posting questions either.
• There are a lot of loose ends that one must string together in FreePBX to accomplish a complete dictation workflow. If you're new to the system, you're just going to have to get familiar with all the features and figure out what works best for you scenario. The reason there are so many loose ends is because it should be that way. There is no one-size-fits-all configuration for anything in Asterisk.
• Also, I have noticed that when one asks for assistance on the forums with regard to using the Dictate feature, everyone seems to asume that the users are part of the local network. If you are setting up a Asterisk dictation system where the users are calling in from a PSTN then you need to make that clear when you post your questions. It does make a difference when the users of your dictation system are not part of the local phone system.

That being said, my current solution simply involves having calls from an inbound route go to an IVR which I use as a "main menu" for the inbound calls and some other routing, until the user simply dials *34 to begin the dictation app. I would prefer if the dictation app started automatically, but I haven't quite figured that out yet.

The next tricky part is in the configuration of the [app-dictate-record] dialplan. Using my routing model above, the ${AMPUSER} variable is null and hence the default extension in that dialplan cannot work as provided by the FreePBX distro. The intention of how they have it coded is to have the dictation go into a folder which is named after the users extension. Hence, if the user from extension 101 uses the dictation feature, their dictations would go into folder 101. In my case, since the callers from my system are inbound over the PSTN, they are not authenticated as local users, hence ${AMPUSER} is null and all the dictations get dumped into the same folder. Also, the Dictate app asks the users to enter a file name, which is a really really bad idea for my solution, because if a user uses a filename they have used before, Dictate will over-write the existing one - YIKES!

To solve all these issues, I over-write the existing dictation dialplan with this one:

[app-dictate-record]
                                                                     

exten => *34,1,Answer

exten => *34,n,Macro(user-callerid,)
exten => *34,n,Noop(CallerID is ${AMPUSER})
exten => *34,n,Set(DICTENABLED=${DB(AMPUSER/${AMPUSER}/dictate/enabled)})
exten => *34,n,Set(CIDPRE=${CALLERID(name)})
exten => *34,n,Set(CIDPRE=${CIDPRE:0:2})
exten => *34,n,GotoIf($[$["x${DICTENABLED}"="x"]|$["x${DICTENABLED}"="xdisabled"]]?nodict:dictok)
exten => *34,n(nodict),Playback(feature-not-avail-line)
exten => *34,n,Hangup
exten => *34,n(dictok),Dictate(/var/lib/asterisk/sounds/dictate/${CIDPRE},${STRFTIME(${EPOCH},,%Y%m%d-%H:%M:%S)})
exten => *34,n,Macro(hangupcall,) 

I use the CIDPRE variable to capture the "CID Prefix" configured elsewhere in my routing model, and that corresponds with a specific user as selected through the IVR and I use an auto-generated date-time file name so that each filename will be unique. One of my main goals is to do as much of the configuring through the FreePBX GUI as possible. The code snippet above is the only thing I had to customize under the hood.

Done, now we're ready to begin testing!

[Update]
This is what I eventually came up with:

 ;tmsoa custom 1.2  
 [app-dictate-record-tmsoa]  
 exten => _.,1,Answer  
 ;exten => _.,n,SayDigits(1.2)  
 exten => _.,n,Macro(user-callerid,)  
 exten => _.,n,Set(CIDPRE=${CALLERID(name)})          ;capture the CID info  
 exten => _.,n,Set(CIDPRE=${CIDPRE:0:3})          ;get the first 3 characters of the CID prefix  
 ;set a file name that matches the date + time and the and then the paths  
 exten => _.,n,Set(DICTFILENAME=${STRFTIME(${EPOCH},,%y%m%d-%H.%M.%S)})  
 exten => _.,n,Set(DICTFILEPATH=/var/spool/asterisk/dictate)  
 exten => _.,n,Set(DICTFILEPATHRAW=${DICTFILEPATH}/raw/${CIDPRE})  
 exten => _.,n,Set(DICTFILEPATHWAV=${DICTFILEPATH}/wav/${CIDPRE})  
 ;make sure the audio file directories exist  
 exten => _.,n,System(mkdir -p ${DICTFILEPATHRAW})  
 exten => _.,n,System(mkdir -p ${DICTFILEPATHWAV})  
 exten => _.,n,System(mkdir -p ${DICTFILEPATHRAW}/.archive)  
 ;authenticate with VM pwd in MB matching CID prefix  
 exten => _.,n,VMAuthenticate(${CIDPRE}@default)  
 ;record dictation to folder matching CID prefix with the file name in date-time format  
 exten => _.,n,Dictate(${DICTFILEPATHRAW},${DICTFILENAME})  
 ;** DO NOT ADD ANYTHING AFTER THIS DIAL-PLAN SINCE IT  
 ;** WILL JUST BE IGNORED IF THE CALL IS DROPPED/HUNGUP  

Note:
I use the "SayDigits(1.2)" on development hosts to indicate the version number for dial plans I am working on. This gets commented out befor deployment.

Convert Asterisk dictation .raw files to .wav

I recently implemented an Asterisk solution and made use of the Dictate application which creates .raw audio files of the Dictations. These files need to be encoded to wav format even though specifying in FreePBX, that the output of dictated files should be WAV files. When researching how to do the conversion I did find various examples of using the sox app to convert these files, however none of these examples worked exactly as documented. Some documentation was incomplete and others were outdated an often, the forums which raised the question, went unanswered.

For the version of FreePBX which I am using, I found the following to command sequence to work:

sox -t auto -w -s -r 8000 -c 1 {input_file}  {output_file}

An excellent way to increase the volume, is to add the -v option with a multiplier to increase the volume by. This is especially helpful in our case since we have a few doctors that like to dictate very softly.

I hope this information is helpful to anyone else that has run into this issue.

To bubble exceptions or not to bubble exceptions. That is the question.

The phrase "exceptions must be truly exceptional" is thrown around a lot, but I find that many developers can’t fully understand what that actually means or how to apply that information. They only think they do.

For example, I had a developer argue that an invalid date format is not truly an exception, that it is "simply a MINOR user data entry issue, which could happen often", therefore there was no acceptable case for bubbling, or even throwing an exception for that. He said that having a service go down is a real example of an exception, because that would be a BIG DEAL, and that it rarely occurred.

His distinction was that a web service going down is "rare and exceptional" (true in our case, but it has happened more than once) and that it was a "BIG DEAL", hence the cost of bubbling an exception was acceptable because one would rarely have to pay that price, but at least you're paying the price for something that's EXCEPTIONALLY important and that a user data entry problem is not an exceptionally important problem in the system.
That kind of answer tells me that the developer defines "exceptional" only as something that's SUPERIOR and hence in his mind the issue of a web service going down is superior in importance to the a data entry issue. The developer was ignoring the fact that "exceptional" primarily means "rare".

I had him investigate our logs to see the frequency at which an invalid date format issue, made it through all the layers of our app and he found that it had only happened once, and it never made it past the first service layer.

Now consider the real "expense" between a service going down and a user data entry issue:
The aforementioned service which had an issue that had caused an exception to bubble was 4 layers down in the app stack, and it occurred for hundreds of transactions in the app because of all the users making requests to that area of the app all at the same time. That means that we had hundreds of exceptions bubbling through to the UI, through 4 layers of the app. Also, this happened at least twice in a twelve month period.

In the case of the invalid date issue, this occurred because the input validation did not work in the web app (because the user had JavaScript turned off). The next layer of the app received the bad data, and according to my policy, went no further, but instead bubbled up the exception (just one layer up to the UI). Only IF the first service layer missed that the data provided by the user was invalid, would it ever have gotten to the 2nd layer, which would have to throw an exception to bubble up. Since it was very RARE for invalid data entry to get past the UI, has never before got past the first service layer, it was clear that the data entry error was more RARE than the issue of the web service going down, hence in terms of defining exceptional as RARE, the data entry issue was more exceptional than the web service going down. In fact this is true for ANY data entry errors in our apps (as it should be for any ones apps).

So exceptions bubbled by a service going down occurred with much greater frequency, but it’s also important to note that it went through more layers of the app compared with the data entry error. Hence in terms of actual resource EXPENSE, the magnitude of resource cost on our servers for throwing exceptions for data entry errors, is entirely NEGLIGIBLE in this case when compared with the resource expense incurred for a web service going down.
His rebuttal was that an isolated data entry error is negligible compared to the IMPACT of critical service not being available at all. His statement is true and irrefutable; however that is not the point. In fact, that’s the wrong way to look at the issue entirely.

Good quality standards must also dictate that at no point should a developer minimize the importance of the elementary components of an application (such as data entry), simply because their related issues are “low impact”. When one minimizes importance of the elementary components such as data entry validation, simply because they are “low impact”, you end up with COMMON issues in your application for which you would never throw exceptions, because now, you treat them as “common” and “low-impact”. Essentially you’ll end up with spaghetti code where you’re coding to mitigating un-exceptional issues wherever you might encounter them. This approach is unintuitive and highly counter productive.

To me, anything which fails a basic quality standard must be treated with priority, because if you can’t even maintain good standards for the basics, then your whole app is rubbish. Therefore, I would argue that everything should be treated as an exception, simply because problems should be EQUALLY RARE at every level, and when any exception is rare, then expense is not an issue.

If someone you know is creating apps that have so many issues that they have to start worrying about the expense of all the exceptions all over the place in all the applications various layers, then it’s probably time to suggest they scrap their app and start over.

My point is this: if you treat even minor, common issues in your applications as exceptionally important then they will become exceptionally rare, in which case it would be valid and inexpensive to treat them as exceptions.