OS X Mountain Lion Webserver SSL nightmare

I'm having an issue with OS X Webserver that I'm trying to figure out with Apple. Apparently many other people have also been having SSL issues when using Server.app, since before Mountain Lion.

Read more: https://discussions.apple.com/thread/4299762

Setting up a dial-plan for dictations in Asterisk

Setting up a dictation system in FreePBX is a snap ... IF you're familiar with Asterisk and FreePBX.
I have spent 3 days toying around with it and finally setup an Asterisk based system with no prior experience.

First though, let me give out due kudos to the guys over at FreePBX.org for putting out a great product. Also check out their SipStation.com site for getting SIP trunking service. Since the creators of FreePBX are closely affiliated with SipStation.com it's no wonder that setting up your SipStation.com SIP account details into your FreePBX server is drop-dead-easy. Thank you guys!

So why was setting up an Asterisk dictation system so time consuming?

• Well for one thing, there is a lot of documentation to go through if you are a complete noob like me. I am a firm believer that one should read the documentation and ask questions if you're still stuck. The best book I have found to date for Asterisk is Practical Asterisk 1.4 and 1.6
• One of the opening quotes of that book states in part "...how much fun tinkering could be" which I whole-heartedly agree with. If you're noob like am, you're going to want to play with your FreePBX system for a few days get a feel of all it can do.
• For questions that you may post on various forums, dont always expect an accurate; fast or courteous reply, but dont let that discourage you from posting questions either.
• There are a lot of loose ends that one must string together in FreePBX to accomplish a complete dictation workflow. If you're new to the system, you're just going to have to get familiar with all the features and figure out what works best for you scenario. The reason there are so many loose ends is because it should be that way. There is no one-size-fits-all configuration for anything in Asterisk.
• Also, I have noticed that when one asks for assistance on the forums with regard to using the Dictate feature, everyone seems to asume that the users are part of the local network. If you are setting up a Asterisk dictation system where the users are calling in from a PSTN then you need to make that clear when you post your questions. It does make a difference when the users of your dictation system are not part of the local phone system.

That being said, my current solution simply involves having calls from an inbound route go to an IVR which I use as a "main menu" for the inbound calls and some other routing, until the user simply dials *34 to begin the dictation app. I would prefer if the dictation app started automatically, but I haven't quite figured that out yet.

The next tricky part is in the configuration of the [app-dictate-record] dialplan. Using my routing model above, the ${AMPUSER} variable is null and hence the default extension in that dialplan cannot work as provided by the FreePBX distro. The intention of how they have it coded is to have the dictation go into a folder which is named after the users extension. Hence, if the user from extension 101 uses the dictation feature, their dictations would go into folder 101. In my case, since the callers from my system are inbound over the PSTN, they are not authenticated as local users, hence ${AMPUSER} is null and all the dictations get dumped into the same folder. Also, the Dictate app asks the users to enter a file name, which is a really really bad idea for my solution, because if a user uses a filename they have used before, Dictate will over-write the existing one - YIKES!

To solve all these issues, I over-write the existing dictation dialplan with this one:

[app-dictate-record]
                                                                     

exten => *34,1,Answer

exten => *34,n,Macro(user-callerid,)
exten => *34,n,Noop(CallerID is ${AMPUSER})
exten => *34,n,Set(DICTENABLED=${DB(AMPUSER/${AMPUSER}/dictate/enabled)})
exten => *34,n,Set(CIDPRE=${CALLERID(name)})
exten => *34,n,Set(CIDPRE=${CIDPRE:0:2})
exten => *34,n,GotoIf($[$["x${DICTENABLED}"="x"]|$["x${DICTENABLED}"="xdisabled"]]?nodict:dictok)
exten => *34,n(nodict),Playback(feature-not-avail-line)
exten => *34,n,Hangup
exten => *34,n(dictok),Dictate(/var/lib/asterisk/sounds/dictate/${CIDPRE},${STRFTIME(${EPOCH},,%Y%m%d-%H:%M:%S)})
exten => *34,n,Macro(hangupcall,) 

I use the CIDPRE variable to capture the "CID Prefix" configured elsewhere in my routing model, and that corresponds with a specific user as selected through the IVR and I use an auto-generated date-time file name so that each filename will be unique. One of my main goals is to do as much of the configuring through the FreePBX GUI as possible. The code snippet above is the only thing I had to customize under the hood.

Done, now we're ready to begin testing!

[Update]
This is what I eventually came up with:

 ;tmsoa custom 1.2  
 [app-dictate-record-tmsoa]  
 exten => _.,1,Answer  
 ;exten => _.,n,SayDigits(1.2)  
 exten => _.,n,Macro(user-callerid,)  
 exten => _.,n,Set(CIDPRE=${CALLERID(name)})          ;capture the CID info  
 exten => _.,n,Set(CIDPRE=${CIDPRE:0:3})          ;get the first 3 characters of the CID prefix  
 ;set a file name that matches the date + time and the and then the paths  
 exten => _.,n,Set(DICTFILENAME=${STRFTIME(${EPOCH},,%y%m%d-%H.%M.%S)})  
 exten => _.,n,Set(DICTFILEPATH=/var/spool/asterisk/dictate)  
 exten => _.,n,Set(DICTFILEPATHRAW=${DICTFILEPATH}/raw/${CIDPRE})  
 exten => _.,n,Set(DICTFILEPATHWAV=${DICTFILEPATH}/wav/${CIDPRE})  
 ;make sure the audio file directories exist  
 exten => _.,n,System(mkdir -p ${DICTFILEPATHRAW})  
 exten => _.,n,System(mkdir -p ${DICTFILEPATHWAV})  
 exten => _.,n,System(mkdir -p ${DICTFILEPATHRAW}/.archive)  
 ;authenticate with VM pwd in MB matching CID prefix  
 exten => _.,n,VMAuthenticate(${CIDPRE}@default)  
 ;record dictation to folder matching CID prefix with the file name in date-time format  
 exten => _.,n,Dictate(${DICTFILEPATHRAW},${DICTFILENAME})  
 ;** DO NOT ADD ANYTHING AFTER THIS DIAL-PLAN SINCE IT  
 ;** WILL JUST BE IGNORED IF THE CALL IS DROPPED/HUNGUP  

Note:
I use the "SayDigits(1.2)" on development hosts to indicate the version number for dial plans I am working on. This gets commented out befor deployment.

Convert Asterisk dictation .raw files to .wav

I recently implemented an Asterisk solution and made use of the Dictate application which creates .raw audio files of the Dictations. These files need to be encoded to wav format even though specifying in FreePBX, that the output of dictated files should be WAV files. When researching how to do the conversion I did find various examples of using the sox app to convert these files, however none of these examples worked exactly as documented. Some documentation was incomplete and others were outdated an often, the forums which raised the question, went unanswered.

For the version of FreePBX which I am using, I found the following to command sequence to work:

sox -t auto -w -s -r 8000 -c 1 {input_file}  {output_file}

An excellent way to increase the volume, is to add the -v option with a multiplier to increase the volume by. This is especially helpful in our case since we have a few doctors that like to dictate very softly.

I hope this information is helpful to anyone else that has run into this issue.

To bubble exceptions or not to bubble exceptions. That is the question.

The phrase "exceptions must be truly exceptional" is thrown around a lot, but I find that many developers can’t fully understand what that actually means or how to apply that information. They only think they do.

For example, I had a developer argue that an invalid date format is not truly an exception, that it is "simply a MINOR user data entry issue, which could happen often", therefore there was no acceptable case for bubbling, or even throwing an exception for that. He said that having a service go down is a real example of an exception, because that would be a BIG DEAL, and that it rarely occurred.

His distinction was that a web service going down is "rare and exceptional" (true in our case, but it has happened more than once) and that it was a "BIG DEAL", hence the cost of bubbling an exception was acceptable because one would rarely have to pay that price, but at least you're paying the price for something that's EXCEPTIONALLY important and that a user data entry problem is not an exceptionally important problem in the system.
That kind of answer tells me that the developer defines "exceptional" only as something that's SUPERIOR and hence in his mind the issue of a web service going down is superior in importance to the a data entry issue. The developer was ignoring the fact that "exceptional" primarily means "rare".

I had him investigate our logs to see the frequency at which an invalid date format issue, made it through all the layers of our app and he found that it had only happened once, and it never made it past the first service layer.

Now consider the real "expense" between a service going down and a user data entry issue:
The aforementioned service which had an issue that had caused an exception to bubble was 4 layers down in the app stack, and it occurred for hundreds of transactions in the app because of all the users making requests to that area of the app all at the same time. That means that we had hundreds of exceptions bubbling through to the UI, through 4 layers of the app. Also, this happened at least twice in a twelve month period.

In the case of the invalid date issue, this occurred because the input validation did not work in the web app (because the user had JavaScript turned off). The next layer of the app received the bad data, and according to my policy, went no further, but instead bubbled up the exception (just one layer up to the UI). Only IF the first service layer missed that the data provided by the user was invalid, would it ever have gotten to the 2nd layer, which would have to throw an exception to bubble up. Since it was very RARE for invalid data entry to get past the UI, has never before got past the first service layer, it was clear that the data entry error was more RARE than the issue of the web service going down, hence in terms of defining exceptional as RARE, the data entry issue was more exceptional than the web service going down. In fact this is true for ANY data entry errors in our apps (as it should be for any ones apps).

So exceptions bubbled by a service going down occurred with much greater frequency, but it’s also important to note that it went through more layers of the app compared with the data entry error. Hence in terms of actual resource EXPENSE, the magnitude of resource cost on our servers for throwing exceptions for data entry errors, is entirely NEGLIGIBLE in this case when compared with the resource expense incurred for a web service going down.
His rebuttal was that an isolated data entry error is negligible compared to the IMPACT of critical service not being available at all. His statement is true and irrefutable; however that is not the point. In fact, that’s the wrong way to look at the issue entirely.

Good quality standards must also dictate that at no point should a developer minimize the importance of the elementary components of an application (such as data entry), simply because their related issues are “low impact”. When one minimizes importance of the elementary components such as data entry validation, simply because they are “low impact”, you end up with COMMON issues in your application for which you would never throw exceptions, because now, you treat them as “common” and “low-impact”. Essentially you’ll end up with spaghetti code where you’re coding to mitigating un-exceptional issues wherever you might encounter them. This approach is unintuitive and highly counter productive.

To me, anything which fails a basic quality standard must be treated with priority, because if you can’t even maintain good standards for the basics, then your whole app is rubbish. Therefore, I would argue that everything should be treated as an exception, simply because problems should be EQUALLY RARE at every level, and when any exception is rare, then expense is not an issue.

If someone you know is creating apps that have so many issues that they have to start worrying about the expense of all the exceptions all over the place in all the applications various layers, then it’s probably time to suggest they scrap their app and start over.

My point is this: if you treat even minor, common issues in your applications as exceptionally important then they will become exceptionally rare, in which case it would be valid and inexpensive to treat them as exceptions.

Passing variables to SubReports in Telerik Reporting

Anytime you have created a complex report with embedded sub-reports, you might have faced the need to pass a variable from data in the the master report to a sub-report. I have seen this problem solved various ways with differing degrees of unnecessary complexity.

The biggest mistake made in all the code I have seen, is to assume that one must pass the variable through existing mechanisms provided by the Telerik Report class. I think this reply from Steve @ Telerik sums it up best: "Telerik Reports are standard .NET classes so you can abstract yourself from the fact that you're working with reports and achieve your requirement". In other words, use .Net and standard OOP principals without relying on being spoon-fed every feature your project requires.

One of the more interesting solutions I have seen implemented was this one, which was probably not intended as a solution for passing parameters from reports to sub-reports, but which the developer managed to getting working anyhow.

Basically, the common approach people use is to pass the variable using the report parameters collection, which while not wrong, simply takes more code than is necessary.

I find that the most graceful solution is possible when does what Steve recommends and abstracts oneself from the fact that your're working with reports and remember that you're working with standard .Net classes. So what would you do if you needed to get a variable into a instantiated class. Well, you could give the class a property, but even then you have to instantiate the class, and then set the property. So the other solution is to add the variable as a parameter to the class constructor. "But when I create a report it provides the following code where there is not parameter for my variable!" ... I have been told. What developer was talking about was this constructor created when you add a Telerik Report to a C# project:


        public AccountAllocationsSub()
        {
            InitializeComponent();
        }

True, that constructor has no parameters, but if you understand that you are just working with a standard .Net class, then there' reason you cant create your own constructor which takes the parameter you want to pass. Consider this constructor:


       public AccountAllocationsSub(int transactionKey)
        {
            InitializeComponent();
            this.sqlDataSource1.ConnectionString = SqlConnectionString;
            this.sqlDataSource1.Parameters["@TransactionKey"].Value = transactionKey;
            DataSource = sqlDataSource1;
        }

Now all you need to do is get the data from the master report when it's binding to the controls, and invoke your sub report with your custom constructor:


     private void TransactionGroupHeader_ItemDataBinding_1(object sender, EventArgs e)
     {
            Telerik.Reporting.Processing.GroupSection section =
                (sender as Telerik.Reporting.Processing.GroupSection);
            int transactionKey = 0;
            string strTransactionKey = section.DataObject["TransactionKey"].ToString();
            int.TryParse(strTransactionKey, out transactionKey);
            subReport1.ReportSource = new AccountAllocationsSub(transactionKey);
      }


The biggest advantage I have found using this method is that when I use the preview tab to preview the report, it will call the standard constructor, and I can have that constructor populate the report based on the properties I set for the data object at design time and nicely preview the report.

Then at run time when the report is called from the master report, it will use the variable parameter for the master report and the data the other constructor will set the data based on the variable from the master report.

With the solution developed previously by another developer on our team there was no way to preview the data on the report. In fact using the code from the sample he followed the data-source was specifically neutered unless one was viewing the report at runtime. This was the code in the standard ctor (which is called by the previewer):


public MyTelerikMasterReport()
{
// Required for Telerik Reporting designer support
InitializeComponent();
this.DataSource = null;
if (System.ComponentModel.LicenseManager.UsageMode ==
             System.ComponentModel.LicenseUsageMode.Runtime)
this.SqlDataSource1.ConnectionString = ...etc
}

Getting control of your Telerik Reports

Recently I took over a reporting project from another developer. We basically have multiple reports that had to be ported to Telerik Reports (ver 2011 Q3). Upon inspecting what he had created I noticed the following problems throughout the project:

1. He implemented 4 different ways of connecting to the DB instead of one standardized method for all reports [no, there was no good reason for doing what he did, one standardized method would work for all in our situation].
2. He implemented 3 different ways of retrieving data, using 3 different types of objects instead of standardizing. One method used a standard .Net data object that would attempt to lock all the records in the DB while report was loading, which created major headaches for reports that took a while to generate.
3. He used in-line SQL in the DLL - yes, I know, there are different schools of thought on this subject, but my team has a specific policy that dictates that developer's applications only use stored procedures and this is considered good practice in many circles.
4. He created a query for each field in the report header that had to retrieve common header data.
5. In every report, he had code for retrieving report footer data for stuffing into a text box in the footer of the report.
6. He had no standard convention for naming parameters which were used in most reports. For example, most reports would require a "customerId" parameter, yet the variable would be coded as any one of the following (as an example): cust / customerID / custId / cusomterAccountKey / accountKey / customerAccountNumber -- I guess some people like variety.

To mitigate some of these issues I simply created a class named "StandardFormatReport" which inherits from Telerik.Reporting.Report:

public class StandardFormatReport : Telerik.Reporting.Report

To mitigate the first issue, I added a property to the StandardFormatReport which would return the one and only connection string the app is supposed to use according to the one and only approved method for retrieving connection strings:

        protected string SqlConnectionString

        {

            get { return Conf.GetDBConnectionString(); }

        }

In order to make sure that I can more easily know where to look to audit how the report is setting the data source, I added the following virtual function, which the developer would have to override in his report:

        protected virtual void SetDataSource()

        {

            throw new System.Exception("The report class which inherits from RRFS.Reports.StandardFormatReport must override SetDataSource()");

        }

All I would need to do in a code review process is look for where the developer would override the function to set the datasource, and make sure that he was connecting to the database correctly:

        protected override void SetDataSource()

        {

            this.sqlDataSource1.ConnectionString = this.SqlConnectionString;

            ....

        }

Lastly, the SetDataSource function is the logical place to set the reports parameters, which the developer named every different way under the sun with a string. This drives me nuts, not just because of the lack of consistency, but also because the developer could easily typo the param name and the report would not work. I deplore the using strings where constants should be used instead. To remedy this, I added the necessary constants which would define the standardized way to name params, in the StandardFormatReport class:

        protected static class StandardParams

        {

            public const string CollectionAccountKey = "@CustomerAccountId";

            public const string AssociationKey = "@CompanyId";

        }

Thus instead of the developer using the following code:

        protected override void SetDataSource()

        {

            this.sqlDataSource1.ConnectionString = this.SqlConnectionString;

            this.sqlDataSource1.Parameters.AddRange(new Telerik.Reporting.SqlDataSourceParameter[] {

            new Telerik.Reporting.SqlDataSourceParameter("@CollectionAcountKey", System.Data.DbType.Int32, _collectionKey)});

            this.DataSource = sqlDataSource1;

        }

[notice, I deliberately spelled the parameter incorrectly in this case, because this is highlights the typo issue]

... he would have to use the following code:

        protected override void SetDataSource()

        {

            this.sqlDataSource1.ConnectionString = this.SqlConnectionString;

            this.sqlDataSource1.Parameters.AddRange(new Telerik.Reporting.SqlDataSourceParameter[] {

            new Telerik.Reporting.SqlDataSourceParameter(StandardParams.CollectionAccountKey, System.Data.DbType.Int32, _collectionKey)});

            this.DataSource = sqlDataSource1;

        }

You cant typo that and get your app to compile.

Finally, for all the instances of reports that had certain repeated fields that the developer had coded into each report, I added the following function to the StandardFormatReport class:

        public void SetStandardFields()

        {

            Telerik.Reporting.TextBox tb = null;

            

            tb = this.Items.Find("txtFooterDate", true)[0] as Telerik.Reporting.TextBox;

            if (tb != null) tb.Value = "Information as of " + DateTime.Now.ToString("mm/dd/yy");

            tb = this.Items.Find("txtFooterNote", true)[0] as Telerik.Reporting.TextBox;

            if (tb != null) tb.Value = GlobalFunctions.GetFooterNote();

            tb = this.Items.Find("txtFooterAddress", true)[0] as Telerik.Reporting.TextBox;

            if (tb != null) tb.Value = GlobalFunctions.GetFooterAddress();

        } 

Then in the ctor for the StandardFormatReport, I simply add the call to SetStandardFields, and if they exists with the correct names, they will be set, this eliminating the need to have this coded in every single report -- standard DRY compliance issues.

There is more that one could do to put standard functionality into a class that derives from Telerik.Reporting.Report and enforce standardization through everyday OO programming, but I wont go into detail. The point is this: just because the Report Wizard will create a Telerik.Reporting.Report class for you in Visual Studio, it does not mean that you have to use it as is. It's still basically a C# object, subject to all the benefits that come with the OOP feature C# offers. Since most reports for business applications usually need common functionality, it only makes sense to put all that functionality into one custom-class that derives from Telerik.Reporting.Report, then have your developers create their reports by deriving from your custom-class.

I would argue that if you have a large reporting project where there are many typical elements (a common logo, same size, position, etc) required on just about every report, and you're not doing it this way, but repeating your code per report instead of inheriting from a class that already has the correct code, then you're doing it wrong.

So you think you know Software Engineering?

Someone emailed me this. And I have to say I agree with it:

Common problems in the business world (from what I have experienced):

*everyone* thinks they are a developer/software designer
*everyone* thinks they could run IT and the software staff (and thinks they should!)
*everyone* thinks that innovative solutions can be scheduled, and thus can have predictable end dates.

I have to add to that however, that in my experience, none of these people can even do job #1, which is to say, document properly.