The Reverse What-if: more hard-won insight into OR practice

The real fun in O.R practice is when your optimization product (say a recommendation system) is evaluated by it's first customer. Usually this is the point where you throw away the last 6 months of your textbook work and build something that your customer can actually use rather than what you wanted to build.

If you are in the business of providing software-as-a-service, you can do several iterations with a customer and that practice is more forgiving, but i am learning the hard way that building an air-tight OR product and then handing it off to a customer is mighty hard and you only have one or maybe two chances to get it right.

(note: some dramatic license exercised for illustrative purposes)
Today's tab focuses on the 'what-if', the practice of walking a customer thru a choreographed sequence of optimization scenarios. An obvious choice is to start with a somewhat unconstrained model, say, only having boxed variables, and then show the user or the consultant how the solution changes as you throw in, one by one, all those beautiful constraints your system can handle. At the end of this process, if everything goes well, you end up with a real and optimized solution that business can use and will save them many $$.

Unfortunately, this approach of "increased complexity" rarely works. The procedure described above is nice if you are trying to teach a student about OR. The average business user doesn't really care about this. He/She probably understands real-life solutions much better than you. She/we wants your system to work well and maybe help get a bonus at the end of the fiscal year. In fact, you are better of looking at its dual, the reverse-what if.

The reverse what-if, as you would guess, starts with the best approximation of the customer's reality and then you navigate from there. Otherwise, the customer has little patience for all the n-1 steps in the 'straight' what-if which represent imaginary solutions he doesn't care about. The n'th step brought him to reality, which he already understands better than you, so you haven't helped him much.

The reverse what-if perfectly ties in with Rosenthal and Brown's practical philosophy of starting with a legacy solution. Your recommendation system should be able to represent the legacy recommendation, else your product is not going to earn your customer's vote. Even if they purchased it, they may end up just using your GUI and throw your analytics into the trash. Unless you are providing an order of magnitude improvement in business value (in which case, you should start your own company!), revolutionary solutions are a hard sell. Yes, a customer values seemingly non-intuitive solutions that your OR unlocks to make him more money, but to truly ensure user-adoption, you need demonstrate this improvement from the legacy solution.

It also goes to user experience. In fact, it's about avoiding home-sickness. Being able to return to something familiar is critical for user acceptance. How many times have we initially cursed websites who suddenly redesigned and changed everything and you cant find your favorite menus in the same place? Windows OS always provides us the option of returning to something that resembles their previous release, even though they know they fixed a million bugs since their prior release. And we gradually outgrow the old if the new performs as advertised. I still use my favorite music player Winamp's classic skin. Feels like home.

Comments welcome.