Pre-Design Optimization

Is it ever sensible to consider optimization before building, or even designing, a solution?

One might think that, before considering optimization, one must first have some working solution to optimize. However, what if the design of the solution can be impacted by the speed with which computations are performed? What if the complexity of the design (and the implementation) is at least partly determined by the performance of its components?

Consider a simple example. Suppose we are building a web service to solve the following “use case”:

In order to reduce personal work load, a financial analyst wants a service that 
performs a comparative financial analysis on sets of related financial documents. 
The analysis should, broadly speaking, provide information about convergences and 
divergences between the documents in the following areas:
  - risk assessment 
  - valuation methodology
  - forecasts

Actually, from a certain perspective, the specific problem being solved is irrelevant.

We can thing of this problem more abstractly as follows: we are building a web (http) service that does some computation f on some client provided input x, and returns some result y.

As soon as we start thinking about how this solution might be implemented, we realize that the communication exchange (the protocol) and the complexity of the implementation will be constrained and determined by the amount of time f could take.

I like to consider extremes to make distinctions more evident.  On one end, f takes next to nothing to compute (say 0.01 ms), so the client makes the request to get y from x and receives a response immediately.  On the other end, f takes days to compute.

In the first case, the client can open a connection, get the response, and then close the connection.

In the second case, the client can’t be expected to make a request and wait with an open connection for days until the service responds. In this situation, something more complex is needed. Likely, the server would now have to 

1. Notify the client, somehow, somewhere, about job completion, on completion, or
2. Give the client, on job request, some token for the client to later ask for the status of the job, and eventually get the result.

In either case, the service now needs to be stateful and the communication protocol between service and client will be more involved.

Thus, in cases like this, optimizing a component before building the whole solution can simplify the overall design, implementation, and UX (user experience).