Nice analysis.

(1) I did in fact notice this, and thought about loading column 2 into a second array, so that any blanks in my output array did not overwrite any values in that column, but it was getting late….

(2) Not true. I can test the element of the array, then bold the corresponding cell in the sheet. This still minimizes interaction with the worksheet (though of course reading a value is much faster than changing a format).

(3) I assumed this, and was going to run a nice model to demonstrate, but the data didn’t show the nice behavior I’d expected.

(4) Interface matters. I don’t kill myself optimizing a two second routine if the user spends five minutes thinking about what the analysis means.

(5) I like this approach, depending on what the “do something” means. If you put a formula in place that is based on column 1, it updates from that point on without rerunning the code.

(7) “Optimization is rarely as easy as people make it out to be.” Yes sir. To me, optimization is about making something modular so I can reuse it with little fuss, and making it self-documenting so I can remember how it works quickly. The whole array thing works for me as for you, because I have a set of library routines that process arrays, and supporting routines that do range-array and csv-array and similar translations. Optimization for my projects is more about streamlining development processes rather than code execution processes.

Thx,
JP

(1) Since Dick did not output the result of his “do something,” he missed something that Jon ought to have caught with his version of the code. The code using the variant=range.value and range.value=variant is *not* functionally identical to the one that loops through the cells one at a time! The variant version replaces all the cells in the output column whereas the cell-at-a-time code only updates those cells where the If test yields True!

(2) The var=range.value method works only if we want to work with data and not with the Excel object. So, if Dick wanted to change the font color or the background color or, as Andy did, bold the cells, this method would not work.

(3) Optimizations are not always data-independent. I created 2 data sets. The first consisted of 65536 cells in column A with the formula =IF(MOD(ROW(),100)=0,”abc”&ROW(),”Total”&ROW()). This yields a lot of cells starting with “Total”. I also created the “inverse” set with the formula =IF(MOD(ROW(),100)<>0,”abc”&ROW(),”Total”&ROW()) This of course, had only 1 out of every 100 rows starting with “Total”

I also reported the results not just in total time but also on a “per-cell” basis. My results were

So, while the variant=range.value code is always faster than the cell-at-a-time code, the benefit varies based on the data set.

(4) All the different samples relied on testing just the interface without accounting for what the core “do something” code does. Let’s use the above numbers to establish a baseline. If the core “do something” code runs much faster than 1.2E-5 to 7.2E-4 seconds, the interface does matter! However, the longer that the core code takes, the less the interface matters.

Some time ago, I was asked by a client to improve the performance of some code. It used several PivotTables to analyze the raw data and then extracted and organized the data from the resulting PivotTables into tables and charts that the client wanted. The client told me their IT group was overloaded and couldn’t work on the problem but the IT person was certain the problem was with the tables and charts that were being created.

The first thing I did was add a bunch of timing code to find where the problem was. It turned out the original code created a PT, used the data from the first row, and then ‘hid’ that PT element. The last part, i.e., hiding the element, could, under certain circumstances, take a long time. So, I rewrote the code to simply step through each row of the PT and performance improved by a magnitude!

Contrary to the IT expert, the performance problem had nothing to do with the number of charts or updating the charts or whatever.

My point about the story is preemptive (and premature) decisions about optimization may often be nothing but a waste of time.

(5) Using other algorithms: Data independent performance. Can we make our code performance independent of data? Given that we are dealing with Excel, yes, within limits. If we want to add something in an adjacent column (as Jon did in his example), we can use

We could also add conditional formatting, etc., though we would have a hard time working with the entire row.

How well does the above code perform? With only a few cells starting with “Total”, it took 0.25 seconds and with lots of cells starting with “Total”, it took 0.27 seconds. Clearly, it is almost totally data-independent. Not to mention it is easy to write, understand, and maintain! And, as a bonus, it is very fast!

(6) Using other algorithms: Convenience. How about an algorithm that provides programming convenience? I used the FindAll method I wrote some time ago (http://www.tushar-mehta.com/excel/tips/findall.html)

Note that this is the first algorithm that does exactly what the original code did. It only updates those cells where the adjacent cell starts with “Total.” So, in that sense this is the only truly comparable method! It also retains the ability to work with the Excel object.

How well does it perform. Well, with a lot of cells starting with “Total” it took about 56 seconds and with a few cells it took about 0.6 seconds. So, it is faster than the cell-by-cell method by just under 25%.

Summary: Optimization is rarely as easy as people make it out to be. Most people focus on micro-optimizations that are rarely truly beneficial. Optimizations can also be such that they don’t truly duplicate the functionality of the original code. Finally, optimizing without understanding the true nature of the bottleneck doesn’t always yield the expected benefits, not to mention that it may take away some of the flexibility embedded in the original code.

You are absolutely correct,
apologies to fzz …

As fzz wrote

“If InStr(1, c.Text, s2m) = 1 Then”

Only checks passes for “Total” at the left. I too tend to use

If InStr() Then

but fzz was more specific.

…mrt

Some minor quibbles with other solutions:

- I almost never use .Text because it can return #### if the column size is too small or the user may have changed the formatting(and .Text can do funny things in UDFs)
- If Left$(vArr(i, 1), Len(sFIND)) = sFIND Then does not work if any of the text is shorter than sFind
- Using Instr does not necessarily give the same solution as Left or Like Total* because it will also find SubTotal for example

(yes, I had a reason for using a variant in the first place(I hope!))

Does anyone else get that?