"Testing, schmesting"
The Placebo Effect is in Full Swing in Rob's Latest 1DMKIII Update An analysis in search of a problem
"When I first got an EOS-1D Mark II back in the summer of 2004, I immediately became a better sports photographer. While I was still the same guy looking through the viewfinder, the camera's ability to get so many of the key frames in focus, and get so many frames in extended sequences in focus too, meant I was immediately producing better moments, and more of them. "
Rob Galbraith seems to hold the Canon 1D MKII series cameras in very high regard indeed. The original Canon 1D MKII made him a "better sports photographer."
Rob started out this most recent article by asking the following question.
"Does the Canon EOS-1D Mark III now offer reliable autofocus?" The answer is- compared to what?
Is he comparing Canon against Nikon? No. Because, if he did
he would find that the current Canon AF system still outperforms the best Nikon has to offer. He notes in the current article that he has a Nikon D3. But, he doesn't seem to be using it much for sports. No, he is comparing a new Canon to an old Canon. One that made him become a "better sports photographer."
Fair enough, I say. Let's pit Canon against Canon. So,
we did. The Canon 1D MKIII repeatedly outperformed the Canon 1D MKII in our scientific testing.
His beliefs about the root cause of the problem(s) and his interpretation of results has often been very different. Let's look at some of his original hypotheses vs. current statements. Below are two samples of his initial statements regarding the AF problems contrasted with his current assessments.
Then: "It means that when the light is especially bright and the temperature is warm, the camera's autofocus performance drops like a stone." Now: "First, while we identified bright, sunny conditions as being the environment that tripped the camera up, what's now clear is that the tendency, with v1.1.3, to overestimate subject speed is present in more types of light than just sunlight; the only thing that changes with the light is the degree to which the problem makes itself apparent."
No more mention of heat as being the culprit. He is still trying to work in the bright light theory by expanding it to include all light.
Then: "Under certain conditions, the EOS-1D Mark III has difficulty acquiring focus initially." Now: "For any type of shooting that involves composing, autofocusing and tripping the shutter all in one quick motion - such as trying to snare a crisp photo of a diving volleyball player - this camera delivers the goods."
Now the camera (with the same firmware as referenced above) has no difficulty acquiring initial focus.
Having done an exhaustive array of our own tests throughout this entire episode. I can now hazard a guess as to why Canon may have taken the following action to cease their testing collaboration with Rob. And it is not just because they didn't like his results. It is because his results don't square with solid scientific testing.
"Canon USA abruptly ended our EOS-1D Mark III testing collaboration."
You can easily see from the following statement how readily suspect his testing methods are.
"After a few weeks of shooting with the Mark III cameras and the new firmware, poring over the photos, it was becoming apparent that I was losing my frame of reference, losing my sense of what good action autofocus is." "This prompted two changes: additional side-by-side shooting involving the EOS-1D Mark II N and, more importantly, reviewing a portion of the tens of thousands of sports photos we've shot with Mark II cameras in the last four years. This helped set things straight. If you put aside all the this-camera-does-this and that-camera-does-that, if you forget about trying to microanalyse autofocus behaviour and just look over a few dozen assignments shot for no reason other than to try and make good pictures, the Mark IIs show themselves to be strong autofocus cameras and powerful sports picture making tools, particularly outdoors. Conversely, the Mark III cameras get the focus wrong too much of the time in the same situations in which Mark II cameras excel. The EOS-1D Mark II, EOS-1D Mark II N and EOS-1Ds Mark II don't autofocus perfectly, but on balance they got the job done well."
So there you have it folks. In his own words. When he lost his frame of reference did he turn to science to set him straight? No. Because the scientific approach wasn't giving him the clear cut answer he wanted. He turned to his own general assessment of a lot of jobs he had done over the past four years with the camera he loved.
So was all this just a bunch of hoopla? No. As I've
stated over and over Rob has provided a very valuable service to the community by being the first to point out a problem. We replicated his findings. We found that there were a number of original bodies that had some sort of hardware defect. We did not know what it was, but, we knew we had one Canon 1D MKIII that performed well, and one that did not.
As much as I like conspiracy theories. The most likely explanation for all of the original serious AF problems with this camera stemmed from hardware defects. I am much more inclined to believe the following official statement from Canon than I am someone utilizing the testing methods discussed above.
"March 5, 2008
Updated Information
After the AF mirror Adjustment, including the updated firmware version 1.1.3, the EOS-1D Mark III AF function performed better in our tests than all previous EOS camera models.
Canon strives at all times to continuously improve our products in order to provide our customers with the best performance and value.
We are aware that some customers have raised questions about the performance of the EOS-1D Mark III AF system under certain conditions. We will continue to investigate, and look for opportunities to improve, the performance of the AF system to ensure the satisfaction of all of our customers"
Linear Mode
