PART 16 : Qualitative criteria: Credible witnesses
Reports from eye-witnesses play a very important role in relation to UFO reports. Indeed, it is common for there to be only a report from an eye-witness, with no other material being available. How much weight is to be given to such testimony? Who are the most reliable witnesses?
UFO skeptic Robert Sheaffer has voiced the following opinions in relation to “reliable” witnesses (see Footnote 16.02 and Footnote 16.03):
“In evaluating a UFO sighting, the question invariably comes up of whether the witness is reliable. This reflects the commonsense assumption that human testimony falls more or less into two clear categories: reliable, or otherwise. If a witness is reliable, the assumption goes, his testimony fairly accurately depicts actual events. The testimony of an unreliable witness is held to be probably tainted by fantasy and exaggeration. Yet how distinct are these categories?
Many judge a person’s presumed reliability as a witness in much the same way as we estimate his socioecomic status. College education and respectable careers are a strong plus for reliability, especially in the case of doctors, scientists, and pilots. The testimony of unskilled and uneducated individuals is not found to be so convincing. Our assignment of ‘reliability’ to various persons’ testimonies generally reflects society’s hierachies. However, the notion that social status does in some way ensure accurate observation and reporting seems to rest on little more than conjecture.
Just how reliable is ‘reliable’? When we have obtained a firsthand narrative from an apparently credible source, can the facts of the case be considered reasonably well established? A quick glance at the inconsistency of courtroom testimony bids us to be cautious: in thousands of trials, whether for a traffic accident or for murder, there are as many different accounts as there are eyewitnesses.”
Witness testimony has been discussed by Robert Hall (see Footnote 16.01) : “Our legal system is based largely on the assumption that, under certain conditions, we can accept human testimony as factual. Many people, including attorneys and judges, as well as behavioural scientists, have rather clear-cut criteria for assessing the credibility of testimony: the witness’s reputation in his community, previous familiarity with the events and persons involved in the testimony, apparent motives for prevarication or distortion, and internal characteristics of the testimony such as consistency, recency, verifiable detail, and so forth. Also, testimony is more credible with multiple witnesses, especially independent ones, and with multiple channels of observation (e.g. both visual and auditory; both unaided observation and observation through instruments)”.
The (non-exhaustive) factors listed by Robert Hall used as criteria for assessing the credibility of testimony give an indication of the ideal depth of investigation of witness testimony. While the evidence of witnesses may be subjected to this sort of scrutiny in a court of law, it is rarely tested to this degree in the context of UFO reports. Often, such detailed data is not available (at least without considerable expense and inconvenience) to researchers.
Ufologist Hilary Evans has observed : “Ideally, no doubt, we would not accept a single UFO report until we had obtained a comprehensive profile of the witness – his psychological make-up, his socio-cultural background, his economic status and so forth, to which we would add an assessment of the social factors prevailing at the time – the level of UFO activity as reported in the media, current attitudes and the like. Since, however, this state of perfection can be attained only in a minute fraction of cases, the question arises, to what extent should any witness’ statement be accepted?” (see Footnote 16.04).
Given the limited time and resources available to ufologists, it is important to ask : Are there quick and easy ways to identify the most reliable UFO witnesses?
Some UFO researchers clearly consider that there is one factor that is particularly important – the profession of the UFO witness.
This is a belief that is sometimes expressed, and on other occasions is simply implied (e.g. by researchers that dedicate their time and energy to investigating and collating cases involving members of a particular profession, e.g. policemen, pilots, members of the military etc).
In the relatively limited number of articles setting out criteria for assessing the weight of UFO reports, the profession or qualifications of a witness regularly feature.
Some sceptics have questioned the weight given by some ufologists to sightings by members of certain professions. For example, librarian and UFO researcher John Rimmer has written:
“The ‘myth of the credible witness’ refers to the fact that some ufologists believe that there are people who, because of the job or their training, are unlikely to be mistaken no matter what they report. Typically these are policemen, airline pilots, astronomers, servicemen, librarians (oops, sorry, that one seemed to just slip in), and the assumption is that their reports are more reliable than that of any old Joe Soap because they are a (small fanfare) ‘credible witness’” (see Footnote 16.05).
Of course, few ufologists would argue that the members of any profession never make mistakes when making UFO reports. The issue is whether member of certain professions are less likely to make mistakes and their reports therefore deserve particular attention.
So, are the members of certain professions better UFO witnesses?
Before turning to consider the evidence, as a matter of common sense this would seem likely. In my experience of court cases, the profession of a witness is commonly taken into account when deciding upon the weight to be given to the evidence from each witness. If there is a conflict in the evidence given by, say, an accountant on the one hand and a homeless person on the other hand, I'd be pretty confident in predicting who (all else being equal) a judge is more likely to believe. The weight given to a person's occupation or qualifications is, in my opinion, perfectly understandable. We generally expect highly paid professional to be less likely to tell untruths than others. Also, we expect members of certain professions (e.g. the police) to be better witnesses, either due to their training (e.g. in relation to observation or the making of contemporaneous notes) or due to qualifications required to enter that profession (e.g. eyesight must be 20/20, or some other relatively high level). In short:
1. Some professions require individuals to meet certain physical/medical requirements. For example, to become a police officer in
2. Some professions involve relevant training. For example, the training of members of police forces may include attempts to develop powers of observation.
3. Some professions involve relevant knowledge and expertise. For example, astronomers (both amateur and professional) may be expected to be more familiar with objects seen in the night sky than other members of the general public.
4. Some professions have a reputation for honesty, such as policemen, judges and vicars. However, hoaxes account for a relatively small percentage of the total number of UFO reports. Thus, in relation to many UFO reports, the honesty of the witness is not usually a key factor. Nonethelessm hoaxes do play a significant role in relation to high strangeness cases (particularly in cases involving video or photographic evidence).
So, what about the data in the UFO literature?
Well, it appears that the relevant UFO literature is pretty limited and superficial.
In fact, the ufological literature has a truly remarkable lack of discussion of why certain professions or qualifications are likely to make a person a better UFO witness. Allan Hendry has commented that “… no direct experiments have been performed by [ufologists] in thirty years to verify witness reliability. To be sure, such parameters as `solid occupations', `adulthood', `good education' have been cited … but these are superficial at best” (see Footnote 16.06).
There is reason to doubt the weight (if any) to be given to the above common sense factors in the context of UFO reports.
The discussion below focuses on comments made by two ufologists - J Allen Hynek and Allan Hendry.
J Allen Hynek has stated that, while examining the cases in Project Blue Book, he and his colleagues kept careful records of the occupations of witnesses. He considered the correlation between occupation and what was perceived – or misperceived – to be "extremely interesting" and presented the following table:
Occupation % of Misidentification
Military pilot
(single witness) 88%
(multiple witness) 76%
Commercial pilot
(single witness) 89%
(multiple witness) 79%
Radar technicians
(multiple witness) 78%
Technical person
(single witness) 65%
(multiple witness) 50%
Other
(multiple witness) 83%
As J Allen Hynek observered, from his results it would seem that, as a rule, the best witnesses are multiple engineers or scientists; only 50 percent of their sightings could be classified as misperceptions.
However, Hynek expressed surprise that commercial and military pilots appear to make relatively poor witnesses.
The percentage rates of misidentification by commercial and military pilots are considerably higher than the relevant rates in relation to technical persons.
Hynek suggested that “what we have here is a good example of a well-known psychological fact: `transference' of skill and experience does not usually take place. That is, an expert in one field does not necessarily `transfer' his competence to another one. Thus, it might surprise us that a pilot had trouble identifying other aircraft. But it should come as no surprise that a majority of pilot misidentifications were of astronomical objects” (see Footnote 16.07).
Why did pilots appear to make relatively poor witnesses?
Robert Sheaffer has referred to an article by Dr Robert Buckhout, director of the Center for Responsive Psychology at Brooklyn College of the City University of New York, which states that “research I have done with Air-Force flight crews confirms that even highly-trained people become poorer observers under stress” (see Footnotes 16.09, 16.10 and 16.11). However, while the effects of stress may explain pilots becoming “poorer observers”, this does not explain why they appear to become worse witnesses than non-pilots.
Possibly more important than the physical characteristics of pilots is a consideration of the circumstances in which pilots report UFOs. Many of the sightings by pilots are from made from the air. Even experienced pilots have spent a relatively limited fraction of their lives in the air. They may not be as able to judge relevant visual features (or be as familiar with the visual appearance of objects in the sky) from that vantage point as ground-based observers that have spent virtually their entire lives observing the visual appearance of objects from that vantage point. This may be due to a reduction in visual cues while in the air (e.g. other objects nearby against which comparisons can be made). This would be in line with UFOlogical Principle #5 proposed by Philip J Klass: “No human observer, including experienced flight crews, can accurately estimate either the distance/altitude or the size of an unfamiliar object in the sky, unless it is in very close proximity to a familiar object whose size or altitude is known” (see Footnote 16.16).
Or perhaps Hynek’s data is misleading. Hynek’s methodology has been heavily criticised by ufologist Brad Sparks (see Footnote 16.14) in response to an article by skeptic James Oberg which cited some of the above data (see Footnote 16.13). Brad Sparks stated that the the Hynek study that “supposedly” found that pilots had the highest rate of IFO's in the Blue Book files:
- “is completely undocumented and unsubstantiated. It appears on one page of The Hynek UFO Report and there is no source data on how many reports were categorized, the criteria used, etc., and it appears the methodology was utterly confused between ‘misperception’ and ‘misidentification,’ which are two different things. Given that kind of fundamental confusion it is unclear whether the Blue Book ‘IFO's’ that were used were the actual ‘Known’ IFO's and not the ‘Possible’ and ‘Probable’ categories, to give one example of deeply flawed methodology.
- “ ‘Misperception’ involves errors in the observational details and my study shows there are very few of those in any category of witness occupation. ‘Misdentification’ involves erroneous witness opinion of the ultimate cause of the sighting, which are subject to great error because it is asking essentially for a conclusion that only a PhD scientist investigation could properly render and no such thing is available to the witness. Yet if the witness gets that wrong he or she is slandered from here to eternity.
While Brad Sparks’ allegations that Hynek’s methodology was “deeply flawed” may be considered unfair (e.g. it seems to me that it is reasonablte to take into account the “possible” and “probable” categories when assessing results), his questioning of Hynek’s conclusion gets some support from data from two other ufologists: Giorgio Abraini (of the Italian UFO group CISU) and Allan Hendry.
Giorgio Abraini’s data has not been published, but he was kind enough to share it with me (although he stressed the data was “far from conclusive”). He mentioned to me that that he had tried to do a similar exercise to that performed by Hynek, based on Italian data. He had tried to compare and contrast the level of identifiable reports in an Italian general catalogue and the Italian Aircat catalogue of reports by pilots.
The compiler of the Italian Aircat catalogue of reports by pilots (Marco Orlandi) considered that 31.5 percent of the 244 sightings by pilots in
I have suggested to Giorgio Abraini that the results he mentioned could simply indicate that he is more skeptical than Marco Orlandi and therefore prepared to consider more sightings as having a cause that can be identified. Giorgio Abraini accepted that this was a possibility. He considered that there could be issues arising from the fact that the subjective assessment of whether the reports were due to an IFO was not made by one person only.
More significant reason for questioning Hynek’s data arises from the work of Allan Hendry. In his book "The UFO Handbook" (1979), Hendry asked whether "Do certain occupations really guarantee a strong foundation for accepting a UFO account?" (see Footnote 16.14).
Hendry compared the number of IFO and UFO reports for various professions within his sample, creating a “misperception `failure rate' for the different occupations”. The ratio of IFOs reported to all reports was as follows:
All known occupations 89%
Pilot/air personnel 75%
Skilled trade 75%
Forestry/farmer 86%
Blue collar 87%
White collar 88%
Retired/disabled 88%
Education 88%
Student 89%
Housewife 89%
Unemployed 90%
Retail 91%
Medical 93%
Law enforcement 94%
Hendry concluded that “Apparently, occupation does not guarantee discrimination". He commented that "to say that pilots did the best job with a 75 per cent failure rate is not much of a victory".
In relation to sightings by police witnesses, Hendry commented: "Why did police do the worst job? Perhaps they are more conscious of small details during their quiet night patrols. Scintillating stars were a favourite target of police witnesses”.
I would add to one further possibility to Hendy’s speculations. Perhaps members of certain professions (e.g. police officers) are more likely to feel obligated to report something they cannot identify, even if it is not really very strange.
However, it is noteable that the results for “pilot/air personnel” witnesses in Hendry’s study have the lowest failure rate (i.e. 75%) – significantly better than the results for all know occupations (which had a failure rate of 89%).
Thus, Hendry’s data in relation to pilots appear to be at odds with the results of Hynek’s study.
In my view, the evidence indicates that the additional weight to be given to the evidence of members of certain professions (due to their training, experience, eyesight requirements etc) may well be outweighed by other factors relating to sightings be members of those professions. However, the sparce collection and consideration of relevant objective data makes it difficult to draw any firm conclusions.
In summary, it seems to me that witness occupation may well be considerably overrated by most ufologists as a criteria for selecting the best UFO cases. The evidence does not convincingly demonstrate that, for example, pilots or policemen report fewer IFOs than other witnesses.
However, when considering whether the members of certain professions generally provide “better” evidence than the average UFO witness, one of the questions that arises is : “better” at what? When it comes to persuading astronomers and other scientists to take ufology seriously, being able to present them with details of UFO sightings by scientists and astronomers may be more valuable than their evidential value alone may suggest.
After all, scientists are only human and therefore may place an inflated value on their perceptual abilities and those of their colleagues.
Thus, if ufologists want (as many state) scientists to get involved in ufology, then compilations of sightings by scientists and astronomers would seem to be a useful tool for ufological advocates. Unfortunately, compilations of such sightings have tended to be less popular than databases of sightings by pilots and police officers.
FOOTNOTES
[16.01] Robert Hall, “UFO’s: A Scientific Debate” (1972) (edited by Carl Sagan and Thornton Page) at pages 218-219 (in Chapter 9) of the Barnes and Noble hardback edition (with the same page numbering in the Norton paperback edition).
[16.02] Robert Sheaffer, “The UFO Verdict” (1980) at page 99 (in Chapter 10) of the Prometheus softback edition.
[16.03] Robert Sheaffer, “UFO Sightings: The Evidence” (1998) at page 135 (in Chapter 9) of the Prometheus hardback edition.
[16.04] Hilary Evans “The Evidence for UFOs” (1983) at page 82 (in Chapter 4) of the Aquarian softcover edition.
[16.05] John Rimmer, UFO UpDates List, August 2000
http://www.virtuallystrange.net/ufo/updates/2000/aug/m18-030.shtml
[16.06] Allan Hendry,“The UFO Handbook” (1979) at page 14 (in Chapter 1) of the Sphere softback edition
[16.07] J Allen Hynek, "The Hynek UFO Report" (1977) at page 261 (in Chapter 11) of the Barnes & Noble hardback reprint (1997) at page 271 of the Dell paperback edition (with the same page numbering in the Sphere paperback edition).
[16.08] Allan Hendry, "The UFO Handbook" (1979) at pages 101-102 (in Chapter 8) of the Sphere softback edition)
[16.09] Robert Buckout, “Eyewitness Testimony”, Scientific American, December 1974, page 23.
[16.10] Robert Sheaffer, “The UFO Verdict” (1980) at page 106 (in Chapter 10) of the Prometheus softback edition.
[16.11] Robert Sheaffer, “UFO Sightings: The Evidence” (1998) at pages 145-146 (in Chapter 9) of the Prometheus hardback edition.
[16.12] 'Visual-Spatial Abilities Of Pilots' from the Journal of Applied Psychology, 1993, Vol 78, No 5, (p. 763-773).
[16.13] James Oberg, “Case Studies In Pilot Misperceptions Of ‘UFOs’”, reposted on UFO UpDates discussion List in May 2007.
http://ufoupdateslist.com/2007/may/m05-002.shtml
[16.14] Brad Sparks, UFO UpDates discussion List, May 2007.
[16.15] Allan Hendry, "The UFO Handbook" (1979), Hendry asked (at pages 101-102 (in Chapter 8) of the Sphere softback edition)
[16.16] Philip J Klass “UFOs Explained” (1974) at page 49 (in Chapter 5) of the Random House hardback edition, at page 57 of Random House paperback edition.