“Best UFO Cases” by Isaac Koi
PART 24: Quantitative criteria : Olsen’s Reliability Index
There have been various proposals for quantitative criteria to assess the reliability of UFO reports. Most are considerably less well-known than, say, the schemes proposed by Hynek and Vallee (see PART 20: Quantitative criteria : Hynek – Strangeness and Probability and PART 21: Quantitative criteria : Vallee’s SVP ratings respectively). Some of those less known schemes have been implemented due to support from large UFO groups, e.g. BUFORA and MUFON (see PART 22: Quantitative criteria : BUFORA’s case priority and PART 23: Quantitative criteria : Ballester/MUFON index respectively). Some, such as the scheme outlined below, have largely remained unapplied.
Thomas Olsen wrote a book entitled “The Reference for Outstanding UFO Sighting Reports” (1966). That book was discussed in the Condon Report in 1969 (see Footnote 24.11), which stated the following: “There apparently exists no single complete collection of UFO reports. ... Proposals have been made from time to time for a computer-indexing of these reports by various categories but this has not been carried out. Two publications are available which partially supply this lack: one is The UFO Evidence (Ha11, 1964) and the other is a collection of reports called The Reference for Outstanding UFO Reports (Olsen)”.
While the first of the two books mentioned in that paragraph of the Condon Report, i.e. Richard Hall’s book “The UFO Evidence” (1964) remains well-known and the complete text of that book is now available on several websites, Olsen’s book is now relatively obscure.
In his book, Thomas Olsen discussed at length the calculation of a “reliability index” for UFO reports in a book he wrote in 1966 (see Footnote 24.01)
Olsen acknowledged that his “reliability index” was an approximation, but suggested that it was nonetheless useful since it gave “in a single number, some general, conservative indication of probable reliability” (see Footnote 24.08). He also claimed that another use of the reliability index is “in arranging thereports according to their relative reliability” (see Footnote 24.09).
Olsen’s “reliability index” is a value between zero and which represents “the probability that the sighting report accurately describes a real event - that it is not the result of a hoax or hallucination”. This value is obtained by multiplying the probabilities of three factors, summarised by Olsen as follows (see Footnote 24.04):
1. Witness factor: probability that the witnesses, reporting in concert, accurately described a valid experience;
2. Investigation factor: probability that the investigating agency correctly documented a reported experience which has no explanation in terms of known man-made or natural phenomena.
3. Transcription factor: probability that intermediate sources for the report have related it as originally obtained, without omission, distortion or addition of spurious details.
The relevant chapter of Olsen’s book discusses his proposals for assigning values to each of these three factors, as summarised below.
1. Olsen’s “witness factor”
At the heart of Olsen’s proposed method of calculating this factor appear to be the following points (see Footnote 24.05):
a. Olsen assumes that the more experienced a witness has of “aerial phenomena” then the less likely that individual is to provide “false, inaccurate testimony”. Thus, the probability of an astronomer or commercial pilot (individuals with “extensive” experience with “aerial phenomena”) providing “false, inaccurate testimony” is assumed to be 12.5%, while the probability of a baker or plumber (individuals with “essentially” no experience with “aerial phenomena”) providing “false, inaccurate testimony” is assumed to be 50%. Olsen's discussion of this factor should now be considered in the light of the matters covered in PART 16 : Qualitative criteria: Credible witnesses.
b. Olsen himself notes that calculating a more accurate (or “literal”) value of the probability that the witness has provided false or inaccurate evidence would require “information about the character, personality and moral integrity” of the witnesses, but such information is “often not stated in a sighting report”. Such further information should be taken into account when available.
c. Multiple witnesses have a very dramatic effect on the value assigned by Olsen to the “witness factor”. The formula put forward by Olsen for assigning a value to the “witness factor” assumes that in cases with multiple witnesses, the probability of testimony either being inaccurate or false decreases _exponentially_ with the number of witnesses. Thus, a sighting report from one pilot has a 12.5% (1 in 8) probability of being false or inaccurate, but if the report is from two pilots then the probability decreases to 1.56% (1 in 64). Similarly, a report from one plumber has a 50% (1 in 2) probability of being false or inaccurate, from if the report is from 3 plumbers from the probability decreases to 6.25% (1 in 16). Olsen's discussion of this factor should now be considered in the light of the matters covered in PART 17: Qualitative criteria: Multiple witnesses.
d, The dramatic effects of multiple witnesses on the witness factor results in considerably more weight being given to a report from several individuals of unidentified profession than a report from a single astronomer, meteorologist or pilot.
2. Olsen’s “investigation factor”
This factor represents the probability that the following two conditions are met (see Footnote 24.06):
a. The reported details were documented correctly and completely.
b. They have no explanation in terms of known man-made or natural phenomena.
According to Olsen, the values assigned to this probability “reflect the importance given to a check for correlations with other phenomena … a high value of the investigation factor is assurance that a report can be used with confidence, without having to double-check for a conventional explanation”.
Olsen suggests the following numerical values for the investigation factor:
(a) 99.9% - Quality of Investigation: High - (1) On-site investigation by Federal investigator, supported by local Weather Bureau data, evaluated by USAF consultant, and documented in detail. (2) Lengthy interview by professional astronomer, filing a detailed report privately.
(b) 75% - Quality of Investigation: Intermediate - (1) Detailed report filed by one crew member of squadron of fighter-bombers, after discussion with accompanying witnesses. No check for correlation with other phenomena. (2) Professional astronomer filing report of personal sighting.
(c) 50% - Quality of Investigation: None or unknown - (1) Detailed newspaper report. (2) Personal sighting report filed by mechanic; no check for correlation with other phenomena.
3. Olsen’s “transcription factor”
This factor reflects the reduction in reliability resulting from secondary reports, which may be abbreviations of the originals or reflect other alterations (to dramatize the account, or simply due to typographical or translation errors) – see Footnote 24.07.
Olsen suggests reducing the value to be assigned to the reliability of an account _exponentially_ with the number of transcriptions involved (the “n-th handedness” of the report). Thus, the transcription factor is equal to 100% if the report is a primary report, 50% (1 in 2) if a second-hand report, 25% (1 in 4) if a third-hand report.
Actual applications of Olsen’s “reliability index”
Olsen applies (or at least purports to apply) his system to generate “reliability indexes” for a relatively large sample of UFO reports. He includes a table which lists each of the UFO reports included in his book according to their Reliability Index (R.I.), i.e. “160 oustanding UFO sigting reports” with each report being “so clear-cut, detailed, unambiguous and unconventional that rational misinterpretation of natural or man-made phenomena is obviously impossible”.
The top 12 entries in Olsen’s table (see Footnote [610]) are as follows:
1) R.I. = 0.99999, Summer 1952 - Haneda Airport
2) R.I. = 0.99999, 14 September 1954 - Vendee, France
3) R.I. = 0.99999, 22 October 1954 - Marysville, Ohio
4) R.I. = 0.99999, 22 May 1962 - Paraiso Del Tuy, Venezuela
5) R.I. = 0.99999, 12 October 1961 - Indianapolis, Indiana
6) R.I. = 0.99975, 16 January 1951 - Artesia, New Mexico
7) R.I. = 0.99975, 3 April 1964 - Monticello, Wisconsin
8) R.I. = 0.99804, 3 August 1951 - Silver Lake, Michigan
9) R.I. = 0.99804, 27 July 1952 - New Jersey, opposite NYC
10) R.I. = 0.99804, 25 July 1957 - Niagara Falls, Municipal Airport
11) R.I. = 0.99609, 11 April 1964 - Homer, New York
12) R.I. = 0.99218, 14 September 1952 - Hill near Sutton, West Virginia
Olsen’s “reliability index” does not appear to have been discussed on the Internet prior to this article. No one appears to have been inclined to provide a summary online of the factors which it took into account.
However, this “reliability index”:
a. is briefly discussed and applied to one case in an appendix to Illobrand Von Ludwigger’s book “Best UFO Cases : Europe” (1998) – see Footnote 24.02. That discussion adds little, if anything, to the original presentation of the proposed scheme by Olsen in his own book. However, Von Ludwigger’s book does make several references to Von Ludwigger’s participation in MUFON-CES (see Footnote 24.12), i.e. the Central European Section of MUFON. It is therefore possible that MUFON-CES has applied Olsen’s “Reliability Index” scheme to some extent. I note, however, that MUFON’s centralised database appears to have implemented the scheme outlined in PART 23 rather than Olsen’d criteria. Von Ludwigger’s was published by NIDS (i.e. the National Institute for Discovery Science), but I am not aware of NIDS ever applying the “Reliability Index” scheme.
b. Olsen’s “Reliability Index” scheme is discussed in the book “Ufology” by James McCampbell at slightly greater length (see Footnote 24.03). That discussion includes a comment that Realibity Theory had “been successfully applied to UFO reports”, stating that “as with any complex system, the problem was first broken down into its finest elements”. It noted that “such factors as the number of witnesses, their training in aerial observation, and the circumstances of the sighting were isolated” and stated that “details of the original documentation were accounted for with emphasis upon interviews of the witnesses and the professional qualifications of the interviewers”. While the McCampbell’s book notes that 160 sightings “were selected and analysed”, he was referring to Olsen’s own application of the “Reliability Index” rather than any later application.
FOOTNOTES
[24.01] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at pages 4_1 to 4_13 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.02] Illobrand Von Ludwigger, “Best UFO Cases : Europe”, 1998, NIDS, Appendix A: “Reliability Index according to Olsen”, page 159.
[24.03] James McCampbell, “Ufology” at pages 2-4 of the version online at:
http://www.nicap.org/ufology/ufology.htm
[24.04] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at page 4_1 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.05] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at pages 4_1 to 4_2 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.06] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at pages 4_2 to 4_5 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.07] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at page 4_5 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.08] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at page 4_6 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.09] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at page 4_7 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.10] Thomas Olsen, “The Reference for Outstanding UFO Sighting Reports” (1966) at page 4_11 (in Chapter 4) of the UFOIRC spiral-bound edition.
[24.11] Thomas Olsen’s book is discussed at: Condon Report (“Scientific Study of Unidentified Flying Objects”, Edward U Condon (Director) and Daniel S Gillmor (Editor) (1969)) at page 32 (in Section 2 “Summary of the Study”, by Edward U Condon) of the uncorrected version submitted to the Air Force (with the same page numbering in the 3 volume paperbound edition distributed by the National Technical Information Service, US Department of Commerce) at page 23 of the Vision hardback edition (with the same page numbering in the Bantam paperback edition). The first of these editions has the same page numbering as the edition available free online at the following links:
http://files.ncas.org/condon/text/contents.htm
http://www.project1947.com/shg/condon/contents.html
[24.12] MUFON-CES has a website at the link below, available as at June 2010: