The TV fire and flame retardant controversy: deconstructing the data

March 24, 2014 at 10:56 am | Posted in Feature Articles | 1 Comment
Tags: , , , , , ,

Whether or not candles cause TV fires has been a cause of controversy, with a recurring argument between those who believe that a standard for ignition resistance is necessary for television manufacture, and those who believe such a standard results in excessive use of environmentally hazardous flame retardant chemicals.

Instrumental to the case in favour of the ignition standard is data presented in a series of papers derived from an obscure estimate of causes of fire in Sweden. Here, Tom Muir, an independent researcher and former employee of Environment Canada (retired), explains how flaws in these derivations ultimately result in failure to demonstrate a need for an ignition standard.

This is a long article. Click here to download a printer-friendly PDF.

Update (May 2014). There is a discussion on LinkedIn of points raised, with comments by one author of the research critiqued below. Also see here a response by another of the authors.

Tealight by Supafine - Morgeufile

In this article:

  • Context to the controversy
  • First encounters
  • Observations about the intent of the SEMKO-Vällingby study
  • The actual findings of the SEMKO-Vällingby study
    • Method 1: Calculating Swedish TV fire incidence
    • Method 2: Modifying Sweden-Level Insurance Statistics
  • Simonson et al.’s treatment of the Vällingby data: from Vällingby to Sweden to the whole of the EU
  • Critical errors in the Simonson et al. analysis of TV fires caused by external ignition
  • Comparing Simonson et al. to other estimates of fire incidence in TV sets
  • Quantifying the possible magnitude of error in Simonson et al.’s model
  • The limited extent of peer-review of Simonson et al.’s figures
  • Concluding remarks

Context to the controversy

In 2012 the Chicago Tribune ran a series of stories about the influence industry has had over the setting of flame retardant (FR) legislation and standards. The paper said: “Industry has disseminated misleading research findings so frequently that they essentially have been adopted as fact. They have been cited by consultants, think tanks, regulators and Wikipedia, and have shaped the worldwide debate about the safety of flame retardants.” (CT 2012)

The Tribune identified a particular series of papers authored by Margaret Simonson McNamee, a research manager at the SP Technical Research Institute of Sweden, as fundamental components of the “misleading research findings”. Simonson McNamee subsequently defended her research, stating that the fire information was based on “a Swedish statistical model” and was “conducted using the best information available at the time”. (Simonson McNamee, M. (2012) Fires and the Environment – a Burning Issue! Fire Safety Science News #33)

For the purposes of advancing understanding of the evidentiary basis on which fire safety standards are set, resulting in the use of large quantities of flame retardant in TV housings, the merits of the Simonson research and presentation of its findings are worth exploring in detail.

First encounters

My first encounter with this matter began in 2006 at the Dioxin Symposium, in Oslo Norway, with the presentation of a paper titled “Cost Benefit Analysis Model For Fire Safety: Deca-BDE Case Study”, by Margaret Simonson, Petra Andersson and Martin van den Berg. Among various costs and benefits, the paper claimed that up to 160 deaths and 2000 injuries in Europe could be saved annually by the use of deca-BDE flame retardant in TV enclosures:

“Thus, the benefits that will be included in this direct application of the Fire-CBA model will be the value of lives saved through the avoidance of TV fires and treatment of injuries. The number of lives saved through the use a high fire performance material in the TV enclosure was estimated to be 160/year in Europe each year in the Fire-LCA TV Case Study. Similarly, the number of injuries avoided was estimated, in that study, to be approximately 2000/year in Europe.”

Since these are substantial benefits from the use of a flame retardant which is an  environmental pollutant, my initial reaction was that I wanted to know how the deaths and injuries had been determined.

After spending considerable time searching, I found the initial text on fire numbers in the unpublished report; Simonson et al. 2000. Fire-LCA Model: TV Case Study. I also found a number of related studies by Simonson et al. that all repeated identical text describing fire numbers and asserted rates of TV fires per million TVs. Through following these texts, I was able to discover which study it was that underpinned all the fire-related numbers in Simonson’s reports.  This study is a Swedish National Electrical Safety Board (SEMKO) examination titled “ELECTRICAL FIRES- fact and fiction: Final Report of the ‘Vällingby Project’” [often referred to as the “Vällingby study” but referred to in this article as “SEMKO-Vällingby”].

At the time of this discovery, the SEMKO-Vallingby report  was unpublished and only available in Swedish. I contacted SEMKO and they directed me to Simonson, who provided me with a Swedish language copy. Using limited translation, I was able to decipher enough of the fire numbers for my purposes at the time. Several years after this initial encounter, a Chicago Tribune (CT) reporter, Sam Roe, contacted me. I provided him with a copy of the SEMKO-Vallingby report which he had translated into English. The CT did its own search and found that only one library in the world, in Sweden, had a copy of this report in its holdings.

This situation intrigued me from the start, leading me on a long and convoluted journey to explore how the fire numbers and assertions made by Simonson and co-authors were derived. I began with the question as to how it could be that the Simonson numbers could either be considered peer-reviewed or legitimately form the basis of international fire safety standards, if the underlying SEMKO-Vallingby study were only available in Swedish, was not published, nor was publically accessible?

Ultimately, I found that the underlying methods and materials were never described in the Simonson et al papers: the SEMKO-Vallingby fire numbers were just asserted in identical text in all the relevant Simonson et al studies that I located. Eventually, SEMKO-Vällingby was no longer even referenced as the primary data source; instead other reports were self-referenced as the supporting studies, creating a barrier between the evidence which is used to justify the use of flame retardant in TV casings and the ability of regulators and researchers to check the credibility of that evidence.

Observations about the intent of the SEMKO-Vällingby study

The overall intent of the SEMKO-Vällingby study was to try and clarify uncertainties around Swedish fire data and to examine the possibilities for making better use of available information sources. It was initiated by the Vällingby Police Department, which was investigating “all” fires in an area consisting of the entire west side of Stockholm, in a period spanning 1995 to 1996.

The study became a collaborative effort when SEMKO decided to participate. SEMKO  experts worked with the Police and Fire Departments and insurance companies to categorize every fire in the trial area over the time period.

In design, the SEMKO-Vällingby study was an experimental, observational “field” study of a trial area. The area to be observed was not selected randomly but specified a priori. Although the word “statistics” is often used in reference to the numbers presented, it is misleading to describe it as such:

  • It is not based on an independent, random sample from a defined larger population, which can then be used to make statistical inferences about the larger population as a whole.
  • It is not a method using a sampling frame and design that permits statistically valid and unbiased extrapolations to a larger population – the trial area itself is the only population observed.

The SEMKO-Vällingby study attempted to make estimates of fire numbers, first for the Vallingby trial area, then for Sweden as a whole, based on a couple of methods, to be described. Neither of these methods is a statistically valid extrapolation from the trial area to any larger area.

Simonson’s extrapolation therefore faces the following basic obstacles with regard to the use of the SEMKO-Vallingby report:

  1. The report gives no estimates of the total number of TVs in the Vällingby area, only an imprecise estimate of the total number of electrical fires, and of these, which were audio-visual or TV related.  Thus, there is no basis to estimate the fire incidence rate, or the number of fires per million TVs (as an example metric), for the trial area, or to extrapolate such a rate to Sweden, or elsewhere.
  2. Nor are there any reports of associated fire deaths or injuries in Vällingby. So again, there is no basis to assert fire deaths and injuries in the Vällingby trial area over the period of observation, or to extrapolate such events to Sweden or elsewhere.
  3. Vällingby primary fire numbers are simply estimates for one small, specific geographical area. If they were extrapolated to Sweden, and particularly the rest of Europe, this would introduce a huge selection bias into any final figures so derived.

Nonetheless, extrapolation to both Sweden as a whole, and then to all of the EU, is exactly what Simonson et al. studies proceed to do in producing an estimate of Swedish and EU fire incidence.

The actual findings of the SEMKO-Vällingby study

Using two different extrapolation methods based on two different source data-sets, the SEMKO-Vällingby study estimated that a total of between 2,400 and 3,500 electrical fires were likely to be occurring each year in Sweden.

Method 1: Calculating Swedish TV fire incidence.

The first SEMKO-Vällingby method was based on fire data from the Vällingby and nearby suburbs of Western Stockholm through 1995-96.

In this suburb, an estimate of 180 possible electrical fires was derived from all fire reports made by various agencies including firefighters, police and insurers. 32 of these were considered by the researchers to be actually electrical, of which 8 were attributed to TVs. (For reasons which will become clear, It is of key importance that the SEMKO definition, or standard, of electrical fire, is one in which: “damage (or fire) must have occurred outside the device.”) 

To estimate fire incidence for Sweden as a whole, SEMKO-Vällingby then performed a crude extrapolation assuming that the total number of electrical and TV fires in Vällingby was representative of the whole country.

However, since there was no data on the number of TVs in Vällingby, only on frequency of fires within certain areas of building type and land use, the researchers could only estimate total TV and electrical fires for Sweden by scaling-up assumptions of relative floor area of building or land use type where fires had occurred from the Vällingby area to the whole of Sweden. Because some land uses such as agriculture were not in the Vällingby study area, further assumptions about fire frequency in these areas had to be made. The SEMKO-Vallingby study cautions that all of these estimates are based on highly uncertain and imprecise measures of numerous factors and assumptions that are then combined to produce the results.

This method yielded an overall estimate of 2400 electrical fires per year for the whole of Sweden.

Notes on Method 1. A major limitation of this first method is the very small sample size, demanding that we take only 32 electrical fires (8 of which were TV fires, a point we will return to later) and apportion them to several building types and their respective floor areas, in order to produce an estimate of the number of fires for the whole of Sweden. This introduces an imprecision from the start, which will be amplified as it is scaled up.

Another limitation is the data is from only one very specific place in Sweden. Since there is little reason to assume that the Vällingby suburb represents Sweden as a whole (indeed, it cannot possibly do so, since there are many land uses in Sweden which are not present in Vällingby), it is highly likely that there is a very strong sampling bias in the model.

Method 2: Modifying Sweden-Level Insurance Statistics

The second method presented in the SEMKO-Vällingby study is based on Swedish national insurance statistics for electrical fires from 1994. This reported a total of 6123 electrical fires, of which 2557 were TV fires. The researchers then checked these figures for differences in definition of what counts as an actual electrical fire, the presence of coding errors, and so forth, to determine that the number of electrical fires had  been over-estimated by about 3000.

Other adjustments and imputations produced a final estimate of 3500 electrical fires in the whole of Sweden. Of these, 895 were considered to be TV fires – reducing the original insurers’ estimate by 65%.

Notes on Method 2. Part of the SEMKO-Vällingby study’s reduction of the insurance estimate is due to their interpretation of what counts as an electrical fire. Here, as noted above, they followed SEMKO’s standards, which classify possible smoke emission, absence of open flame and no fire outside the device as component failure and not fire (the rationale being that the component failure presents no risk of fire and the device can be repaired by replacing the faulty component).

Furthermore, the definition of a fire being damage (or fire) outside the device, would logically result in external fire causes being included in the count, as how could one tell the difference? (This is an important detail in terms of what Simonson et al. later add back in to the data from which they derive their own fire estimates.)

Simonson et al.’s treatment of the Vällingby data: from Vällingby to Sweden to the whole of the EU.

Simonson et al. used the SEMKO-Vällingby estimates of TV fires to produce two estimates of how many TV fires there were in Sweden.

In its fundamentals, their method is simple: since 8 fires out of 32 means 25% of electrical fires are TV fires, they assume it follows that 25% of the two electrical fire totals given in the SEMKO-Vällingby report will yield two estimates of the total number of TV fires for the whole country. This gives 600 fires for Vällingby Method 1 (25% of 2400) and 875 fires for Vällingby Method 2 (25% of 3500). They round up the Model 2 figure to 900, treat this as an upper limit for the number of TV fires in Sweden, and treat Model 1 as a lower limit.

Without giving any estimate of the number of TVs that were in Vällingby for the period in which the 8 recorded TV fires were observed, Simonson et al. then use these extrapolated numbers to estimate the incidence rate of TV fires in Sweden, i.e. the number of fires per million TV sets. They do this by taking 750 (the mid-point of their two estimates) as the “true” number of TV fires in Sweden; then they assume, without a supporting data reference, that there are 7.5 million TV sets in Sweden. This yields an incidence rate of 100 TV fires per million sets. Recall, that all these numbers pertain to the years 1995 to 1996.

Note also that there is an error in the Simonson et al. procedure, in that it fails to include the adjustment in the Model 1 scale-up for 750 fires in the land use categories not identified in Vällingby. The correct scale-up from Vällingby would not include these fires, and would thus be 1650, which would give a lower estimate of 413 TV fires – assuming, of course, that the ratio of electrical to TV fires found in Vällingby applies across the whole of Sweden.

In their final move, Simonson et al. extrapolate this Swedish TV fire rate of 100 TV fires/million TVs to the entire EU. Extrapolating from Vällingby to Sweden is contentious enough, but to extrapolate to the whole EU entails the same set of assumptions be made again only on a far larger scale.

Overall, it is impossible to see how these assumptions have sufficient general validity to allow this move. There is no mention in Simonson et al.’s model of possible errors, variability in the estimate of TV fires, TV numbers, TV types or TV use rates, nor the subsequent errors or uncertainties these might introduce into the TV fire rate. In order to extrapolate to Europe this is essential information: after all, what do the social habits and climatic influences of southern Italy have to do with northern Sweden? It would seem there are an endless number of influencing differences between Vällingby, Sweden as a whole, and the EU.

Comparing Simonson et al. to other estimates of fire incidence in TV sets. Of a different and lower estimate than their own figure, Simonson et al. state: “In conclusion, the Sambrook study provides a sound basis for comparison of fire statistics from different European countries, but it is too conservative in its estimate of the frequency of TV fires. The Vällingby data provided a better model for European TV set fire behaviour. It is also this data that has been used as the basis of the TV fire model described in chapter 3.

“These estimates are approximately a factor 10 higher than the Sambrook estimates. In terms of consumer fire safety this indicates that the number of deaths as a direct result of TV fires could also be a factor 10 higher, or closer to 160 deaths in the EC each year as a direct result of TV fires. Making a similar calculation for the number of people injured as a direct result of TV fires each year in the EC this number increases to 2000.” (Unpublished Simonson and Stripple 2000; Simonson, Tullin, and Stripple 2002; Poortere et al 2000; Oslo Dioxin paper 2006; Simonson, Andersson, and van den Berg. 2006. Cost Benefit Analysis Model for Fire Safety: Methodology and TV (DecaBDE) Case Study. SP Swedish National Testing and Research Institute.)

Note that in the Sambrook (1996) study referred to here, the fire rates reported for EU countries ranged from 8.2 to 17.5 TV fires/million TVs, with about 16 deaths and 197 reportedly minor injuries. Other fire rates ranging from 11 to 22 TV fires/million TVs are reported for the UK and the Netherlands. The overall range spans 2 to 3 fold, much less invariant than Simonson et al. claim: “Significantly, the Sambrook study has concluded that the occurrence of fires throughout Europe seems to be essentially the same (normalised per million TV sets) in each individual country. The Sambrook study relies on statistics from similar sources in each country. Assuming that the Sambrook conclusion is correct in indicating this similarity in fire behaviour the Swedish data can be used as a model for Europe.”

The phrasing is understated but in fact amounts to the very bold assertion that the Sambrook data  supports extrapolating from a mere 8 fires observed in Vallingby in 1995-96 to TV fire incidence for the whole of Europe.

Finally, it is important to note that the Insurance Federation Statistics are for 1994. At this time cathode ray tubes (CRTs), with their high operating heat levels and voltages, were the dominant TV technology. Technological changes since then have been towards flat panel TVs with much lower operating temperatures and voltages, making this internal heat from CRT operation much less a source of fire risk in 2013.

Critical errors in the Simonson et al. analysis of TV fires caused by external ignition

Concerns about external ignition have been important in on-going discussions about the need for an external ignition standard in TV fire safety. The Simonson et al. figures play a pivotal role in justifying a standard for open-flame resistance of TV sets which would result in greater use of flame retardants.

Regarding this, Simonson et al. write a piece of text that appears in the several studies examined:

“Using SEMKO’s definition, the Vällingby study estimated that approximately 750 (or between 600-900) audio/visual fires occur per year in Sweden. These fires were all large enough to have breached the TV enclosure. SEMKO concluded that the additional 1750 fires reported by the Insurance Federation were either wrongly classified, e.g., so small that they had not breached the enclosure, or were caused by an external ignition source. Assuming that approximately half of the Insurance Federation fires did not breach the housing would leave approximately 500 due to external ignition sources. These data correspond to approximately 100 TV fires/million TVs in Sweden due to internal ignition and 65 TV fires/million TVs due to external ignition, and 160 TV fires/million TVs where the fire does not beach to enclosure.”

Usually, only the most severe TV set fires find their way into electrical safety board or fire brigade statistics. We suggest that the Vällingby project results, because of the thoroughness of the methodology, are more representative of a wider European reality.”

There are several mistakes in this text.

Firstly, there is no basis in the SEMKO-Vällingby method which justifies reintroduction into analysis by Simonson et al. the 1750 insurance-related incidents originally excluded by SEMKO. There is certainly no language in the SEMKO-Vällingby study which suggests that SEMKO itself “concluded” that these other incidents were actual fires that did not breach the housing or were externally caused. And further, it is Simonson et al, not SEMKO-Vällingby, that make the assumptions, conclusions, and assertions, about the fire numbers that are then reintroduced.

Recall, SEMKO’s definition of an electrical fire is one where there is fire and damage outside of the device. If there was no such external fire or damage, such a fire report was attributed to component failure or lightening surge, where the device operated normally after repair.  Moreover, as noted above, fires outside the device would capture external causes, such as candles. SEMKO-Vällingby concluded that overall, TV fires are far fewer than in insurance reports. Fires outside the device would be captured by the SEMKO-Vällingby definition and therefore included in the count; reintroducing them here inflates the fire count.

And Simonson et al. made no assumptions or assessments of the numbers involved except as totals and the 65% reduction in the initial Insurance Federation TV fire numbers; in all, they simply reintroduce into analysis fires which were excluded in the SEMKO analysis under Method 2.

Secondly, although it is possible that some TV fires are caused by external sources (e.g. candles), there is no actual data on which to base an estimate. SEMKO-Vällingby make little mention of external causes and candles are mentioned in only one phrase. The rate asserted by Simonson et al of 65 TV fires/million TVs due to external causes is about 3 to 8 times greater than any other estimate of EU fire rates for all TV fires estimated by others, from Sambrook (1996) and other sources. In any case, it is worth repeating that fires outside of the device would be captured by the definition, and included in the official count.

Thirdly, there is a noteworthy contradiction in Simonson et al.’s reintroduction of alleged fire incidents rejected by the SEMKO-Vällingby models. Since the components in which Simonson’s internal ignition fires originate are already treated with flame retardants, Simonson et al. are using internal electrical fires in TVs that can only originate in parts that are already treated with FRs, to make a case for the introduction of flame retardants. However, if the flame retardants were effective, then we would not see the fires; if anything the Simonson data only proves that the FRs don’t actually work – if they did, the fires wouldn’t have happened.

Quantifying the possible magnitude of error in Simonson et al.’s model

One can give an impression of possible magnitude of error in the Simonson et al. analysis. by using a Poisson distribution to model a national fire rate from the 8 TV fires observed in Vällingby and assuming the number of televisions in the Vällingby area is proportional to the rest of Sweden (i.e. 1% of all the TVs in Sweden are in Vällingby – not a good method, but similar to the one used by Simonson et al.). The result of the observed (mean) 8 TV fires yields a 95% confidence interval of 3.5 to 15.8 TV fires per year.

For the whole of Sweden, an estimate of fire incidence based directly on the SEMKO-Vällingby rate is 113-510 (mean 263) TV fires per year. Simonson et al.’s method using the SEMKO-Vällingby data produces a higher estimate, of 600-900 per year (750 mid-point).

In terms of the Poisson distribution, this 750 fires would correspond to observed or mean SEMKO-Vällingby TV fire events of 23.24, instead of 8, or about 4 standard deviations higher. The Poisson probability that this number or higher would occur is 0.000011. Thus it appears that the Simonson TV fire numbers are an outlier value and extremely unlikely to be representative of real incidence.

The limited extent of peer-review of Simonson et al.’s figures. It is worth examining the extent to which, in spite of being published in  two journals, Simonson et al.’s figures have genuinely been peer-reviewed. The point is not whether the papers were read by reviewers before they appeared in journals but whether the reviewers realistically had sufficient access to the source data to judge the credibility of Simonson et al.’s results.

One study, “The Fire Safety of TV Set Enclosure Materials, A Survey of European Statistics”, by M. De Poortere, C. Schonbach, and M. Simonson was published in 2000 in Fire and Materials.

In this study, the fire numbers sections are written in the identical text, more or less word for word, from an original unpublished paper (Simonson et al, 2000. Fire-LCA Model: TV Case Study); however, this unpublished paper is not cited as the source of this text, meaning the fire numbers are effectively presented as original work. There is only a vague reference to the  SEMKO-Vällingby study, but nothing acknowledging that this study underpins all the Simonson et al. fire numbers.

In the second and other peer-reviewed study (M. Simonson, C. Tullin, and H. Stripple. “Fire-LCA study of TV sets with V0 and HB enclosure material.” Chemosphere, 46(5), pp 737-744 (2002)), the emphasis is on the Life Cycle model. The same fire numbers are briefly asserted, based on self-referencing to De Poortere et al 2000, several conference abstracts, and excerpted text from the original Simonson et al. LCA TV Case Study based on the SEMKO-Vällingby report. There is, however, no reference to this original Fire-LCA TV Case Study as a source of the fire numbers.

In this paper, it is asserted: “Finally, when considering the risk associated with the use of flame retardants, it is also important to consider the risk associated with fires. Based on the in-depth analysis of available fire statistics, conducted as a part of this study, it has been estimated that as many as 160 people may die each year in Europe as a direct result of TV fires and as many as 2000 may be injured in the same period.”

This is the fundamental problem with the Simonson et al. analysis: fire numbers, fire deaths and injuries are asserted with neither evidence (since the source data is not given) nor legitimate basis (because no death or injury numbers appear in the source SEMKO-Vällingby report, nor any data that would support the Simonson et al. estimate). There is no such “in-depth analysis” to be found.

When you look at the reports written by Simonson et al. which are under examination here, you see the identical sections of text, originally appearing in the 2000 LCA Model TV Case Study report, repeated over and again: in De Poortere et al 2000, presented as original work without reference; and over again in Simonson and Stripple 2000, referencing De Poortere et al 2000 and a conference abstract; and over again in Simonson, Tullin, and Stripple, 2002,  which again references De Poortere et al 2000 and a Simonson and De Poortere 1999 conference abstract. These reports and studies never describe the underlying primary methods and fire numbers from the SEMKO-Vällingby report. Moreover, these reports never attribute this repeated, identical text to the source of the original text in the Simonson et al 2000 LCA Model.

The SEMKO-Vällingby report is the primary methods and materials for all of these studies but is nowhere to be seen in any of the papers. Only with this information can a reviewer or general reader see how the fire numbers are produced within a methodology, in a reproducible manner, with estimates of accuracy, precision and confidence.

Concluding remarks

Given all of the above it is difficult, if not impossible, to understand how 8 fires of concern to the Swedish organisation SEMKO, in a relatively small suburb of Stockholm, Sweden, can be validly extrapolated to an EU population of 500,000,000: not only does it seem to be the case that the available data is used selectively, there are wide differences in climate, lifestyle, culture, built form, technological adoption, TV ownership and viewing frequency, while numbers generated in 1995/6 are still being taken as relevant to TV technology in 2013. The Simonson et al. fire numbers are no basis for fire prevention policy. These 8 fires do not a European conflagration make.

1 Comment »

RSS feed for comments on this post. TrackBack URI

  1. Reblogged this on jagabaldondominguez and commented:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Blog at
Entries and comments feeds.

%d bloggers like this: