A closer look at: School Playground Surfacing and Arm Fractures in Children: A Cluster Randomized Trial Comparing Sand to Wood Chip Surfaces by Dr. Andrew Howard
This study compares Engineered Wood Fiber (EWF) to granitic sand and was published on December 15, 2009 by the Public Library of Science (Plos).
As playground practitioners, we applaud all efforts to produce reliable injury data. As quoted in an article, written by the Canadian Association of Playground Practitioners:
“injury data is the fundamental basis for the technical and dimensional criteria found within the National Standard of Canada for Children’s Playspaces and Equipment (CAN/CSA-Z614)”. ***
However, this study’s premise is based on an inaccurate interpretation of the CSA standards and a disregard of the manufacturer and owner/operator specifications. Its narrow choice of data collection and its disregard of critical elements, all add up to a failed document. Its conclusions are not reliable and it’s assertion that its results are justification for changes to our standards are not congruous.
At issue are the following points:
- The interpretation of the CSA standards.
- The criteria used to develop the results and conclusions.
- The presentation of the study to the public and media.
- The Peer Review Process.
- Interpretation of the CSA standards.
The CAN/CSA-Z614 National Standard of Canada for Children's Playspaces and Equipment, while being a complex document, is presented in a format that allows the user to freely access information of specific interest. Section 10 of the document deals specifically with protective surfacing and the issues surrounding its compliance. A thorough knowledge of this section is necessary for all those who wish to study the properties of protective playground surfacing and to make conclusions or recommendations on its use.
“There is no CSA standard stating that 12 inches or 300mm of engineered wood fiber is required”.*
The CSA standard does give critical height figures for a 12” depth of surface, also for 9” depth and also for a 6” depth. So quoting part of a standard does not help”**
We have urged the author to re-read the CSA standards Z614-98, Z614-03 and Z614-07. The latter two CSA standards do not give critical height levels for different materials and for good reason.
CSA standards are constantly evolving and reviewed. The author is quoting from Table 1 in the 1998 version of the CSA standard. This 12 year old version was replaced by the revised version Z614-03 in which table 1 was removed. This removal was due to the fact that the CPSC testing used to create the table was site specific and could not be applied generally to all surfacing materials of the same generic type. Critical heights for generic use cannot be supplied. Critical heights can only be supplied for specific materials, tested under laboratory conditions and at specific ambient temperatures as per ASTM F1292.
The Z614-03 version was subsequently replaced by the Z614-07 version which was in effect at the time of the study and while not a requirement, it clearly states the recommended depth of 300mm for both EWF and Granitic sand as shown in the attached table D2††
“Playground surfacing is dynamic. What exists one day, cannot be assumed to exist on another”†
The CSA standards are clear:
Section 10: Critical Height
The surfacing material used within the protective surfacing zone of the piece of playground equipment shall have a critical height of at least the defined fall height.The following illustrates how the study’s premise is flawed. When we use the EWF critical height data shown in table 1 of the 1998 version (as the study did), a compressed depth of 225 mm (9 inches) is shown to produce a critical height of 1.8 m. The study’s stated, defined fall height ranged from 2.13 m to 2.18 m and the compressed surface depth was 178 mm (7 inches) - a higher fall height and less material. This shows quite clearly, by the author’s standards, that the surfaces in the study were not compliant.
If the CSA standard and the manufacturer say the surface depth should be a compacted 12” and no critical height analysis or impact attenuation testing was done, by what measure were these surfaces compliant? Playground surfacing is dynamic. What exists one day cannot be assumed to exist on another. To assert that the surfaces were compliant on installation and therefore, continued compliance is assured, does not hold water, especially when an installed, 8” depth was considered compliant. The author’s claim that the study is “comparing two different surfaces with similar compliant depths” is simply not accurate. We agree with the author that quoting part of a standard doesn’t help and this is precisely what he has done.
- The criteria used to generate the results and conclusions.
An examination of the study’s table 1 will show that it considered an appropriate material depth to be equal to or greater than 7 inches or 17.78cm. Further, depth measurements were only taken 3 times per year and only 72.5% of the EWF surface depths met these criteria. In other words, 27.5% of the studied wood fiber surfaces measured less the 7 inches. Again, the CSA standard and the manufacturer specifications both recommend a minimum maintained depth of 12" or 300cm to maintain a safe condition. Many playground practitioners would consider 7" of material to be a dangerous condition and would advise that the playground be taken out of service.
“All playgrounds were installed and maintained according to CSA playground standards”*
“Because most surfaces were constructed with an eight inch fill depth, we chose to report a seven inch threshold due to settling. The presence of surfacing material at this depth indicates maintained playgrounds”*
These are contradictory statements, spoken in the same breath. Further, the study's stated criteria appears to counter its' own claim that the surfaces were well maintained.
“The first three variables (impact attenuation, surface depth, and fall height) were all explicitly controlled in the study by installing CSA compliant playgrounds and measuring surface depths and equipment heights on every playground with and without actual falls”*
We have previously established on CSA table D2, what would be required for appropriate and recommended depths. Levels of 7” or less are clearly a misuse of the product. While the playground surfaces may or may not have been CSA compliant on installation, compliance was not necessarily the case at the fall site or the time of the fall. Further, the study has considered the equipment heights to be the actual heights of the fall. Equipment heights are not the fall heights. Falls may have occurred at varying heights. A fall from a 2 foot height would presumably result in a less severe injury than a fall from an 8 foot height but the actual heights of the falls were not recorded. In addition, impact attenuation readings were not recorded at the fall site or the time of the fall. Therefore, impact attenuation, fall heights and product depth cannot be considered “explicitly controlled in the study”. In fact, these elements were not controlled at all.
“The temperature and moisture variables are not reported in the paper”*
“No fractures occurred on either surface under frozen conditions”*
In order to gauge whether a surface is frozen, one must have reliable criteria to determine this. Some of the variables are: What percentage of the surface is frozen? Was it the top portion only or the entire surface? What are the qualifications needed by the person performing the test? What was the ambient temperature at the time of the fall? Did the surface melt and re-freeze? Did the tester do a probe test? According to the study, members of the school staff were given the responsibility of the determination but the study offered no criteria for establishing an accurate baseline or qualifications/training of the tester.
“Our data do, however, support the contention that granitic sand has a lower fracture rate than fibar, when used in similar depths, under similar fall heights, and in similar environmental conditions”.*
“data were collected at these 28 schools during the school months between January 2005 and June 2007"*
The data have not shown that depths, fall heights and environmental conditions were similar because none of these elements were recorded at the fall site or time of fall. We do not know and will never know if the conditions were similar. Further, the author does not explain how environmental conditions in January could be similar to those in June.
Particle size analysis was not performed on the products used in the study. Of the hundreds of thousands of EWF surfaces in North America, the study has observed the EWF surfacing at 12 of them. Company records indicate that those 12 surfaces consisted of a particular grind and species that is no longer used or available. Not only was the sample size grossly inadequate but the results are redundant and based on an obsolete product.
Finally, the study concludes that fracture rates are lower on sand because of lower surface friction and provides 2 explanations in support of this.
“Playground surface friction has been shown substantially lower for sand than for Fibar playgrounds [26] and this likely explains the protective effect”****
Support for this statement is given as reference #26 which refers to Chesney DA, Axelson PW (1996) Preliminary test method for the determination of surface firmness. IEEE Trans Rehabil Eng 4: 182–187.
This is a method developed by Beneficial Designs and is used in the determination of surface properties to meet the Americans with Disabilities Act (ADA) requirements for surface accessibility. This is not a friction test and has never been used in the study of injury. It does not compare granitic sand to EWF. It is a “Smart Wheel” test which utilizes a computerized wheelchair and has nothing to do with fractures or limbs being driven into a surface. Using this unrelated study to solidify the study’s position is inappropriate and should not be recognized as confirmation that Granitic sand has lower friction properties than EWF.
“Granitic sand as specified for this study has very uniform and very round particles, which maximize its fluid-like properties and minimize surface friction”****
The manufacturer’s specifications for the Granitic sand as specified for this study show this sand as having sub-angular to slightly rounded particles. This would place the product on the mid to below mid range on the roundness scale. The Granitic sand as specified for this study does not have very rounded particles as the study claims.
- Presentation of the study to the media and public.
The study and resulting media coverage have consistently referred to "Granitic Sand" as "Sand". This is evident in the study’s title and many times throughout the body of its presentation. There are many types of sand used as playground surfaces around the world and most do not provide the qualities of Granitic Sand. Granitic sand is only available in certain geographic locations and freight costs to other parts are prohibitive. To generalize and conclude that “Sand is safe or safer" is irresponsible and could be damaging to the industry. Owner operators around the world could now conclude that any sand is safe or safer when in fact this is not the case. The study makes no attempt to educate the public in this regard.
- The Peer Review Process.
One has to ask, how did this study with all its blatant errors make it to publication? Peer reviewers have the power to demand changes to a manuscript. When making our feelings known to the CBC reporter who broke the story, he replied; “It was peer reviewed, it must be legitimate”. People place a great deal of faith in the peer review process but what does it really mean? Well it should mean that if you are publishing a document about a product and industry that is outside of the author’s area of expertise, you had better be using a peer reviewer who is from that industry. And if you are publishing a document that is quoting and recommending changes to our national standard, you had better be using a peer reviewer who is from that standard body or is well versed in that standard.
- Conclusion.
It’s like taking a trampoline and cutting its legs down so that it is say 1 foot above the ground and then standing back to count the broken ankles. Then in turn, saying ‘see, this trampoline is not as good at reducing fractures as some other trampolines’. It is elementary and straightforward that the trampoline was never designed to have its leg cut off. Reliable observations and conclusions about a product cannot be gleaned from its improper use. Unfortunately, this is what this study has done and is part of the reason why it is a failed document. At best, it might conclude that the improper use of surfacing products could lead to increased injuries. Our advice to the publisher is to pull this article before any further damage is done. Our recommendation to the author is to go back to square one and start a legitimate study that involves the industry experts. Let’s come up with a reliable and useful study that is beneficial to the public and relevant to our children’s safety.
Roger Williams CCPI
Per Playcare Inc.
* This quote can be found in the comments section of the Plos web site** This quote was posted by the author on the Plos open access website but later disappeared from public access.
*** This article can be found in the CAPP Articles Section
**** This quote can be found in the body of the study under the heading ‘Discussion’
† Quotation by Roger Williams
†† Table D2:Z614-07 © Canadian Standards Association
110 March 2007
Table D.2
Loose fill protective surfacing material and critical height range
(See Clause D.2.)
Loose fill protective surfacing
Material / Recommended minimum depth of material (compacted) / Critical height
Wood/bark mulch 300 mm (11.81 in) Up to 3 m (118.11 in)
Engineered wood fiber 300 mm (11.81 in) More than 3 m (118.11 in)
“Washed”, round, pea gravel* 300 mm (11.81 in) Up to 2.5 m (98.43 in)
Specified sand† 300 mm (11.81 in) More than 2.5 m (98.43 in)
Shredded tire crumb 200 mm (7.87 in) More than 3 m (118.11 in)