NASA sponsors independent peer reviews of its K-12 space science education materials. The purpose of the reviews is to ensure that education products distributed by NASA are of high quality and meet rigorous standards, as well as to provide feedback to product developers from educators and scientists. The reviews are conducted for NASA by the Institute for Global Environmental Strategies (IGES) - a non-profit education organization.
Education products are reviewed by panels including both scientists and educators (e.g., classroom teachers, college faculty, NASA teacher trainers, curriculum developers, and national education standards experts). This peer review provides NASA with information regarding the effectiveness of specific products and recommendations for distribution.
IGES has developed the following guidelines to help education product developers design outstanding education materials for K-12 education audiences.
The Path to Excellence (or Avoiding Common Pitfalls)
Communications Materials Review (CMR)
The following is a list of some of the most common areas in which products that don't pass the review have needed improvement. While this list is not exhaustive, we strongly believe that if you address all of these items, then you will have developed an outstanding NASA education product.
Conduct Formative Reviews and Field Testing
All education products should go through an appropriate formative review and testing before they are submitted to the NASA education products review. What is a reasonable review? It's one in which the product is reviewed by appropriate science content experts, as well as field tested and/or reviewed with the target audience (or number of audiences if the product is aimed at a broad range of grade levels or audiences). Any shortcomings that are found should be corrected.
Target a Specific Audience
Products that are developed for a very broad audience, tend to not serve any specific audience well. Often a product will be identified as K-12. Unless that product is actually broken into sections that are specifically designed for a smaller grade bands (e.g., K-1, 2-4, 5-6, 7-8, 9-10, 11-12), it is not likely to be useful to that wide an education range. Even products that are identified as appropriate for K-4 are attempting to serve students with very different capabilities and interests.
Design Products for the Targeted Audience
We see many products that are not appropriate for the targeted audience. They are written at too high (or low) a level, the concepts are too complex (or simple), or the graphics aren't age appropriate for the target audience. One common mistake is to pair text that is written at a higher level, with graphics that would appeal to younger children. Another problem often identified by reviewers is an inconsistent reading level. Individual paragraphs, pages or sections show a marked increase or decrease in difficulty. Watch for consistency throughout the product.
Support National Education Standards
Identify the primary national education standards (science, mathematics, technology, and/or geography) that the product addresses. Be selective and specific about what standards are supported and identify how the product helps a teacher address those standards. Rarely does a product address several standards – list only those standards that the product supports in a meaningful, and not superficial, way.
Incorporate Student-Centered Approach
While some teachers will take NASA content and incorporate it into an investigative, student-centered teaching/learning style, developers should build this kind of strategy in from the start. Many still use multiple choice and "cookbook" style activities; to do well in the review, developers need to incorporate activities that provide inquiry-based, problem-solving opportunities.
Provide Appropriate Assessments
How can a teacher using this education product evaluate student performance? Does the teachers guide provide sample rubrics, grading/scoring guidelines, answer keys for specific questions, or other appropriate assessments/resources?
References and Further Investigations
Provide accurate, accessible and age appropriate references for outside investigation. This can include books, magazine articles, journal publications, Web sites, etc. (see also "Maintain a Web" below). Exercises, labs, and slightly more advanced investigations are a plus for students willing to go further with the material.
Provide Color Keys/Legends/Units of Measurement
Most teachers and students are not familiar with NASA imagery. Be sure to provide the tools that are essential for interpreting images. At a minimum, provide color keys/scales, legends, captions, and identify the units of measurement that are used. (See "Go Metric," below)
Maintain a Web, not Cobweb, Site
Regularly check and update WWW links. One solution is to host all critical WWW pages (so that you can ensure the links are kept active) or obtain permission to mirror critical sites.
Go Metric
Metric measurements are the agency standard and should be consistently used. Including English equivalents is also very good, as well as other comparisons that would be meaningful to the target audience (e.g., 1.35 mm or as thick as one dime). Metric units should be given first, with equivalent measurements in parentheses, e.g., 10 km (6.2 miles).
Check your Math
Does your product include math problems (or metric equivalents/conversions)? Be sure to check your math and make sure your answers and conversions are correct.
Address Accessibility and Section 508
NASA electronic and information technology-based education products (e.g., WWW, DVD, CD, videotape) must meet Section 508 requirements. Section 508 of the Rehabilitation Act requires that Federal agencies' electronic and information technology is accessible to people with disabilities, including employees and members of the public. This includes WWW sites and PDF files, as well as providing closed captioning and audio for videotapes, DVDs, and CDs. For information on Section 508, visit http://www.section508.gov/. For a checklist for making a site accessible, visit: http://web508.gsfc.nasa.gov/developing/checklist/index.cfm.
Follow Visual Standards for NASA Communications Products
NASA communications products, which include education products, should follow
the design guidelines presented in Visual Standards for NASA Communications
Products, which is available at http://communications.hq.nasa.gov.
This site is only accessible within NASA.
A copy of the guidelines is also available for those outside NASA at: http://www.strategies.org/SMD_CommunicationsReview/interim_style_guide.pdf.
While this document is noted as an "interim" guide, the final guidelines are
not expected to differ from this document, but to provide additional details
that clarify information in the interim guide.
Conduct Market Research. Talk to educators and identify whether a need exists for your product. Consult the NASA Space Science Resource Directory at http://teachspacescience.stsci.edu/ to ensure a comparable NASA product doesn't already exist.
Ensure the proposed product would be relevant to and support NASA Space Science Research and Education Priorities. Information on NASA space science and education programs can be found at http://science.hq.nasa.gov/.
Review Visual Standards for NASA Communications Products
NASA communications products, which include education products, should
follow Visual Standards for NASA Communications Products, which is available
at http://communications.hq.nasa.gov.
This site is only accessible within NASA. A copy of the guidelines
is also available for those outside NASA at: http://www.strategies.org/SMD_CommunicationsReview/interim_style_guide.pdf.
While this document is noted as an "interim" guide, the final
guidelines are not expected to differ from this document, but to provide
additional details that clarify information in the interim guide.
The only permissible NASA logo to use is the official NASA "meatball" insignia. Education products printed by NASA should include the NASA "meatball" insignia, as well as the NASA publication number and education product reference box. Guidelines for proper use of the NASA logo can be found in the in Visual Standards for NASA Communications Products, which is available at http://communications.hq.nasa.gov. This site is only accessible within NASA. A copy of the guidelines is also available for those outside NASA at: http://www.strategies.org/SMD_CommunicationsReview/interim_style_guide.pdf. Guidelines for the education publication number and education product reference box are included in the NASA Curriculum Support Product Development and Dissemination Guide, which is available from NASA Center Education Offices.
(Web, Video, CD, DVD, Software)
http://section508.nasa.gov/ - NASA Website for section 508. The site includes an overview of section 508 requirements and contact information for Section 508 coordinators at NASA HQ and field centers. Note: some sections are restricted to NASA internal use only.http://serch.cofc.edu/special/overview.htm - Website by the NASA Southeast Regional Clearinghouse (SERCH), which provides excellent resources regarding accessibility and Section 508, including information on best practices and designing accessible education resources.http://webxact.watchfire.com/ - WebXACT is a free online service that lets you test single pages of web content for quality, accessibility, and privacy issues.
Most products submitted to the Space Science review contain very good resources and great content. However, most still do not incorporate learning styles that are in tune with the national education standards and systemic change efforts. While some teachers will take this content and incorporate the material into an investigative, student-centered teaching/learning style, developers should build this kind of strategy into their education products. Many still use multiple choice and programmed learning styles; to do well in the review, developers need to incorporate activities that provide intuitive problem-solving opportunities.
Curriculum materials should be well-aligned with appropriate national education standards, such as science, mathematics, technology, and/or geography. Other standards should be identified as appropriate (e.g., English language arts – reading, writing, and speaking). Content should match the standards and specific outcomes that are identified.
National standards can be found online at:
In general, it is expected that NASA education materials will support national standards; in some cases it may be appropriate to identify relevant state standards, either in addition to or in place of national standards.
back to topProduct developers should ensure that the reading level of their product matches that of their intended audience. This is especially true for products targeted for elementary students – many K-4 materials that are submitted for review are too sophisticated for the target audience.
There are guidelines for assessing and validating the reading levels of products. Many word processing applications include readability formulas. For example, Microsoft includes the Flesch Reading Ease Score and Flesch-Kincaid Grade Level Score in their tools. Under the "Tools" menu, select "Spelling and Grammar" and then "Options." You can then select "Show readability statistics." After running the spell checker, the reading level will be displayed with the results.
The Fog Index is another method for analyzing written materials by computer or by hand and is used primarily in education. If your grammar checker doesn't include the reading level, let it count the number of sentences and words and plug them into the Fog formula below:
Caveat: Please be aware that these formulas are based on measuring words and sentences, but they cannot measure indicators such as how complex the ideas are, whether the content is in a logical order, and if the vocabulary is appropriate and the writing engaging for the intended audience. Slavishly adhering to these formulas, for example to produce text for younger readers, can result in a monotonous writing style consisting of several short sentences.
Therefore, these formulas should be used as a general guide and reading level/suitability should also be determined in combination with other assessments. For example, test student comprehension and engagement and use experts, such as teachers and librarians, to review material and assign an appropriate grade level to the text.
Product developers targeting a broad education range need to be aware of the developmental differences in that range. This is especially true for products targeted for K-4. For example:
Rarely can an education product be used effectively for as broad an audience as K-12. Unless that product is actually broken into sections that are specifically designed for a smaller grade bands (e.g., K-1, 2-4, 5-6, 7-8, 9-10, 11-12), it is not likely to be useful to that wide an education range.
Call for Products and Schedule—the latest schedule, guidelines, product submission form, and review criteria are available at: http://www.strategies.org/SpaceScienceReview.html
Product Submission Form. Submit your product with a copy of the Product Submission Form, which will be available with the call for products. Send 12 copies or sets of the best-quality print available. Some materials that are expensive to produce (e.g., color posters) can be submitted on a Web site for reviewers to assess.
Reviewers receive a copy of the product submission form for every product that they review. This form collects information that our reviewers have consistently found to be very useful in their assessments. Please provide as complete information as possible. Developers are also encouraged to include any additional information that they believe will help reviewers evaluate the product. For example:
Education products that pass the SMD review will then need to pass the NASA HQ Communications Materials Review (CMR) to be printed and/or distributed by NASA. The CMR takes approximately 10 days, and it is a requirement that before entering the communications review, education products must have passed the SMD education product review. Products are entered into the CMR through the Website: http://communications.nasa.gov.
Theresa Schwerin
theresa_schwerin@strategies.org
Associate Director, Education Programs
Institute for Global Environmental Strategies
October 2005