Data produced with federal funds are required to include standardized metadata, which provide common attributes in a consistent format that make it possible to document diverse data collections. NCEI follows the International Organization for Standardization (ISO) series of standards, such as 19115, 19115-2, and 19139. ISO 19115 and the associated standards are endorsed by the Federal Geographic Data Committee (FGDC).
Sample XML for a NOAA Climate Data Record
Other kinds of metadata or data documentation should also be considered, such as NetCDF Templates, platform and sensor metadata, data management plans, and others that vary by project. See the Metadata Templates and Guidance section for more information.
Collect the Neccesary Information
Plan Ahead
Planning for metadata development during the initial planning stages of a project saves time, effort, and money. Establishing formats and protocols for data documentation is crucial. It is much easier to document these elements during the data development phase than trying to capture them retroactively.
Document Core Fields
The following descriptive metadata elements are required for all NOAA environmental data. NCEI uses management and submission tools to confirm that submissions include these specific fields and values.
Review Metadata Templates and Guidance
NCEI has developed a Collection Level Metadata Template containing technical and content specifications to further guide data providers while creating and maintaining metadata. These specifications are meant to ensure consistency and interoperability of NCEI metadata records. This guidance document is meant to be used by NCEI staff who manage collection level metadata or develop tools to support metadata management.
NCEI NetCDF Templates allow data provider to create NetCDF datasets that conform to Unidata’s Attribute Convention for Dataset Discovery (ACDD) and Climate and Forecast (CF) Conventions. These templates are provided as guidance to our data providers, and are also used for internal data development efforts.
ISO Workbook
The ISO workbook was created to act as an educational and implementational guide to be used in conjunction with ISO 19115-2 Geographic information ─ Metadata Part 2: Extensions for imagery and gridded data ISO 19115-2:2009(E).
Standardize Format and Style
- Use Plain Language: Don’t assume readers will have background knowledge of your subject matter. Explain and articulate foundational concepts, even if they seem basic.
- Follow a Stylesheet: Develop and apply standard grammar, style, and format conventions, and use them consistently throughout the document. Always place similar references in the same spot in your metadata files. Use consistent style and naming conventions for titles within your organization. Spell out abbreviations at the beginning of a paragraph or file if you plan to use them.
Use Persistent Identifiers
NCEI recommends using persistent identifiers (PIDs) to identify objects, people, and organizations in your metadata. PIDs can make your metadata more consistent, and facilitate linkages with other information systems.
NCEI assigns Digital Object Identifiers (DOIs) to its data using the metadata. Open Researcher and Contributor IDs (ORCID® iDs) are used to identify people, and Research Organization Registry (ROR) IDs are used to identify organizations.
Review
Review your files for accuracy and completeness. Add any details or context needed to ensure the data can be used accurately. Have someone else perform a second review.
Metadata Submission
Metadata submissions coincide with data submissions. There are multiple ways to submit metadata to NCEI. Send2NCEI (S2N) and Advanced Tracking and Resource Tool for Archive Collections (ATRAC) are data submission portals that guide data providers through the submission process. These portals have built-in requirements that match NCEI metadata requirements to help with metadata creation. For more information about archiving data at NCEI, see the Archive page.
Data providers may use one of the data submission interfaces hosted by NCEI to provide metadata. Each tool has its own unique user interface and provides helpful guidance on submitting data and metadata to NCEI.
Other Data-Type Specific Submission Methods
Many types of data archived at NCEI have their own specific metadata needs and data submission methods. If the methods listed above do not meet your needs, please contact NCEI for more information.
Metadata Management Tools
In addition to archive tools like Send2NCEI that integrate metadata generation into the submission process, NCEI has also developed a suite of tools called Collection Manager to support the creation, maintenance, and publication of metadata. Collection Manager allows NOAA users to import, create, update, store, reuse, export, validate, assess, and publish collection-level metadata. This system ensures that all metadata meet NOAA metadata standard practices. The following are part of the Collection Manager suite. The user guide for each tool are linked in the dropdown sections below.
The Collection Metadata Editing Tool (CoMET) is a metadata creation and editing software, and a helpful tool for meeting all NCEI and NOAA standards and best practices to support FAIR data principles.
Features
- Uploading, creating, editing, deleting, and exporting records with full revision history support
- Displaying various views (landing page, simple ISO, etc.)
- Validation and evaluation, including NOAA Rubric completeness scoring
- Publication and review workflows
- Inserting references to components (in Docucomp, for example)
- Rich web service API, and user interface built on that API
- Data Stewardship Maturity Questionnaire (DSMQ)
- Upload records to check if XML is well-formed
- Validate, translate, resolve, and improve ISO 19115-2 metadata records
DSMQ
The Data Stewardship Maturity Questionnaire (DSMQ) is an editor within CoMET that assigns Data Stewardship Maturity Matrix (DSMM) scores to values entered into a questionnaire by the user. Records created with an associated DSMQ result in higher relevance ranking within OneStop. These records will be listed before similar records without Data Stewardship Maturity Matrix (DSMM) scores or records with lower DSMM scores.
Resources
Docucomp is a limited access application that allows NOAA staff with an account to create, edit, and manage Extensible Markup Language (XML) components that are reusable by the data management and scientific community.
- Manages reusable components referenced by URLs within collection records
- Enables many collections to use the same component
- Increases consistency and reduces maintenance effort
- Components are inserted into metadata records via Xlinks
- Component Groups are used to group related components together
- Editors can create and manage components within their own groups
NOAA Staff Access
Currently, only NOAA staff can manage components in Docucomp.
Other Permissions
- View Component
- Edit Component
- Copy Xlink
- View Revision History
- View XML
- Delete this component
Public Access
Non-NOAA users and staff without a login can still view components, copy Xlinks, and view XML files created in Docucomp.
Resources
Metaserver is responsible for Web Accessible Folder (WAF) ON-DEMAND and evaluating metadata records
Web Accessible Folders (WAF)
Controls frequency of processing WAFs
Record Services (function also available in CoMET)
- Upload records to check if XML is well-formed
- Validate, translate, resolve, and improve ISO 19115-2 metadata records
- Generate NCEI Landing page view