If you would like to help us understand our user community better, please fill out this short Paleoclimatology User Survey
NCEI Paleo Web Service
This page documents the URL formats and parameters used to get data from the NCEI Paleo web service.
All calls to the web service should be GET requests. Successful requests return a JSON or XML object with two fields: contactInfo which is the same for every request, and studywhich contains a list of study objects. Requests that do not use the headersOnly flag — i.e. those that return full studies — are limited to at most 10 results, but will include a page node with links to the next and previous page of results.
Note that when searching via https://www.ncei.noaa.gov/access/paleo-search/, the api call used to retrieve the results is displayed. This can be used to generate examples relevant to your use case.
Sections on this page:
Examples
The base URL for the web service is located here:https://www.ncei.noaa.gov/access/paleo-search/study/search[.json|.xml]
The return type, JSON or XML, can be specified in the URL. The default is XML.
If you know the id of the study you want, you can provide it as a query parameter:.../search.json?xmlId=14323.../search.json??NOAAStudyId=16617
If you don't know exactly which study you want, or you want a collection of studies, you can search the service with the available parameters:.../search.json?dataPublisher=NOAA&dataTypeId=18&locations=Continent>Africa>Eastern Africa>Kenya.../search.json??dataPublisher=Neotoma&investigators=Wahl, Eugene Richard
Available parameters are documented below. A list of searchable values for several parameters (investigators, locations, keywords, and tree species) can be found here, with an optional dataTypeId parameter:https://www.ncei.noaa.gov/access/paleo-search/study/params.jsonhttps://www.ncei.noaa.gov/access/paleo-search/study/params.json?dataTypeId=18
Some parameters can only be used when searching specific data publishers, these are noted next to the parameter name. Arguments to all parameters exceptsearchText must be complete values, e.g. partial names are not permitted. For investigators, locations, keywords, and tree species these values are listed in the params.json file, and multiple values can be searched by joining them with a ‘|’.
The following is example code that demonstrates the use of the WDS-Paleo API in a search, access, and analysis workflow in the R and Python programming languages.
#!/usr/bin/env python#### Demonstrate use of WDS-Paleo API in a programmatic search, access, and analysis workflow## Search via API for sites meeting criteria, extract site latitudes from returned metadata,## plot latitudes in histogram##import matplotlib.pyplot as pltimport numpy as npimport jsonimport requests ## Search via API for speleothem (dataTypeId=17) studies with magnesium/calcium data,## retrieving "pages" of 10 studies at a time (limit=10). Setting this limit is a best## practice to avoid overwhelming the API with large requests. Results are returned in## JSON format as a dictionary called "json_data." Note that when searching via the user## interface at: ncei.noaa.gov/access/paleo-search/, the API call used to retrieve## the results is displayed immediately above the search results and can be used to## generate search strings ("req_str") relevant to your use case. ## specify API requestapi_base = "https://www.ncei.noaa.gov/access/paleo-search/study/search.json?"req_params ="dataPublisher=NOAA&dataTypeId=17&cvWhats=chemical%20composition%3Eelement%20or%20compound%20ratio%3Emagnesium%2Fcalcium&limit=10"req_str = api_base + req_params lats = [] # initialize empty vector to store latitudes ## retrieve search results from APIwhile req_str: # loop through search results one page at a time speleos = requests.get(req_str) if not speleos.content: print("No results from search") break json_data = json.loads(speleos.text) # load JSON-formatted search results ## extract latitudes numstudies = len(json_data['study']) # count number of studies returned study_list = json_data['study'] # extract list of studies returned for i in range(numstudies): # loop through studies numsites = len(study_list[i]['site']) # count number of sites in the study for j in range(numsites): # loop through sites within the study # extract latitude, convert to float, and append to vector lat_value = json_data['study'][i]['site'][j]['geo']['geometry']['coordinates'][0] lats.append(float(lat_value)) req_str = json_data['page'][0]['next'] # specify API request for next page of studiesnp_lats = np.array(lats) # convert list to numpy array required for histogram ## plot histogram of study latitudesfig = plt.figure(figsize=(10,7))ax = fig.add_subplot(1, 1, 1, title="Distribution of datasets by latitude", xlabel="Latitude", ylabel="Frequency")ax.hist(np_lats,range=[-40,60])plt.show()
Query parameters
xmlIdNOAANEOTOMAPANGAEA
The id of the original xml document containing the study.
xmlId=([0-9]+|)[0-9]+.../study/search.json?xmlId=1840The id of the original xml document containing the study.
NOAAStudyIdNOAA
The unique NOAA study id number.
NOAAStudyId=([0-9]+|)[0-9]+.../study/search.json?NOAAStudyId=13156The unique NOAA study id number.
searchTextNOAANEOTOMAPANGAEA
Any text contained in the contents of the study.
This field supports Oracle Text parameters, such as AND, OR, and wildcards (% for multiple characters, _ for single character). Note that some characters (e.g. -, ;, ~) have special meaning in this context. This field cannot contain HTML reserved characters (e.g. <, >, &). Special characters can be searched literally by escaping them with a backslash, as discussed in the Oracle Special Characters documentation.
searchText=[a-zA-Z0-9%_-;~]+.../study/search.json?searchText=younger dryas.../study/search.json?searchText=loess AND stratigraphy.../study/search.json?searchText=stratigra%25.../study/search.json?searchText=f_re.../study/search.json?searchText=noaa%5C-tree%5C-19260Any text contained in the contents of the study.
This field supports Oracle Text parameters, such as AND, OR, and wildcards (% for multiple characters, _ for single character). Note that some characters (e.g. -, ;, ~) have special meaning in this context. This field cannot contain HTML reserved characters (e.g. <, >, &). Special characters can be searched literally by escaping them with a backslash, as discussed in the Oracle Special Characters documentation.
dataPublisherNOAANEOTOMAPANGAEA
The original provider of the study data.
dataPublisher=[NOAA|NEOTOMA|PANGAEA].../study/search.json?dataPublisher=NOAA&dataTypeId=4The original provider of the study data.
dataTypeIdNOAANEOTOMA
Limit the search to only studies of a certain data type, or which contain variables of that data type. Multiple data types can be selected at once like
Available data type id's are as follows:
dataTypeId=([0-9]{(1, 2)}|)*[0-9]{(1, 2)}.../study/search.json?dataPublisher=NOAA&dataTypeId=4Limit the search to only studies of a certain data type, or which contain variables of that data type. Multiple data types can be selected at once like
dataTypeId=18|5.Available data type id's are as follows:
- 1: BOREHOLE
- 2: CLIMATE FORCING
- 3: CLIMATE RECONSTRUCTIONS
- 4: CORALS AND SCLEROSPONGES
- 6: HISTORICAL
- 7: ICE CORES
- 8: INSECT
- 9: LAKE LEVELS
- 10: LOESS
- 11: PALEOCLIMATIC MODELING
- 12: FIRE HISTORY
- 13: PALEOLIMNOLOGY
- 14: PALEOCEANOGRAPHY
- 15: PLANT MACROFOSSILS
- 16: POLLEN
- 17: SPELEOTHEMS
- 18: TREE RING
- 19: OTHER COLLECTIONS
- 20: INSTRUMENTAL
- 59: SOFTWARE
- 60: REPOSITORY
minLat/maxLatNOAANEOTOMAPANGAEA
Valid latitude values are in the range -90 to 90.
minLat=-?[0-9]{(1, 2)}maxLat=-?[0-9]{(1, 2)}.../study/search.json?minLat=68&maxLat=69Valid latitude values are in the range -90 to 90.
minLat and maxLat can be used individually or together.minLon/maxLonNOAANEOTOMAPANGAEA
Valid longitude values are in the range -180 to 180.
minLon=-?[0-9]{(1, 3)}maxLon=-?[0-9]{(1, 3)}.../study/search.json?minLon=68&maxLon=69Valid longitude values are in the range -180 to 180.
minLon and maxLon can be used individually or together.minElev/maxElevNOAANEOTOMA
Elevation bounds in meters. Elevation can be less than 0.
minElev=-?[0-9]+maxElev=-?[0-9]+.../study/search.json?minElev=100&maxElev=110Elevation bounds in meters. Elevation can be less than 0.
minElev and maxElev can be used individually or together.earliestYear/latestYearNOAA
Apply a time bound to the search. These fields should be used in conjunction with
earliestYear=-?[0-9]+latestYear=-?[0-9]+.../study/search.json?earliestYear=1500&latestYear=0&timeFormat=BP&timeMethod=overEntire&dataTypeId=13Apply a time bound to the search. These fields should be used in conjunction with
timeFormat and timeMethod.earliestYear and latestYear can be used individually or together.timeFormatNOAA
Specify the time unit format, either years CE or years BP, to use when searching. Choice of time format will not affect search results (when using properly converted dates) and is provided for convenience only.
timeFormat=[CE|BP].../study/search.json?earliestYear=1500&latestYear=0&timeFormat=BP&timeMethod=overEntire&dataTypeId=13Specify the time unit format, either years CE or years BP, to use when searching. Choice of time format will not affect search results (when using properly converted dates) and is provided for convenience only.
timeMethodNOAA
Specify the way in which to apply the time bounds defined by
timeMethod=[overAny|entireOver|overEntire].../study/search.json?earliestYear=1500&latestYear=0&timeFormat=BP&timeMethod=overEntire&dataTypeId=13Specify the way in which to apply the time bounds defined by
earliestYear and latestYear.- overAny: Matches studies where any part of the time coverage lies between the specified min and max
- entireOver: Matches studies with time coverages that completely bound the specified min and max
- overEntire: Matches studies with time coverages that are completely bounded by the specified min and max
investigatorsNOAANEOTOMAPANGAEA
Search for studies containing one or more investigators. The available investigators, grouped by data publisher and data type, can be found in the params.json file.
investigators=([a-zA-Z]+|)*[a-zA-Z]+.../study/search.json?investigators=Wahl, E.R.Search for studies containing one or more investigators. The available investigators, grouped by data publisher and data type, can be found in the params.json file.
investigatorsAndOrNOAANEOTOMAPANGAEA
When searching for multiple investigators, specifies whether to find studies containing any of those investigators or all of them. If this field is not supplied the default value is
investigatorsAndOr=[and|or].../study/search.json?investigators=Wahl, E.R.|Vose, R.S.&investigatorsAndOr=orWhen searching for multiple investigators, specifies whether to find studies containing any of those investigators or all of them. If this field is not supplied the default value is
or.locationsNOAA
Search for studies containing one or more locations. The available locations, grouped by data publisher and data type, can be found in the params.json file.
locations=([a-zA-Z>]+|)*[a-zA-Z>]+.../study/search.json?locations=Continent>Africa>Eastern Africa>ZambiaSearch for studies containing one or more locations. The available locations, grouped by data publisher and data type, can be found in the params.json file.
locationsAndOrNOAA
When searching for multiple locations, specifies whether to find studies containing any of those locations or all of them. If this field is not supplied the default value is
locationsAndOr=[and|or].../study/search.json?locations=Continent>Africa>Eastern Africa>Zambia|Continent>Africa>Eastern Africa>Zimbabwe&locationsAndOr=orWhen searching for multiple locations, specifies whether to find studies containing any of those locations or all of them. If this field is not supplied the default value is
or.keywordsNOAA
Search for studies containing one or more keywords. The available keywords, grouped by data publisher and data type, can be found in the params.json file.
keywords=([a-zA-Z>]+|)*[a-zA-Z>]+.../study/search.json?keywords=earth science>paleoclimate>paleocean>biomarkersSearch for studies containing one or more keywords. The available keywords, grouped by data publisher and data type, can be found in the params.json file.
keywordsAndOrNOAA
When searching for multiple keywords, specifies whether to find studies containing any of those keywords or all of them. If this field is not supplied the default value is
keywordsAndOr=[and|or].../study/search.json?keywords=earth science>paleoclimate>paleocean>biomarkers|earth science>paleoclimate>paleocean>lithology&keywordsAndOr=orWhen searching for multiple keywords, specifies whether to find studies containing any of those keywords or all of them. If this field is not supplied the default value is
or.speciesNOAA
Search for studies containing one or more tree species. Species should be indicated by their four letter code. The available species codes, grouped by data publisher and data type, can be found in the params.json file.
species=([a-zA-Z]4|)*[a-zA-Z]4.../study/search.json?species=ABALSearch for studies containing one or more tree species. Species should be indicated by their four letter code. The available species codes, grouped by data publisher and data type, can be found in the params.json file.
speciesAndOrNOAA
When searching for multiple species, specifies whether to find studies containing any of those species or all of them. If this field is not supplied the default value is
speciesAndOr=[and|or].../study/search.json?species=ABAL|AGRO&speciesAndOr=orWhen searching for multiple species, specifies whether to find studies containing any of those species or all of them. If this field is not supplied the default value is
or.cvWhatsNOAA
Search for studies containing one or more terms from the PaST Thesaurus describing the quantity measured ("What"). The available "What" terms, grouped by data publisher and data type, can also be found in the params.json file.
cvWhats=([a-zA-Z>]+|)*[a-zA-Z>]+.../study/search.json?cvWhats=chemical composition>compound>inorganic compound>carbon dioxideSearch for studies containing one or more terms from the PaST Thesaurus describing the quantity measured ("What"). The available "What" terms, grouped by data publisher and data type, can also be found in the params.json file.
cvWhatsAndOrNOAA
When searching for multiple "What" terms from the PaST Thesaurus, specifies whether to find studies containing any of those terms or all of them. If this field is not supplied the default value is
cvWhatsAndOr=[and|or].../study/search.json?cvWhats=chemical composition>compound>inorganic compound>carbon dioxide|chemical composition>compound>organic compound>hydrocarbon>alkane>n-alkane>methane&cvWhatsAndOr=orWhen searching for multiple "What" terms from the PaST Thesaurus, specifies whether to find studies containing any of those terms or all of them. If this field is not supplied the default value is
or.cvMaterialsNOAA
Search for studies containing one or more terms from the PaST Thesaurus describing the material on which a measurement was made ("Material"). The available "Material" terms, grouped by data publisher and data type, can also be found in the params.json file.
cvMaterials=([a-zA-Z>]+|)*[a-zA-Z>]+.../study/search.json?cvMaterials=geological material>identified mineral>carbonate>calcium carbonate>aragoniteSearch for studies containing one or more terms from the PaST Thesaurus describing the material on which a measurement was made ("Material"). The available "Material" terms, grouped by data publisher and data type, can also be found in the params.json file.
cvMaterialsAndOrNOAA
When searching for multiple "Material" terms from the PaST Thesaurus, specifies whether to find studies containing any of those terms or all of them. If this field is not supplied the default value is
cvMaterialsAndOr=[and|or].../study/search.json?cvMaterials=geological material>identified mineral>carbonate>calcium carbonate>aragonite|biological material>bulk biological material>peat&cvMaterialsAndOr=orWhen searching for multiple "Material" terms from the PaST Thesaurus, specifies whether to find studies containing any of those terms or all of them. If this field is not supplied the default value is
or.cvSeasonalitiesNOAA
Search for studies containing one or more terms from the PaST Thesaurus describing the season being explicitly reconstructed ("Seasonality"). The available "Seasonality" terms, grouped by data publisher and data type, can also be found in the params.json file.
cvSeasonalities=([a-zA-Z>]+|)*[a-zA-Z>]+.../study/search.json?cvSeasonalities=annualSearch for studies containing one or more terms from the PaST Thesaurus describing the season being explicitly reconstructed ("Seasonality"). The available "Seasonality" terms, grouped by data publisher and data type, can also be found in the params.json file.
cvSeasonalitiesAndOrNOAA
When searching for multiple "Seasonality" terms from the PaST Thesaurus, specifies whether to find studies containing any of those terms or all of them. If this field is not supplied the default value is
cvSeasonalitiessAndOr=[and|or].../study/search.json?cvSeasonalities=annual|3-month>Aug-Oct&cvSeasonalitiessAndOr=orWhen searching for multiple "Seasonality" terms from the PaST Thesaurus, specifies whether to find studies containing any of those terms or all of them. If this field is not supplied the default value is
or.reconstructionsOnlyNOAA
Specify whether to return only studies which are, or are not, climate reconstructions. By default both are returned.
reconstructionsOnly=[Y|N].../study/search.json?headersOnly=true&dataPublisher=NOAA&dataTypeId=4&reconstructionsOnly=YSpecify whether to return only studies which are, or are not, climate reconstructions. By default both are returned.
headersOnlyNOAANEOTOMAPANGAEA
Returns only minimal information for each study found , but is much quicker than a full query. If you are performing a search that returns a large number of studies and intend to choose only one from that list to examine in detail,
headersOnly=true.../study/search.json?species=ABAL&headersOnly=trueReturns only minimal information for each study found , but is much quicker than a full query. If you are performing a search that returns a large number of studies and intend to choose only one from that list to examine in detail,
headersOnly may give you enough information to find the xmlId or NOAAStudyId for the study that you actually want.skip/limitNOAANEOTOMAPANGAEA
Retrieves only a specific number of results from a search, and chooses where that number should begin. When doing a broad search that takes too long or returns too much data at once, these options can be used to get the same data in smaller pieces. The default, and maximum, limit is 10 studies. Links are provided in the response JSON to the next and previous page of results.
skip=[0-9]+limit=[0-9]+.../study/search.json? dataPublisher=NOAA&investigators=Abbott,%20M.B.&limit=10.../study/search.json? dataPublisher=NOAA&investigators=Abbott,%20M.B.&skip=3.../study/search.json? dataPublisher=NOAA&investigators=Abbott,%20M.B.&limit=10&skip=3Retrieves only a specific number of results from a search, and chooses where that number should begin. When doing a broad search that takes too long or returns too much data at once, these options can be used to get the same data in smaller pieces. The default, and maximum, limit is 10 studies. Links are provided in the response JSON to the next and previous page of results.
recentNOAA
Changes the order of search results so that recently contributed studies are first. Only studies contributed in the past two years can be found with this option. Because this parameter is designed to be used with very broad searches, by default only the 25 most recent studies are returned. This can be changed with the
recent=true.../study/search.json?headersOnly=true&recent=true.../study/search.json?headersOnly=true&recent=true&limit=50Changes the order of search results so that recently contributed studies are first. Only studies contributed in the past two years can be found with this option. Because this parameter is designed to be used with very broad searches, by default only the 25 most recent studies are returned. This can be changed with the
limit parameter.PaST Thesaurus Terms
PaST Thesaurus (CV) terms are programmatically accessible through the cvterms api, by either name or id.
termName
Additionally, a SKOS RDF document exists for the vocabulary.termName=([a-zA-Z>]+|)*[a-zA-Z>]+.../cvterm.json?termName=unit%3Etemperature%20unit%3Edegree%20Celsiushttps://www.ncei.noaa.gov/access/paleo-search/skos/past-thesaurus.rdfData Downloading
We now offer the ability to package and download data that meets specified criteria via our web service. Note that we do not currently distribute third-party data, such as from Neotoma or PANGAEA, but will include links to it in a manifest file when possible. Three new api endpoints are used to support this feature:
.../data/search.json?[search params]
The data search endpoint accepts the same parameters as the study search endpoint, with the exception of
.../data/status.json?[job id]
Check the status of your data job by polling the
.../data/cancel.json?[job id]
If you want to cancel a job you can do so with the
Several parameters apply only to data download requests:
.../data/search.json?[search params]
.../data/search.json?xmlId=1840The data search endpoint accepts the same parameters as the study search endpoint, with the exception of
headersOnly, skip/limit, and recent. Data file packaging operates asynchronously; in response to a /data/search call you will receive a JSON token with confirmation of your request, the parameters being searched, and a job id.{jobId: ”2043180573_2017-05-15”, message: ”request received”, searchParams: [”xmlId: 1000099”]}.../data/status.json?[job id]
.../data/status.json?jobId=2043180573_2017-05-15Check the status of your data job by polling the
/data/status endpoint. The JSON token returned will always have a status field, and will contain other fields depending on the state of the job. If your job is finished, the response will include a link to the data download.{status: ”working”, message: ”packaging data”, percent: ”28”}{status: ”complete”, archive: ”https://www.ncei.noaa.gov/access/paleo-data/1099765442_2017-05-15.zip”}.../data/cancel.json?[job id]
.../data/cancel.json?jobId=2043180573_2017-05-15If you want to cancel a job you can do so with the
/data/cancel endpoint.{status: ”canceled”, message: ”job canceled”}Several parameters apply only to data download requests:
previewNOAA
Returns the list of files that will be included in a download request for the given parameters. Be sure to note the 'download' field as some studies may be excluded from downloading due to issues such as third-party distribution constraints.
preview=true.../data/search.json?preview=true&dataPublisher=NOAA&investigators=Ababneh,%20L.Returns the list of files that will be included in a download request for the given parameters. Be sure to note the 'download' field as some studies may be excluded from downloading due to issues such as third-party distribution constraints.
matchLevelNOAA
Specifies how other parameters should be applied when searching for files to download:
matchLevel=[study|file].../data/search.json?preview=true&matchLevel=file&dataPublisher=NOAA&investigators=Ababneh,%20L.&keywords=earth%20science>paleoclimate>tree-ring>width>more%20info.../data/search.json?preview=true&matchLevel=study&dataPublisher=NOAA&investigators=Ababneh,%20L.&keywords=earth%20science>paleoclimate>tree-ring>width>more%20infoSpecifies how other parameters should be applied when searching for files to download:
- File Match: Only data files which match your search parameters will be included, regardless of matches on other data files in the same study. This is the narrower search option.
- Study Match: All data files will be included from each selected study where any data file matches your search parameters. This is the broader search option.
metadataOnlyNOAA
Downloads only the metadata files for the matching studies. No data files are included.
metadataOnly=true.../data/search.json?metadataOnly=true&dataPublisher=NOAA&investigators=Ababneh,%20L.Downloads only the metadata files for the matching studies. No data files are included.
Page parameters
Several pages can take query parameters to change their default behavior:
The main search page accepts
The study page (https://www.ncei.noaa.gov/access/paleo-search/study/[NOAAStudyId]) takes the NOAA Study Id for a single study as a URL parameter. If the study contains more than one site, a specific site can be highlighted by providing a
All reports pages can accept one or more
The CV Terms thesaurus page can take a term name or id, to link directly to a specific term.
The main search page accepts
dataTypeId to specify the default data type to search. The various shuttle boxes will be populated with only values applicable to that data type. This is the same as selecting that data type from the shuttle box on the page. If no dataTypeIdis specified, the default is 'all'.The study page (https://www.ncei.noaa.gov/access/paleo-search/study/[NOAAStudyId]) takes the NOAA Study Id for a single study as a URL parameter. If the study contains more than one site, a specific site can be highlighted by providing a
siteId.All reports pages can accept one or more
dataTypeId parameters. These pages can also be called with a search param which tells the page to search for the supplied data types as soon as it loads.The CV Terms thesaurus page can take a term name or id, to link directly to a specific term.