pycurator.collectors.base.BaseTermCollector#

class pycurator.collectors.base.BaseTermCollector(repository_name: str, search_terms: Optional[collections.abc.Collection[pycurator._typing.SearchTerm]] = None, credentials: Optional[str] = None)[source]#

Base for API collection classes that utilize search terms.

This base inherits from BaseAPICollector, which provides credential info, as well as general parameters for tracking collection progress, inherited from BaseCollector.

Parameters

  • repository_name (str) – Name of the repository being collected from. Used for providing updates to user, loading credentials, and saving output results.

  • search_terms (list-like, optional (default=None)) – Terms to search over. Can be (re)set via set_search_terms() or passed in directly to search functions to override set parameter.

  • credentials (str, optional (default=None)) – JSON filepath containing credentials in form {repository_name}: {key}.

search_terms#
Type

list of str

See also

pycurator.collectors.term_collectors

__init__(repository_name: str, search_terms: Optional[collections.abc.Collection[pycurator._typing.SearchTerm]] = None, credentials: Optional[str] = None) → None[source]#

Methods

__init__(repository_name[, search_terms, ...])

get_all_metadata(search_dict)

Abstract placeholder method for returning metadata.

get_all_search_outputs(**kwargs)

Queries the API for each search term.

get_individual_search_output(search_term)

Abstract placeholder method for search output.

get_query_metadata(object_paths)

Placeholder method for query metadata retrieval.

get_request_output(url[, params, headers])

Return Response and JSON from requests.get().

get_request_output_and_update_query_ref(url)

Return request output and update base.current_query_ref.

load_credentials(credential_filepath)

Load the credential file from the given filepath.

merge_search_and_metadata_dicts(search_dict, ...)

Merges together search and metadata DataFrames by 'on' key.

request_execution()

Raise flag to stop output.

run(**kwargs)

Queries all data from the implemented API.

terminate()

Handle program execution.

track_determinate_progress(coll)

Generator for iterating data and updating progress bar.

track_indeterminate_progress(...)

Progress bar wrapper for indeterminate-length queries.

validate_search_term(func)

Decorator for validating search term object type.

Attributes

search_terms

Property method for search terms.