pycurator.collectors.base.BaseAPICollector
pycurator.collectors.base.BaseAPICollector#
- class pycurator.collectors.base.BaseAPICollector(repository_name: str, credentials: Optional[str] = None)[source]#
Base for collection classes utilizing external API.
This base inherits from BaseCollector, which provides general parameters for tracking collection progress.
- Parameters
repository_name (str) – Name of the repository being collected from. Used for providing updates to user, loading credentials, and saving output results.
credentials (str, optional (default=None)) – JSON filepath containing credentials in form {repository_name}: {key}.
- credentials#
Refer to credentials parameter.
- Type
str
Methods
__init__(repository_name[, credentials])get_all_metadata(search_dict)Abstract placeholder method for returning metadata.
get_all_search_outputs(**kwargs)Abstract placeholder method for returning search outputs.
get_request_output(url[, params, headers])Return Response and JSON from requests.get().
get_request_output_and_update_query_ref(url)Return request output and update base.current_query_ref.
load_credentials(credential_filepath)Load the credential file from the given filepath.
merge_search_and_metadata_dicts(search_dict, ...)Merges together search and metadata DataFrames by 'on' key.
request_execution()Raise flag to stop output.
run(**kwargs)Queries all data from the implemented API.
terminate()Handle program execution.
track_determinate_progress(coll)Generator for iterating data and updating progress bar.
track_indeterminate_progress(...)Progress bar wrapper for indeterminate-length queries.