8.1.1. Data Grabbers#

Provide imports for datagrabber sub-package.

class junifer.datagrabber.BaseDataGrabber(types, datadir)#

Abstract base class for datagrabber.

For every interface that is required, one needs to provide a concrete implementation of this abstract class.

Parameters:

typeslist of str: The types of data to be grabbed.
datadirstr or pathlib.Path: The directory where the data is / will be stored.

Attributes:

datadirpathlib.Path: Get data directory path.

property datadir: Path#

Get data directory path.

Returns:

pathlib.Path: Path to the data directory. Can be overridden by subclasses.

abstract get_element_keys()#

Get element keys.

For each item in the element tuple passed to __getitem__(), this method returns the corresponding key(s).

Returns:

str: The element keys.

abstract get_elements()#

Get elements.

Returns:

list: List of elements that can be grabbed. The elements can be strings, tuples or any object that will be then used as a key to index the datagrabber.

abstract get_item(**element)#

Get the specified item from the dataset.

Parameters:

elementdict: The element to be indexed.

Returns:

dict: Dictionary of paths for each type of data required for the specified element.

get_types()#

Get types.

Returns:

list of str: The types of data to be grabbed.

class junifer.datagrabber.DataladAOMICID1000(datadir=None)#

Concrete implementation for pattern-based data fetching of AOMICID1000.

Parameters:

datadirstr or Path, optional: The directory where the datalad dataset will be cloned. If None, the datalad dataset will be cloned into a temporary directory (default None).

class junifer.datagrabber.DataladAOMICPIOP1(datadir=None, tasks=None)#

Concrete implementation for pattern-based data fetching of AOMICPIOP1.

Parameters:

datadirstr or Path, optional: The directory where the datalad dataset will be cloned. If None, the datalad dataset will be cloned into a temporary directory (default None).
tasks{“restingstate”, “anticipation”, “emomatching”, “faces”, “gstroop”, “workingmemory”} or list of the options, optional: AOMIC PIOP1 task sessions. If None, all available task sessions are selected (default None).

get_elements()#

Implement fetching list of subjects in the dataset.

Returns:

list of str: The list of subjects in the dataset.

get_item(subject, task)#

Index one element in the dataset.

Parameters:

subjectstr: The subject ID.
task{“restingstate”, “anticipation”, “emomatching”, “faces”, “gstroop”, “workingmemory”}: The task to get.

Returns:

outdict: Dictionary of paths for each type of data required for the specified element.

class junifer.datagrabber.DataladAOMICPIOP2(datadir=None, tasks=None)#

Concrete implementation for pattern-based data fetching of AOMICPIOP2.

Parameters:

datadirstr or Path, optional: The directory where the datalad dataset will be cloned. If None, the datalad dataset will be cloned into a temporary directory (default None).
tasks{“restingstate”, “stopsignal”, “emomatching”, “workingmemory”} or list of the options, optional: AOMIC PIOP2 task sessions. If None, all available task sessions are selected (default None).

get_elements()#

Implement fetching list of elements in the dataset.

Returns:

list: The list of elements that can be grabbed in the dataset after imposing constraints based on specified tasks.

class junifer.datagrabber.DataladDataGrabber(rootdir='.', datadir=None, uri=None, **kwargs)#

Abstract base class for data fetching via Datalad.

Defines a Data Grabber that gets data from a datalad sibling.

Parameters:

rootdirstr or Path, optional: The path within the datalad dataset to the root directory (default “.”).
datadirstr or Path, optional: That directory where the datalad dataset will be cloned. If None, the datalad dataset will be cloned into a temporary directory (default None).
uristr, optional: URI of the datalad sibling (default None).
**kwargs: Keyword arguments passed to superclass.

See also

BaseDataGrabber: For the abstract base class of any datagrabber.

Notes

By itself, this class is still abstract as the __getitem__ method relies on the parent class BaseDataGrabber.__getitem__ which is not yet implemented. This class is intended to be used as a superclass of a class with multiple inheritance.

Methods

install:	Installs (clones) the datalad dataset into the datadir. This method is called automatically when the datagrabber is used within a context.
remove:	Removes the datalad dataset from the datadir. This method is called automatically when the datagrabber is used within a context.

cleanup()#

Cleanup the datalad dataset.

property datadir: Path#: Get data directory path.

install()#

Install the datalad dataset into the datadir.

Raises:

ValueError: If the dataset is already installed but with a different ID.

class junifer.datagrabber.DataladHCP1200(datadir=None, tasks=None, phase_encodings=None)#

Concrete implementation for datalad-based data fetching of HCP1200.

Parameters:

datadirstr or Path, optional: The directory where the datalad dataset will be cloned. If None, the datalad dataset will be cloned into a temporary directory (default None).
tasks{“REST1”, “REST2”, “SOCIAL”, “WM”, “RELATIONAL”, “EMOTION”, “LANGUAGE”, “GAMBLING”, “MOTOR”} or list of the options, optional: HCP task sessions. If None, all available task sessions are selected (default None).
phase_encodings{“LR”, “RL”} or list of the options, optional: HCP phase encoding directions. If None, both will be used (default None).

property skip_file_check: bool#: Skip file check existence.

class junifer.datagrabber.HCP1200(datadir, tasks=None, phase_encodings=None, **kwargs)#

Concrete implementation for pattern-based data fetching of HCP1200.

Parameters:

datadirstr or Path, optional: The directory where the datalad dataset will be cloned.
tasks{“REST1”, “REST2”, “SOCIAL”, “WM”, “RELATIONAL”, “EMOTION”, “LANGUAGE”, “GAMBLING”, “MOTOR”} or list of the options, optional: HCP task sessions. If None, all available task sessions are selected (default None).
phase_encodings{“LR”, “RL”} or list of the options, optional: HCP phase encoding directions. If None, both will be used (default None).
**kwargs: Keyword arguments passed to superclass.

get_elements()#

Implement fetching list of elements in the dataset.

Returns:

list: The list of elements in the dataset.

get_item(subject, task, phase_encoding)#

Index one element in the dataset.

Parameters:

subjectstr: The subject ID.
task{“REST1”, “REST2”, “SOCIAL”, “WM”, “RELATIONAL”, “EMOTION”, “LANGUAGE”, “GAMBLING”, “MOTOR”}: The task.
phase_encoding{“LR”, “RL”}: The phase encoding.

Returns:

outdict: Dictionary of paths for each type of data required for the specified element.

class junifer.datagrabber.MultipleDataGrabber(datagrabbers, **kwargs)#

Data Grabber class for data fetching from multiple sources.

Defines a Data Grabber which can be used to fetch data from multiple datagrabbers.

Parameters:

datagrabberslist of datagrabber-like objects: The datagrabbers to use to fetch data using.
**kwargs: Keyword arguments passed to superclass.

get_element_keys()#

Get element keys.

For each item in the element tuple passed to __getitem__(), this method returns the corresponding key(s).

Returns:

list of str: The element keys.

get_elements()#

Get elements.

Returns:

list: The list of elements that can be grabbed in the dataset. It corresponds to the elements that are present in all the related datagrabbers.

get_item(**element)#

Get item.

Parameters:

elementdict: The element to be indexed.

Returns:

dict: Dictionary of paths for each type of data required for the specified element.

Notes

This function is not implemented for this class as it is useless.

get_types()#

Get types.

Returns:

list of list of str: The types of data to be grabbed.

class junifer.datagrabber.PatternDataGrabber(types, patterns, replacements, datadir)#

Concrete implementation for data grabbing using patterns.

Implements a Data Grabber that understands patterns to grab data.

Parameters:

typeslist of str: The types of data to be grabbed (default None).
patternsdict: Patterns for each type of data as a dictionary. The keys are the types and the values are the patterns. Each occurrence of the string {subject} in the pattern will be replaced by the indexed element.
replacementslist of str: Replacements in the patterns for each item in the “element” tuple.
datadirstr or pathlib.Path: The directory where the data is / will be stored.

get_element_keys()#

Get element keys.

For each item in the “element” tuple, this functions returns the corresponding key, that is, the replacements of patterns defined in the constructor.

Returns:

list of str: The element keys.

get_elements()#

Implement fetching list of elements in the dataset.

It will use regex to search for “replacements” in the “patterns” and return the intersection of the results for each type i.e., build a list of elements that have all the required types.

Returns:

list: The list of elements that can be grabbed in the dataset.

get_item(**element)#

Implement single element indexing in the database.

This method constructs a real path to the requested item’s data, by replacing the patterns with actual values passed via **element.

Parameters:

elementdict: The element to be indexed. The keys must be the same as the replacements.

Returns:

dict: Dictionary of dictionaries for each type of data required for the specified element.

property skip_file_check: bool#: Skip file check existence.

class junifer.datagrabber.PatternDataladDataGrabber(types, patterns, **kwargs)#

Base class for pattern-based data fetching via Datalad.

Defines a DataGrabber that gets data from a datalad sibling, interpreting patterns.

Parameters:

typeslist of str: The types of data to be grabbed.
patternsdict, optional: Patterns for each type of data as a dictionary. The keys are the types and the values are the patterns. Each occurrence of the string {subject} in the pattern will be replaced by the indexed element (default None).
**kwargs: Keyword arguments passed to superclass.