Parallel Runners¶

The parallel_runners module contains the core analysis pipelines used to identify and compare neuronal ensembles.

These functions can be executed exclusively from Python scripts.

Each parallel runner implements a complete analysis workflow for a set of databases, including:

Input validation
Algorithm selection
Execution
Result formatting

Module Reference¶

encore.parallel_runners.sessions.run_parallel_sessions(data: dict[str, dict[str, ndarray]], parameters: dict[str, dict], max_workers_cant: int, default_key='default')¶

Run ensembles identification algorithms in parallel for different databases. The parameters for the analysis can be defined for each database or use a default one as a fallback.

Parameters:

data (dict[str, dict[str, np.ndarray]]) – Dictionary containing the data for each database. The keys for the first dictionary is a string with the name or ID or the database. Each value at this level contains a dictionary with the data of that database. In this dictionary each key is a specific identifier for a numpy array containing experimental data, e.g. matrix of spikes, fluorescence traces, stimulation, behavior. You can use any name for these variables.
parameters (dict[str, dict]) – Dictionary where each key is the ID of a database or a fallback name. The dictionary inside should contain the keys: - “analysis”, with the 3-letter name of the algorithm to use. - “data_names”, This is a dict[str, str] The keys should be one the name of the variables used by the algorithm, e.g “data_neuronal_activity”, “data_dFFo” and the value is the name of that variable in the data parameter. - “parameters” with another dictionary with the parameters for that specific algorithm, as used by the API. - “evaluate_similarity”, a bool variable to evaluate similarity metrics between results/data - “similarity_elements”, a list of strings, each string is the key of variables in data. When evaluate_similarity is True, the batch processing will also calculate the similarity between the ensembles timecourse and this other variables, so these variables should be of the same length as the recording. This is useful to evaluate the performance of each algorithm in every database.
max_workers_cant (int) – Number of workers to use for parallel processing.
default_key (str, optional) – Specific key of the fallback parameter key. These set of parameters will be used for any database that is not explicitly in the keys of the parameters argument. Defaults to “default”.

Returns:

Dictionary with the results of the batch analyses. Contains the following keys. info: dict[str, str], the date and version information of the analyzer parameters: dict, The parameters used for the entire batch analysis. This is a copy of the parameters passed by the user to the function. parameters_used: dict, each key is the name of each database, contains the batch parameters used by this database. If it used the default parameters then it’s a copy of its contents. results: dict, each key is the name of each database. Contains the minimal results of the ensembles algorithm The matrix of similarity and the labels of each similarity matrix and a flag of success for the algorithm.

Return type:

dict[str, dict]

Raises:

RuntimeError – If no fallback parameter is assigned and a database was not in the parameters keys.

encore.parallel_runners.sessions.run_single_session(args: tuple[str, str, dict[str, ndarray], dict[str, str], dict, bool, list[str]]) → tuple[str, dict, dict]¶

Auxiliary function to identify the algorithm to run, organize the user’s data and organize the result for this run. This is an internal function expected to be used only by encore.parallel_runners.sessions.run_parallel_sessions() Every worker uses this function.

Parameters:: args (tuple[str, str, dict[str, np.ndarray], dict[str, str], dict, bool, list[str]]) – Tuple with the arguments for the runner. This is: algorithm: str, The 3-chars name of the algorithm, e.g. ‘ica’, ‘svd’, ‘pca’, ‘sgc’, ‘x2p’ database_name: str, The name of the database used. This is to reference asynchronously the results with the correct database. database_data: dict[str, np.ndarray], The data for the analysis. data_names: dict[str, str], The names of the API variables. parameters: dict, The dictionary with the parameters for the algorithm. eval_similarity: bool, whether or not to perform the evaluation of similarities. similarity_elements: list[str], List with the names of the variables to compute similarities with.
Raises:: RuntimeError – If the selected algorithm is not known.
Returns:: Tuple with the results of the run. This is: database_name: str, the name of the database as it was received parameters: dict, the parameters for the algorithm result: dict, contains the fields “success”: bool, “results”: dict, “similarity”: dict, “similarity_labels”: list[str],
Return type:: tuple[str, dict, dict]