py4ai.data.layer.fs.criteria module

Module containing implementations and abstractions for query to be used in FileSystem persistence layers.

class py4ai.data.layer.fs.criteria.FileSystemCriteriaFactory(path: Path, index_file: Path = PosixPath('indices.json'))

Bases: Generic[KE, E]

Base class to be used for extending CriteriaFactory for FileSystems.

Instantiate the class.

Parameters

  • path – path where to store the objects

  • index_file – file where to store the indexing of the files, to be used to querying and fast-retrieving.

all() → FileSystemSearchCriteria[KE]

Return empty query.

Returns

empty query.

filter_path_by_condition(condition: Callable[[Dict[Any, Any]], bool]) → FileSystemSearchCriteria[KE]

Return query with filtered elements based on a given condition.

Parameters

condition – filtering function to be applied at indexed fields.

Returns

resulting query

static format_name(name: Path, path: Path) → str

Reformat full path to transform it to a file-system key.

Parameters

  • name – full path.

  • path – base path to be used to strip the full path.

Returns

file-system key.

get_index_fields(entity: E) → Dict[str, Any]

Extract the indexed field from an entity.

Parameters

entity – entity

Returns

indices fields

property index: Dict[KE, Dict[Any, Any]]

Return a hash-property dictionary, to be used to filter objects and retrieve the corresponding hash.

Returns

dictionary with entity hash, properties as (key, value) pairs.

sort_by(criteria: FileSystemSearchCriteria[KE], sorting_option: Tuple[str, SortingDirection]) → FileSystemSearchCriteria[KE]

Create a sorted query, based on a sorting option.

Parameters

  • criteria – file-system query.

  • sorting_option – sorting options to be used.

Returns

sorted query.

update_index(key: KE, entity: E) → None

Update indices file in the file-system, with the entity key and the corresponding indexed fields.

Parameters

  • key – entity key

  • entity – entity whose indices are to be computed

class py4ai.data.layer.fs.criteria.FileSystemSearchCriteria(iterable: Iterable[KE])

Bases: SearchCriteria[List[KE]]

Base class for representing a FileSystem query.

Instantiate class based on list of filenames.

Parameters

iterable – list of filenames

property query: List[KE]

Return the underlying query based on a PandasFilter condition.

Returns

underlying query