generative_ai.information_retrieval.utils_retrieval module

Define functionalities for type annotations in information retrieval step.

class TransformerType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

Define supported transformer types.

STANDARD_TRANSFORMERS = 'standard_transformers'

QUANTISED_CTRANSFORMERS = 'quantised_ctransformers'

class PipelineType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

Define supported pipeline types.

TEXT_GENERATION = 'text-generation'

TEXT2TEXT_GENERATION = 'text2text-generation'

class QuantisedModel(*, language_model_type: Literal[TransformerType.QUANTISED_CTRANSFORMERS], quantised_model_name: str, quantised_model_file: str, quantised_model_type: str)

Bases: BaseModel

Store details of a ctransformers library compatible Hugging Face model.

language_model_type

kind of language model

Type:

typing.Literal[TransformerType.QUANTISED_CTRANSFORMERS]

quantised_model_name

name of the Hugging Face model

Type:

str

quantised_model_file

named of quantised model file

Type:

str

quantised_model_type

type of quantised model

Type:

str

language_model_type: Literal[TransformerType.QUANTISED_CTRANSFORMERS]

quantised_model_name: str

quantised_model_file: str

quantised_model_type: str

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'language_model_type': FieldInfo(annotation=Literal[<TransformerType.QUANTISED_CTRANSFORMERS: 'quantised_ctransformers'>], required=True), 'quantised_model_file': FieldInfo(annotation=str, required=True), 'quantised_model_name': FieldInfo(annotation=str, required=True), 'quantised_model_type': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class RetrievalType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

Define supported retrieval types.

MMR = 'mmr'

SIMILARITY = 'similarity'

class StandardModel(*, language_model_type: Literal[TransformerType.STANDARD_TRANSFORMERS], standard_pipeline_type: PipelineType, standard_model_name: str)

Bases: BaseModel

Store details of a transformers library compatible Hugging Face model.

language_model_type

kind of language model

Type:

typing.Literal[TransformerType.STANDARD_TRANSFORMERS]

standard_pipeline_type

kind of Hugging Face pipeline

Type:

PipelineType

standard_model_name

name of the Hugging Face model

Type:

str

language_model_type: Literal[TransformerType.STANDARD_TRANSFORMERS]

standard_pipeline_type: PipelineType

standard_model_name: str

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'language_model_type': FieldInfo(annotation=Literal[<TransformerType.STANDARD_TRANSFORMERS: 'standard_transformers'>], required=True), 'standard_model_name': FieldInfo(annotation=str, required=True), 'standard_pipeline_type': FieldInfo(annotation=PipelineType, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.