Semantic search inside invenio

[mesemus]

Colleagues at Masaryk University in Brno propose to begin work on integrating semantic search capabilities into InvenioRDM.
From the Invenio perspective, this would involve:

1. Modifying the mapping to include a new vector field that would store embeddings for the title, abstract, and other fields.
2. Introducing a new SemanticSearchDumperExt to convert selected fields into vector embeddings.
3. Adding a ParamInterpreterCls that would accept a query parameter (e.g., sq=) and use it to perform searches against the vector field.

We also propose updating the RDM UI to include a switch that controls how the search box behaves—routing queries either through the standard search (?q=...) or through semantic search (?sq=...).

The feature would be opt-in initially.

I’ve invited Terezia Slaninakova (@TerkaSlan) and Jakub Cillik (@xcillik) to this discussion; please feel free to direct any embedding-related questions to them.

[zzacharo]

Thinking out loud, would maybe fulltext search be a valid case to consider in this development effort? ping @egabancho