Azure AI Search Solution: Understand Search Components

By Ashwin Venugopal -

 






Introduction

Each of the several parts that make up an AI search solution is crucial to the extraction, enrichment, indexing, and querying of data.

Data Source

The majority of search solutions begin with a data source that has the information you are looking for. Azure AI Search is compatible with a variety of data sources, including:

  1. Unstructured files in Azure blob storage containers.
  2. Tables in Azure SQL Database.
  3. Documents in Cosmos DB.

Azure AI Search can pull data from these data sources for indexing. Alternatively, applications can push JSON data directly into an index, without pulling it from an existing data store.

Skillset

You may index the data taken from the data source in a simple search solution. The data source determines the information that can be extracted. For example, while indexing data in a database, the fields in the database tables may be extracted. Similarly, when indexing a collection of documents, the text content of the document many be extracted along with file metadata such file name, modified date, size, and author.

Richer insights into the data are required due to the demands of current application users, even though a simple search solution that indexes data values taken straight from the data source can be helpful.

Artificial intelligence (AI) techniques can be used in Azure AI Search as part of the indexing process to add new information to the source data that can be mapped to index fields. An indexer's abilities are contained in a skillset that outlines an enrichment pipeline, where each stage adds insights from a particular AI talent to the raw data.

Examples of the kind of information that can be extracted by an AI skill include:

  1. The language in which a document is written.
  2. Key phrases that might help determine the main themes or topics discussed in a document.
  3. A sentiment score that quantifies how positive or negative a document is.
  4. Specific locations, people, organizations, or landmarks mentioned in the content.
  5. AI-generated descriptions of images, or image text extracted by optical character recognition.
  6. Custom skills that you develop to meet specific requirements.

Indexer

The indexing process as a whole is powered by the indexer. It translates the data and metadata values taken from the original data source, as well as the outputs extracted using the talents in the skillset, to fields in the index.

When an indexer is constructed, it runs automatically. To add new documents to the index, it can be scheduled to operate on demand or at regular intervals. You may need to reset the index before running the indexer again in some situations, such as when you add new fields to an index or new talents to a skillset.

Index

The searchable outcome of the indexing procedure is the index. It is made up of a set of JSON documents with fields that hold the values that were taken out during indexing. Information can be retrieved, filtered, and sorted by client applications using the index.

Each index field can be configured with the following attributes:

  • Key- Fields that define a unique key for index records.

  • Searchable- Fields that can be queried using full-text search.

  • Filterable- Fields that can be included in filter expressions to return only documents that match specified constraints.

  • Sortable- Fields that can be used to order the results.

  • Facetable- Fields that can be used to determine values for facets (user interface elements used to filter the results based on a list of known field values).

  • Retrievable- Fields that can be included in search results (by default, all fields are retrievable unless this attribute is explicitly removed).

Conclusion

We have successfully understood the search components.


Comments

Post a Comment

Popular posts from this blog

Connect Data to Azure Sentinel Using Data Connectors

By Ashwin Venugopal -
Image
  Ingest Log Data with Data Connectors You should connect your data sources with Azure Sentinel Connectors to collect the log data whose page displays a growing list of the connectors provided by Azure Sentinel, after selecting the Open connector page (the detailed connector page has a left and right blade).  The information about the connector, the connector's status, and the last time a log was received if connected is provided by the left blade. Whereas the right tab has two tabs- Instructions and Next Steps. The instructions tab can be different based on the connector having Prerequisites and Configuration which can be followed to connect to the data source. The Next Steps tab offers a quick reference to workbooks, query samples, and analytical templates. Data connectors can only be disconnected/deactivated, not deleted. Note-  The connector does not install Workbooks and Analytical Templates as they are already available in the Sentinel environment for out of the box...
Read more

Azure AI Search plugin in Microsoft Security Copilot (Preview)

By Ashwin Venugopal -
Image
  Introduction Azure AI Search is a service that allows to reliably extract content search, extract, and understand at scale. It is typically used to search for documents, data surveys, and chat-style Copilot apps over proprietary data. Indexing and querying are Azure AI Search's two main workloads. While querying allows to search the content from index, indexing loads the content into the search engine and makes it searchable. By connecting to one of the Azure AI Search indexes from within Copilot, the Microsoft Security Copilot integration for Azure enables to use the content in Copilot to produce contextualized, pertinent, and targeted responses.  Requirement Before using the Azure AI Search plugin in Copilot, the desired Azure AI Search index to connect to Copilot is configured as follows: The text field must be searchable.  The title field must be filterable. The vector-field must use text-embedding-ada-002. Integrated vectorization can be used to setup an index in t...
Read more

Information Protection Scanner: Resolve Issues with Information Protection Scanner Deployment

By Ashwin Venugopal -
Image
  About If you are experiencing problems with the Microsoft Purview Information Protection scanner, check the health of your deployment by utilizing the Start-ScannerDiagnosticsPowerShell cmdlet to start the scanner diagnostic tool. The diagnostics tool checks the following details and then creates a log files with the results: Whether the database is up to date. Whether network URLs are accessible. Whether there's a valid authentication token and policy can be acquired. Whether the profile is defined in the Azure portal. Whether offline/online configuration exists and can be acquired. Whether the rules configured are valid.  The Start-ScannerDiagnostics command does not perform a comprehensive check of prerequisites. If you encounter problems with the scanner, you need to verify that your system meets the scanner's requirements, and that your scanner's configuration and installation are finished.  Verify Scanning Details Per Scanner Node and Repository Run the Get-ScanSt...
Read more