Azure AI Search Push API

By Ashwin Venugopal -

 




Index Any Data Using Azure AI Search Push API

The most adaptable method for adding data to an Azure AI Search index is the REST API. Any programming language or interactive application that can send JSON requests to an endpoint can be used.

Supported REST API Operations

There are two supported REST APIs provided by AI Search- Search and management APIs.

How To Call Search REST API ?

If you want to call any of the search APIs you need to:

  • Use the HTTPS endpoint provided by your search service, you must include an api-version in the URI.

  • The request header must include an api-key attribute.

To find the endpoint, api-version, and api-key go to the Azure portal. In the portal, navigate to your search service, then select Search explorer. Choose Keys to locate the api-key on the left. If you're doing more than just accessing the index with the REST API, you can use either the primary or secondary admin key. You can make and use query keys if you only need to search an index. An admin key is required in order to add, modify, or remove data from an index.

Add Data To An Index

Use an HTTP POST request using the indexes feature. The body of your request needs to let the REST endpoint know the action to take on the document, which document to apply the action too, and what data to use. If your request is successful, the API will return a 200 status code. You can add as many documents in the value array as you want. However, for optimal performance consider batching the documents in your requests up to a maximum of 1,000 documents, or 16 MB in total size.

Use .NET Core To Index Any Data

How your index performs is based on six key factors:

  • The search service tier and how many replicas and partitions you've enabled.

  • The complexity of the index schema. Reduce how many properties (searchable, facetable, sortable) each field has.

  • The number of documents in each batch, the best size will depend on the index schema and the size of documents.

  • How multithreaded your approach is.

  • Handling errors and throttling. Use an exponential backoff retry strategy.

  • Where your data resides, try to index your data as close to your search index. For example, run uploads from inside the Azure environment.

Conclusion

We have successfully learnt about indexing any data using Azure AI Search push API.

 















Comments

Post a Comment

Popular posts from this blog

Connect Data to Azure Sentinel Using Data Connectors

By Ashwin Venugopal -
Image
  Ingest Log Data with Data Connectors You should connect your data sources with Azure Sentinel Connectors to collect the log data whose page displays a growing list of the connectors provided by Azure Sentinel, after selecting the Open connector page (the detailed connector page has a left and right blade).  The information about the connector, the connector's status, and the last time a log was received if connected is provided by the left blade. Whereas the right tab has two tabs- Instructions and Next Steps. The instructions tab can be different based on the connector having Prerequisites and Configuration which can be followed to connect to the data source. The Next Steps tab offers a quick reference to workbooks, query samples, and analytical templates. Data connectors can only be disconnected/deactivated, not deleted. Note-  The connector does not install Workbooks and Analytical Templates as they are already available in the Sentinel environment for out of the box...
Read more

Information Protection Scanner: Resolve Issues with Information Protection Scanner Deployment

By Ashwin Venugopal -
Image
  About If you are experiencing problems with the Microsoft Purview Information Protection scanner, check the health of your deployment by utilizing the Start-ScannerDiagnosticsPowerShell cmdlet to start the scanner diagnostic tool. The diagnostics tool checks the following details and then creates a log files with the results: Whether the database is up to date. Whether network URLs are accessible. Whether there's a valid authentication token and policy can be acquired. Whether the profile is defined in the Azure portal. Whether offline/online configuration exists and can be acquired. Whether the rules configured are valid.  The Start-ScannerDiagnostics command does not perform a comprehensive check of prerequisites. If you encounter problems with the scanner, you need to verify that your system meets the scanner's requirements, and that your scanner's configuration and installation are finished.  Verify Scanning Details Per Scanner Node and Repository Run the Get-ScanSt...
Read more

Azure AI Search plugin in Microsoft Security Copilot (Preview)

By Ashwin Venugopal -
Image
  Introduction Azure AI Search is a service that allows to reliably extract content search, extract, and understand at scale. It is typically used to search for documents, data surveys, and chat-style Copilot apps over proprietary data. Indexing and querying are Azure AI Search's two main workloads. While querying allows to search the content from index, indexing loads the content into the search engine and makes it searchable. By connecting to one of the Azure AI Search indexes from within Copilot, the Microsoft Security Copilot integration for Azure enables to use the content in Copilot to produce contextualized, pertinent, and targeted responses.  Requirement Before using the Azure AI Search plugin in Copilot, the desired Azure AI Search index to connect to Copilot is configured as follows: The text field must be searchable.  The title field must be filterable. The vector-field must use text-embedding-ada-002. Integrated vectorization can be used to setup an index in t...
Read more