Blogs by Ashwin

  Optimize Your Index Size and Schema The effectiveness of your search queries is closely linked to the size and intricacy of your indexes. The quicker Azure AI Search can answer queries, the more compact and streamlined your indexes are. If you don't pay attention, indexes can grow over time. You should review that all the documents in your index are still relevant and need to be searchable. Consider reviewing all the attributes you've enabled on each field. For example, adding support for filters, facets, and sorting can quadruple the storage needed to support your index. Improve Performance Of Your Queries If you're familiar with the workings of the search service, you can optimize your queries to significantly enhance performance. Use this checklist for writing better queries: Only specify the fields you need to search using the searchFields parameter. As more fields require extra processing. Return the smallest number of fields you need to render on your search results...

  Optimize Performance of an Azure AI Search Solution The size and complexity of your indexes may affect search solutions’ performance. Additionally, You also need to know how to write efficient queries to search it and choose the right service tier. Measure Your Current Search Performance If you don't know how well your search service works, you can't optimize. Establish a baseline performance benchmark so you can verify the advancements you achieve and monitor any decline in performance over time. To start with, enable diagnostic logging using Log Analytics: In the Azure portal, select Diagnostic settings. Give your diagnostic setting a name. Select allLogs and AllMetrics. Select Send to Log Analytics workspace. Choose, or create, your Log Analytics workspace. This diagnostic data must be recorded at the search service level. Your end users or apps may experience performance problems in a number of locations. If you can demonstrate that your search service is operating effici...

  Authenticate Requests To Your Search Solution Your attention may go to how to verify search requests from your users and apps once you have the infrastructure in place to lessen the attack surface of your search solution. Key-based authentication is the default setting when you set up your ACS. There are two different kinds of keys: Admin Keys - grant write permissions and the right to query system information (maximum of 2 admin keys can be created per search service) Query Keys - grant read permissions and are used by your users or apps to query indexes (maximum of 50 query keys can be created per search service) Role-based access control (RBAC) is provided by the Azure platform as a global system to control access to resources. You can use RBAC in Azure AI Search in the following ways: Roles can be granted access to administer the service Define roles with access to create, load, and query indexes The built-in roles you can assign to manage the Azure AI Search serv...

  Introduction Understanding how to manage Azure AI Search's two core workloads, indexing and querying, is essential to running a successful solution. The search solution must be as economical as feasible. Manage Security Of an Azure AI Search Solution Businesses must be able to rely on their search solutions' security. You have control over the security of the data you search for with Azure AI Search. Overview Of Security Approaches AI Search security builds on Azure's existing network security features. When you think about securing your search solution, you can focus on three areas: Inbound search requests made by users to your search solution Outbound requests from your search solution to other servers to index documents Restricting access at the document level per user search request Data Encryption Like all Azure services, the Azure AI Search service uses service controlled keys to encrypt the data it stores while it's at rest. Indexes, data sources, synonym maps,...

  Index Any Data Using Azure AI Search Push API The most adaptable method for adding data to an Azure AI Search index is the REST API. Any programming language or interactive application that can send JSON requests to an endpoint can be used. Supported REST API Operations There are two supported REST APIs provided by AI Search- Search and management APIs. How To Call Search REST API ? If you want to call any of the search APIs you need to: Use the HTTPS endpoint provided by your search service, you must include an api-version in the URI. The request header must include an api-key attribute. To find the endpoint, api-version, and api-key go to the Azure portal. In the portal, navigate to your search service, then select Search explorer. Choose Keys to locate the api-key on the left. If you're doing more than just accessing the index with the REST API, you can use either the primary or secondary admin key. You can make and use query keys if you only need to search an index. An a...

  Index Data From External Data Sources Using Azure Data Factory An organization's search solution frequently has to include external data that isn't stored in Azure. Because it offers numerous methods for creating and pushing data into indexes, Azure AI Search is adaptable. Push Data Into Search Index Using Azure Data Factory (ADF) One method is to use ADF to push data into an index using a zero-code option. Almost 100 distinct data stores are connected to ADF. You can link an infinite number of data stores using connectors like HTTP and REST. These data stores are utilized in pipelines as either a source or a target (referred to as sinks). In a copy activity, the Azure AI Search index connector can be utilized as a sink. Create ADF Pipeline To Push Data Into Search Index The steps you need to take to use and ADF pipeline to push data into a search index are: Create an Azure AI Search index with all the fields you want to store data in. Create a pipeline with a copy data step....

  How To Use Custom Azure Machine Learning Skillset ? You can create robust AI models and host them on Azure using Azure Machine Learning. The endpoints of these AI models can be utilized in search strategies to improve indexes. Adding a machine learning custom skill to a search index functions in the same way as adding any other custom talent. Custom Azure Machine Learning Skill Schema An Azure Machine Learning (AML) custom skill enriches a search index at the document level. Your document indexer must have an AmlSkill in its skill set. Note that batchSize options are not included in the custom skill. One document at a time will be processed by the AML model. The remaining parameters, timeout and degree of parallelism, regulate how the skill is performed. The timeout value in the aforementioned schema is set at 30 seconds. One should be the starting point for the degree of parallelism. You may be able to raise this figure, depending on your infrastructure. Scaling up the Kubernet...

  Improve Search Experience By Ordering Results By Distance From a Given Reference Point Users frequently wish to look for products connected to a specific area. For instance, they may want to locate the coffee shop closest to their location. AI Search has geospatial features that you can use in queries to compare locations on the surface of the Earth. What are Geo-Spatial Functions? The hotel's location is a crucial factor to take into account when making a reservation. For instance, you will want a hotel close to the Eiffel Tower if you're planning a trip to view it. To ask AI Search to return results based on their location information, you can use two functions in your query: geo.distance - This function returns the distance in a straight line across the Earth's surface from the point you specify to the location of the search result. geo.intersects - This function returns true if the location of a search result is inside a polygon that you specify. Make sure the locat...

  Create a Custom Analyzer When you define the index, you provide a custom analyzer. The Azure site does not allow you to define a custom index, instead, you must use JSON code. During the design phase, make use of the index's analyzers section. One tokenizer, one or more character filters, and one or more token filters may be included. Use a unique name for your analyzer and set the @odata.type property to Microsoft.Azure.Search.CustomAnalyzer. Test Custom Analyzer You can utilize the REST API's Analyze Text function after defining your custom analyzer as part of your index. To submit test text and make sure the analyzer returns tokens appropriately, use the text function. Create these queries using any REST testing tool, like the well-known Postman program. Be sure to test with lots of different text values until you're sure that the custom analyzer behaves as you expect. Use a Custom Analyzer For a Field You can set up your index to utilize a custom analyzer once you...

  What are tokenizers ? The part that separates the text into the tokens that will be kept in the index is called the tokenizer. Additionally, tokenizers deconstruct words into their most basic forms. Often, a token is a single word, but you might want to create unusual tokens such as: A full postal address. A complete URL or email address. Words based on the grammar of a specific language. There are 13 different tokenizers to choose from. These tokenizers include: classic- This tokenizer processes text based on grammar for European languages. keyword- This tokenizer emits the entire input as a single token. Use this tokenizer for fields that should always be indexed as one value. lowercase- This tokenizer divides text at non-letters and then modifies the resulting tokens to all lower case. microsoft_language_tokenizer - This tokenizer divides text based on the grammar of the language you specify. pattern- This tokenizer divides texts where it matches a regular expression that y...

  Improve an Index With Analyzers and Tokenized Terms By default, Azure AI Search is set up to examine text and find useful tokens for your index. The appropriate tokens guaranty that users may swiftly locate the documents they require. An ideal index is often produced by the default configuration. However, you might wish to specify exactly how text is processed if you have uncommon or unique fields. Analyzers in AI Search Text is retrieved by AI Search when it indexes your material. That text must be processed in order to create a helpful index with terms that aid users in finding documents. For example: The text should be broken into words, often by using whitespace and punctuation characters as delimiters. Stopwords, such as "the" and "it", should be removed because users don't search for them. Words should be reduced to their root form. For example, past tense words, such as "ran", should be replaced with present tense words, such as "run...

  How Search Scores Are Calculated ? The final stage of processing a search query is scoring. The documents that are retrieved from the first three stages are ranked by the search engine. The number of times detected search phrases exist in a document, the size of the document, and the rarity of each keyword all affect the score. The search results are automatically sorted by search score, with the highest score appearing first. An $orderby clause can be included to break a tie between two documents that have the same search score. Improve Score For More Relevant Documents The final score may not be the highest for the most relevant document because the default scoring is based on phrase frequency and rarity. Because every dataset is unique, AI Search allows you to use scoring profiles to affect a document's score. The most straightforward scoring profile defines different weights for fields in an index. Functions like freshness and distance can also be included in the score profil...

  Introduction Azure AI Search is a robust search engine that can index a large amount of data from several sources. Returning pertinent results from search queries is a fundamental aspect of search. Improve Ranking Of a Document With Term Boosting When the most pertinent results are displayed first, search performs best. Every search engine aims to provide the most pertinent answers to inquiries. For full text search, Azure AI Search uses an improved version of Apache Lucene. Search an Index With Azure AI Search, you may use the search explorer tool within the Azure portal or a REST endpoint to query an index. You will use the search explorer to see the difference between using the simple and full query type changes your search results. Improve Relevance Of Results By Adding Scoring Profiles Azure AI Search uses the BM25 similarity ranking algorithm. The algorithm scores documents based on the search terms used. How Search Scores Are Calculated Scoring is the last phase of process...

  Train Your Classification Model It requires recognized data to be trained, just like any other AI model. In order for the model to be tested, it must see examples of how to map data to a class. You have the option to allow the model to automatically divide your training data; by default, it will utilize 20% of the documents for blind testing and 80% of the documents for training. You can designate certain documents for testing if you want to use them to test your model. Choose Data labeling for your project in Language Studio. All of your documents will be visible to you. After choosing each document to include in the testing set, click Testing the model's performance. Create a new training job after saving your amended labels. Create Search Index Creating a search index that will be enhanced by a custom text classification model doesn't require any special actions on your part. Once you have developed a function app, you will be changing the index, indexer, and custom skill....

  Store Your Data Language Studio and Azure AI Services both provide access to Azure Blob storage. The easiest option is to select Container as the container must be available, although private containers can also be used with some extra setup. You also need a method for classifying each document in addition to your data. You can manually categorize each document one at a time using the graphical interface that Language Studio offers. There are two types of projects available, if a document belongs to a single class, utilize a single label categorization project. Use the multi-label categorization project if you are able to map a document to multiple classes. If you don't want to manually classify each document, you can label all your documents before you create your Azure AI Language project. Create Your Azure AI Language Project Your Azure AI Language project can be created in one of two ways. Language Studio will offer to build a language service for you if you begin using it wi...