Blogs by Ashwin

  Introduction Azure AI Document Intelligence analyzes scanned form content and transforms it into data using Azure AI Services. It is capable of identifying text values in both standard forms and forms specific to your company. Imagine that you are employed by a company that polls political parties and private businesses. Responses are submitted by participants via online PDFs or paper forms. Currently, typing these answers into databases costs you a lot of time and money. To determine whether you can utilize Azure AI Document Intelligence to expedite this process, you should evaluate it. Understand AI Document Intelligence Although Azure AI Document Intelligence is simple to use, you must comprehend its objects, such as models, APIs, and tools, in order to develop a dependable solution. You are evaluating Azure AI Document Intelligence at your polling company to see whether it can improve the efficiency of your data entry process. You must comprehend how your development team wil...

  Set Up Semantic Ranking Semantic ranking is available for all indexes and is automatically enabled for the Azure AI Search service at the service level. It is not possible to enable or disable semantic ranking per index. Configure Semantic Ranking Before you configure semantic ranking, you must have an Azure AI Search service with at least one index. Not every area has access to semantic ranking. Make sure your AI Search service's region supports semantic ranking before setting it up. To choose the semantic ranking plan in the Azure portal, perform the following steps: Open the Azure portal and sign in. Select All resources and select your search service. In the navigation pane, select Settings and select Semantic ranker. Select the appropriate service plan. You can alter the service plan after deployment. You can con figure semantic ranking on a per-ind ex basis. You can have multiple semantic configurations on each index. To configure semantic ranking, follow these steps: From ...

  Introduction The top 50 results from the BM25 ranking are used for semantic ranking. A semantic setup divides the findings into several fields. The fields are cut to 256 distinct tokens after being transformed into text strings. In the document, a token is basically equal to a word. After the strings are ready, machine reading comprehension models are used to identify the words and sentences that best fit the query. This summarization sentence produces a semantic caption and, if desired, a semantic response. The semantic relevance of the caption is now used to rate the semantic captions. After that, the results are given back in decreasing relevancy order. This secondary ranking uses multi-lingual, deep learning models adapted from Microsoft Bing to promote the most semantically relevant results. Semantic Ranking Advantages Semantic ranking has two key advantages over traditional search results: Semantic ranking can rank results to more closely match the semantics of the original...

  Introduction Imagine you are employed by a company creating an app aimed at local tourists. You possess a vast amount of data regarding local attractions, yet when users look for information, they occasionally do not see the most pertinent articles at the forefront of their search results. In order to enhance the user experience of your app, you have to explore methods to provide users with more relevant results through the use of Azure AI Search. What is Semantic Ranking? Semantic ranking is a feature in Azure AI Search designed to enhance the order of search results. It leverages language understanding to more effectively align the context of the user's original query with the ranking of the search results. BM25 Ranking Function Azure AI Search, by default, utilizes the BM25 ranking function. The BM25 ranking function evaluates search results based on how often the search term is found in a document. Typically, the BM25 ranking function produces high-quality rankings, as docume...

  Debug Search Issues Using Azure Portal When you initially set up your search service, you need to establish certain assumptions regarding the data you will be indexing. You decide on the structure of the index and the method for ingesting that data. Yet, you won't be able to confirm whether all your decisions were correct until you execute your built indexer. Explore How To Use Debug Session Tool In Azure AI Search The Debug Session tool is an interactive visual editor that allows you to navigate through the enrichment pipeline of a document during its enrichment process. You can enter each specific skill, apply modifications and corrections, and then execute the indexer in real-time. After addressing any problems, you can refresh and republish the indexer so it can be re-executed to enrich all documents within your index. Once you have assigned a name to your debug session and selected the index for debugging, the search service transfers all necessary components to an Azure Sto...

  Monitor an Azure AI Search Solution Azure Monitor may provide you with information on the performance and usage of your search service. Additionally, you can get notifications to let you know about problems before they happen. Monitor Azure AI Search in Azure Monitor You may view your current search latency, queries per second, and the percentage of throttledqueries as soon as you start up your Azure AI Search service. The Monitoring tab of the Overview page displays this data. You can also check what resources your search solution is using on the Usage tab. You can monitor with additional setup, but this basic information is an excellent place to start.You will be aware that Azure Monitor can be utilized for all of your Azure resources if you are accustomed to maintaining other Azure platform resources. Once you have started using Log Analytics, you get access to performance and diagnostic data in these log tables: AzureActivity - Shows you tasks that have been executed like sca...

  Make Your Search Solution Highly Available One straightforward method to enhance the availability of your search solution is by increasing the quantity of replicas. The sole choice is to maintain more than one within the paid search service tiers. The Azure AI Search service has availability guarantees based on the number of replicas you have: Two replicas guarantee 99.9% availability for your queries Three or more replicas guarantee 99.9% availability for both queries and indexing The alternative method to incorporate redundancy into your search solution is by utilizing Availability Zones. This approach necessitates the use of at least a standard tier. When you create replicas, you have the option to place them in various Availability Zones. The advantage of spreading your replicas this way is that they are situated in separate data centers. Distribute Your Search Solution Globally A single resource group and region is the most economical method to design an Azure AI Search serv...

  Manage Costs Of An Azure AI Search Solution The capacity and features you utilize determine how much an Azure AI Search solution will cost to operate. Estimate Your Search Solutions Baseline Costs An excellent tool for estimating the cost of utilizing any Azure service is the Azure pricing calculator. Make use of it to establish a baseline for your search service requirements. Browse to the Azure AI Search pricing calculator. Choose your region, currency, and hour or monthly pricing. Optimizing Azure AI Search's capacity, including the tier you require, the data you're searching, and the features you utilize, is a crucial part of maintaining an affordable Azure AI Search solution. Understand Billing Model Azure AI Search is invoiced similarly to other Azure resources. You are not charged for the number of search queries, responses, or documents ingested. Tips To Reduce Cost Of Your Search Solution These tips can help you reduce the cost of running your search solution: Minim...

  Optimize Your Index Size and Schema The effectiveness of your search queries is closely linked to the size and intricacy of your indexes. The quicker Azure AI Search can answer queries, the more compact and streamlined your indexes are. If you don't pay attention, indexes can grow over time. You should review that all the documents in your index are still relevant and need to be searchable. Consider reviewing all the attributes you've enabled on each field. For example, adding support for filters, facets, and sorting can quadruple the storage needed to support your index. Improve Performance Of Your Queries If you're familiar with the workings of the search service, you can optimize your queries to significantly enhance performance. Use this checklist for writing better queries: Only specify the fields you need to search using the searchFields parameter. As more fields require extra processing. Return the smallest number of fields you need to render on your search results...

  Optimize Performance of an Azure AI Search Solution The size and complexity of your indexes may affect search solutions’ performance. Additionally, You also need to know how to write efficient queries to search it and choose the right service tier. Measure Your Current Search Performance If you don't know how well your search service works, you can't optimize. Establish a baseline performance benchmark so you can verify the advancements you achieve and monitor any decline in performance over time. To start with, enable diagnostic logging using Log Analytics: In the Azure portal, select Diagnostic settings. Give your diagnostic setting a name. Select allLogs and AllMetrics. Select Send to Log Analytics workspace. Choose, or create, your Log Analytics workspace. This diagnostic data must be recorded at the search service level. Your end users or apps may experience performance problems in a number of locations. If you can demonstrate that your search service is operating effici...

  Authenticate Requests To Your Search Solution Your attention may go to how to verify search requests from your users and apps once you have the infrastructure in place to lessen the attack surface of your search solution. Key-based authentication is the default setting when you set up your ACS. There are two different kinds of keys: Admin Keys - grant write permissions and the right to query system information (maximum of 2 admin keys can be created per search service) Query Keys - grant read permissions and are used by your users or apps to query indexes (maximum of 50 query keys can be created per search service) Role-based access control (RBAC) is provided by the Azure platform as a global system to control access to resources. You can use RBAC in Azure AI Search in the following ways: Roles can be granted access to administer the service Define roles with access to create, load, and query indexes The built-in roles you can assign to manage the Azure AI Search serv...

  Introduction Understanding how to manage Azure AI Search's two core workloads, indexing and querying, is essential to running a successful solution. The search solution must be as economical as feasible. Manage Security Of an Azure AI Search Solution Businesses must be able to rely on their search solutions' security. You have control over the security of the data you search for with Azure AI Search. Overview Of Security Approaches AI Search security builds on Azure's existing network security features. When you think about securing your search solution, you can focus on three areas: Inbound search requests made by users to your search solution Outbound requests from your search solution to other servers to index documents Restricting access at the document level per user search request Data Encryption Like all Azure services, the Azure AI Search service uses service controlled keys to encrypt the data it stores while it's at rest. Indexes, data sources, synonym maps,...

  Index Any Data Using Azure AI Search Push API The most adaptable method for adding data to an Azure AI Search index is the REST API. Any programming language or interactive application that can send JSON requests to an endpoint can be used. Supported REST API Operations There are two supported REST APIs provided by AI Search- Search and management APIs. How To Call Search REST API ? If you want to call any of the search APIs you need to: Use the HTTPS endpoint provided by your search service, you must include an api-version in the URI. The request header must include an api-key attribute. To find the endpoint, api-version, and api-key go to the Azure portal. In the portal, navigate to your search service, then select Search explorer. Choose Keys to locate the api-key on the left. If you're doing more than just accessing the index with the REST API, you can use either the primary or secondary admin key. You can make and use query keys if you only need to search an index. An a...

  Index Data From External Data Sources Using Azure Data Factory An organization's search solution frequently has to include external data that isn't stored in Azure. Because it offers numerous methods for creating and pushing data into indexes, Azure AI Search is adaptable. Push Data Into Search Index Using Azure Data Factory (ADF) One method is to use ADF to push data into an index using a zero-code option. Almost 100 distinct data stores are connected to ADF. You can link an infinite number of data stores using connectors like HTTP and REST. These data stores are utilized in pipelines as either a source or a target (referred to as sinks). In a copy activity, the Azure AI Search index connector can be utilized as a sink. Create ADF Pipeline To Push Data Into Search Index The steps you need to take to use and ADF pipeline to push data into a search index are: Create an Azure AI Search index with all the fields you want to store data in. Create a pipeline with a copy data step....

  How To Use Custom Azure Machine Learning Skillset ? You can create robust AI models and host them on Azure using Azure Machine Learning. The endpoints of these AI models can be utilized in search strategies to improve indexes. Adding a machine learning custom skill to a search index functions in the same way as adding any other custom talent. Custom Azure Machine Learning Skill Schema An Azure Machine Learning (AML) custom skill enriches a search index at the document level. Your document indexer must have an AmlSkill in its skill set. Note that batchSize options are not included in the custom skill. One document at a time will be processed by the AML model. The remaining parameters, timeout and degree of parallelism, regulate how the skill is performed. The timeout value in the aforementioned schema is set at 30 seconds. One should be the starting point for the degree of parallelism. You may be able to raise this figure, depending on your infrastructure. Scaling up the Kubernet...