Conversation Language Understanding Model (Part 3)

By Ashwin Venugopal -

 




Train Your Classification Model

It requires recognized data to be trained, just like any other AI model. In order for the model to be tested, it must see examples of how to map data to a class. You have the option to allow the model to automatically divide your training data; by default, it will utilize 20% of the documents for blind testing and 80% of the documents for training. You can designate certain documents for testing if you want to use them to test your model.

Choose Data labeling for your project in Language Studio. All of your documents will be visible to you. After choosing each document to include in the testing set, click Testing the model's performance. Create a new training job after saving your amended labels.

Create Search Index

Creating a search index that will be enhanced by a custom text classification model doesn't require any special actions on your part. Once you have developed a function app, you will be changing the index, indexer, and custom skill.

Create an Azure Function App

For your function app, you can select the technologies and language of your choice. The application must be able to send JSON to the API for custom text classification. Next, handle the model's JSON answer. After that, the function sends a structured JSON message back to an AI Search custom skillset.

There are five things the function app needs to know-

  1. The text to be classified.
  2. The endpoint for your trained custom text classification deployed model.
  3. The primary key for the custom text classification project.
  4. The project name.
  5. The deployment name.

The first is passed from your custom skillset in AI Search to the function as input. The remaining four can be found in Language Studio.

The endpoint and deployment name is on the deploying a model pane and the project name as well as primary key are on the project settings pane.

Update Your Azure AI Search Solution

There are three changes in the Azure portal you need to make to enrich your search index-

  1. You need to add a field to your index to store the custom text classification enrichment.
  2. You need to add a custom skillset to call your function app with the text to classify.
  3. You need to map the response from the skillset into the index.

Conclusion

We have successfully learnt about the training of classification model.























Comments

Post a Comment

Popular posts from this blog

Connect Data to Azure Sentinel Using Data Connectors

By Ashwin Venugopal -
Image
  Ingest Log Data with Data Connectors You should connect your data sources with Azure Sentinel Connectors to collect the log data whose page displays a growing list of the connectors provided by Azure Sentinel, after selecting the Open connector page (the detailed connector page has a left and right blade).  The information about the connector, the connector's status, and the last time a log was received if connected is provided by the left blade. Whereas the right tab has two tabs- Instructions and Next Steps. The instructions tab can be different based on the connector having Prerequisites and Configuration which can be followed to connect to the data source. The Next Steps tab offers a quick reference to workbooks, query samples, and analytical templates. Data connectors can only be disconnected/deactivated, not deleted. Note-  The connector does not install Workbooks and Analytical Templates as they are already available in the Sentinel environment for out of the box...
Read more

Information Protection Scanner: Resolve Issues with Information Protection Scanner Deployment

By Ashwin Venugopal -
Image
  About If you are experiencing problems with the Microsoft Purview Information Protection scanner, check the health of your deployment by utilizing the Start-ScannerDiagnosticsPowerShell cmdlet to start the scanner diagnostic tool. The diagnostics tool checks the following details and then creates a log files with the results: Whether the database is up to date. Whether network URLs are accessible. Whether there's a valid authentication token and policy can be acquired. Whether the profile is defined in the Azure portal. Whether offline/online configuration exists and can be acquired. Whether the rules configured are valid.  The Start-ScannerDiagnostics command does not perform a comprehensive check of prerequisites. If you encounter problems with the scanner, you need to verify that your system meets the scanner's requirements, and that your scanner's configuration and installation are finished.  Verify Scanning Details Per Scanner Node and Repository Run the Get-ScanSt...
Read more

Azure AI Search plugin in Microsoft Security Copilot (Preview)

By Ashwin Venugopal -
Image
  Introduction Azure AI Search is a service that allows to reliably extract content search, extract, and understand at scale. It is typically used to search for documents, data surveys, and chat-style Copilot apps over proprietary data. Indexing and querying are Azure AI Search's two main workloads. While querying allows to search the content from index, indexing loads the content into the search engine and makes it searchable. By connecting to one of the Azure AI Search indexes from within Copilot, the Microsoft Security Copilot integration for Azure enables to use the content in Copilot to produce contextualized, pertinent, and targeted responses.  Requirement Before using the Azure AI Search plugin in Copilot, the desired Azure AI Search index to connect to Copilot is configured as follows: The text field must be searchable.  The title field must be filterable. The vector-field must use text-embedding-ada-002. Integrated vectorization can be used to setup an index in t...
Read more