As a business leader, you know the value of having access to the right information at the right time. Whether you need to make strategic decisions, solve problems, or answer customer queries, you want to leverage the knowledge that is stored in your documents and databases. But how can you find the most relevant and accurate answers from the vast amount of unstructured data that you have?
One possible solution is to use generative AI, a type of artificial intelligence that can create new content based on existing data. Generative AI can help you transform your unstructured data into a searchable and interactive knowledge base that can power natural language conversations with your end users. In this blog post, we will show you how you can use Azure OpenAI and Cognitive Search to create chat bots that can answer challenging questions from your documents.
What is Azure OpenAI and Cognitive Search?
Azure OpenAI is a cloud service that provides access to powerful language models, such as GPT-3, that can generate natural language responses based on user inputs. These models can understand the context and intent of the user's query and provide relevant and coherent answers. Azure OpenAI can also help you create prompts, which are the instructions that tell the model what to do with the user's input.
Azure Cognitive Search is a cloud search service that can index, understand, and retrieve the right information across your enterprise content. It can also enable semantic search, which is a collection of query-related capabilities that bring semantic relevance and language understanding to textual search results. Azure Cognitive Search can help you extract text, tables, structure, and key-value pairs from your documents using Form Recognizer, and enrich your search index with natural language processing capabilities.
How can you use Azure OpenAI and Cognitive Search to create chat bots?
The basic idea is to use Azure OpenAI to interact with the user in natural language and to use Azure Cognitive Search to retrieve the most likely answers from your documents. The solution has the following steps:
- Data ingestion: You upload your documents, such as PDFs, into Azure Blob Storage. A function gets triggered on new file upload, which processes the files and performs document chunking, content parsing, and indexing using Azure Cognitive Search.
- Data embedding: You generate embeddings for all items in your search index using one of the language models from Azure OpenAI. Embeddings are information-dense representations of the semantic meaning of a piece of text, that can be easily utilized by machine learning models and algorithms. You store the embeddings in your search index.
- Query processing: You write a prompt or a question in a chat UI, such as a web app, a chat bot, or a Teams app. The chat UI sends the prompt to Azure OpenAI Completions API, which generates a prompt response to be displayed in the chat UI. The prompt response may also include a query embedding, which is the embedding of the user's input generated by the same language model.
- Answer retrieval: You find the most likely answers by performing a vector similarity search of the query embedding on the search index. This is done by calculating the cosine similarity between the query vector and each of the vectors in the index and selecting the items with the highest similarity values. You may also use semantic ranking, hit highlighting, or summarization techniques to improve the precision and readability of the results.
- Answer presentation: You display the answers to the user in the chat UI, along with the source document and the confidence score. You may also provide feedback mechanisms, such as thumbs up or down, to improve the model's performance over time.
What are the benefits of using Azure OpenAI and Cognitive Search for knowledge management?
By using Azure OpenAI and Cognitive Search for knowledge management, you can achieve the following benefits:
- Improve user experience: You can provide your end users with a natural and engaging way to find answers to their questions, without having to browse through multiple documents or use complex search queries.
- Increase productivity and efficiency: You can reduce the time and effort required to access the information you need, and focus on the tasks that matter most to your business.
- Leverage existing data: You can make use of the data that you already have, without having to create new content or structure it manually.
- Scale and customize: You can scale your solution to handle large volumes of data and queries, and customize it to fit your specific use case and domain.
How can you get started with Azure OpenAI and Cognitive Search for knowledge management?
If you are interested in using Azure OpenAI and Cognitive Search for knowledge management, you can request to get started with a pilot of a chatbot inside teams. Before getting started, you will want to consider:
- The business problem that you want to solve with knowledge management
- The type and number of documents that you want to use for knowledge management
- The type of data in those documents. For instance, are you dealing with any regulated data like controlled unclassified information (CUI).
- The Azure subscription that you want to use for the pilot deployment
- The business and technical decision makers and SMEs that will be involved in the pilot
The pilot should include the following phases:
- Initiation and solution definition: Conduct an initial workshop to understand your data and your use case and define the solution scope and architecture.
- Implementation and testing: Deploy the components of the solution, build the knowledge base, and test the chat UI.
- Deployment and handover: Demo the solution to you, and hand over the source code and the deployment steps.
Example: Chatbot using Azure OpenAI in GCCH
The Defense Industrial Base (DIB) is going through significant changes due to regulatory changes and market incentives. Many are adopting Industry 4.0 by bringing systems online to improve efficiency and operational performance. The DIB want to empower end users to be find the answers they need when they need it. One way to do this is use a chatbot embedded into Microsoft Teams so end users can ask questions in a natural language and receive relevant answers to their questions. Since the DIB uses Microsoft’s GCCH platform, we will share how to build this in GCCH.
- Download the latest version of Visual Studio and Install it
- Launch Visual Studio
- Navigate to the left side where it says extensions
- Search for “Teams Toolkit” and install it
- Search for “Azure App Service” and install it
- Search for “Azure Resources” and install it
- Search for "Azure Account” and install it
- Go into Azure
- Right click and go to “settings”
- Find Azure Settings
- Find the setting for your current cloud and switch to AzureUSGovernment (if you want to do this in a commercial cloud leave this setting as is)
- Log into the extensions with the correct azure and M365 accounts for automated deployment. (Optional)
If not logged in, you cannot use Teams Toolkit for automated deployment, but you can still build your solution and upload/provision it manually with other methods.
- Click on Teams Toolkit
- Navigate to view samples.
- Click on Teams Chef Bot
- Follow directions to create chat bot.
- Link to Azure OpenAI in Gov. You will need to go through the eligibility process.
- Click play
You now have a baseline for a Chatbot operational. From here you can start considering large language models, branding, etc.
To learn more about building a custom copilot teams integration using AzureOpenAI in Azure Government watch our recent Microsoft Build Session: Custom copilot Teams Integration in Government applications.
Conclusion
Knowledge management is a powerful way to unlock the insights hidden in your data and provide them to your end users in a natural and conversational way. By using Azure OpenAI and Cognitive Search, you can create chat bots in Microsoft Teams that can answer complex questions from your documents, and improve your user experience, productivity, efficiency, and scalability. If you need help building out your knowledge management chatbot reach out to your Microsoft Account Team and/or Microsoft Partner to help get started.
Additional Resources
