Image classification & content description is incredibly powerful. Cloud-based computer vision services instantly return a JSON-based description of what they see in photos.
However, most examples are quite complex. As a beginning developer with your main knowledge in HTML + JavaScript, the following code is for you. You don’t need to worry about Node.js or native apps. The code runs directly in your browser from your computer.
Creating apps that work well with Augmented Reality requires some background knowledge of the image processing algorithms that work behind the scenes. One of the most fundamental concepts involves anchors. These rely on keypoints and their descriptors, detected in the recording of the real world.
AR development APIs hide much of the complexity. As a developer, you simply anchor virtual objects to the world. This ensures that the hologram stays glued to the physical location where you put it.
Updated: January 18th, 2023 – changed info from Microsoft LUIS to Microsoft Azure Cognitive Services / Conversational Language Understanding.
In this last part, we bring the vital sign checklist to life. Artificial Intelligence interprets assessments spoken in natural language. It extracts the relevant information and manages an up-to-date, browser-based checklist. Real-time communication is handled through Web Sockets with Socket.IO.
The example scenario focuses on a vital signs checklist in a hospital. The same concept applies to countless other use cases.
In this article, we’ll query the Microsoft Azure Conversational Language Understanding Service from a Node.js backend. The results are communicated to the client through Socket.IO.
In the previous article, we verified that our language understanding service works fine. Now, it’s time to connect all components. The aim is to query our model endpoint from our Node.js backend.
Updated: December 12th, 2022 – changed info from Microsoft LUIS to Microsoft Azure Cognitive Services / Conversational Language Understanding.
Training Artificial Intelligence to perform real-life tasks has been painful. The latest AI services now offer more accessible user interfaces. These require little knowledge about machine learning. The Microsoft Azure Conversational Language Understanding Service performs an amazing task: interpreting natural language sentences and extracting relevant parts. You only need to provide 5+ sample sentences per scenario.
In this article series, we’re creating a sample app that interprets assessments from vital signs checks in hospitals. It filters out relevant information like the measured temperature or pupillary response. Yet, it’s easy to extend the scenario to any other area.
After creating the backend service and the client user interface in the first two parts, we now start setting up the actual language understanding service. I’m using the Conversational Language Understanding service from Microsoft, which is based on the Cognitive Services of Microsoft Azure.
Updated: December 12th, 2022 – changed info from Microsoft LUIS to Microsoft Azure Cognitive Services / Conversational Language Understanding.
The vision: automatic checklists, filled out by simply listening to users explaining what they observe. The architecture of the sample app is based on a lightweight architecture: HTML5, Node.js & the Microsoft Conversational Language Understanding Cognitive service in the cloud.
Such an app would be incredibly useful in a hospital, where nurses need to perform and log countless vital sign checks with patients every day.
In part 1 of the article, I’ve explained the overall architecture of the service. In this part, we get hands-on and start implementing the Node.js-based backend. It will ultimately handle all the central messaging. It communicates both with the client user interface running in a browser, as well as the Microsoft LUIS language understanding service in the Azure Cloud.
Node.js is a great fit for such a service. It’s easy to setup and uses JavaScript for development. Also, the code runs locally for development, allowing rapid testing. But it’s easy to deploy it to a dedicated server or the cloud later.
I’m using the latest version of Node.js LTS (currently version 18) and the free Visual Studio Code IDE for editing the script files.
Updated: December 12th, 2022 – changed info from Microsoft LUIS to Microsoft Azure Cognitive Services / Conversational Language Understanding.
During the last few years, cognitive services have become immensely powerful. Especially interesting is natural language understanding. Using the latest tools, training the computer to understand spoken sentences and to extract information is reduced to a matter of minutes. We as humans no longer need to learn how to speak with a computer; it simply understands us.
I’ll show you how to use the Conversational Language Understanding Cognitive Service from Microsoft. The aim is to build an automated checklist for nurses working at hospitals. Every morning, they record the vital sign of every patient. At the same time, they document the measurements on paper checklists.
With the new app developed in this article, the process is much easier. While checking the vital signs, nurses usually talk to the patients about their assessments. The “Vital Signs Checklist” app filters out the relevant data (e.g., the temperature or the pupillary response) and marks it in a checklist. Nurses no longer have to pick up a pen to manually record the information.
In this article, we’ll create a simple app that uses the conversational language understanding APIs of the Microsoft Azure Cognitive Services. The service extracts the relevant data from freely written or spoken assessments.
Mobile Apps are no longer simple tools and games. They have grown to amazingly complex systems. Which ingredients are necessary to successfully develop a next generation app?
You will need Artificial Intelligence / Machine Learning. Only the best performance will satisfy your customers – or do you want to wait more than a few seconds in a mobile app? How can you visualize your user interface with HoloLens?
At the #mdaw15, you will learn how to develop and plan such apps with modern frameworks. Join the next After-Work event in St. Pölten for free at https://mobility.builders/
#mdaw events are mixtures of conferences with more casual meet-ups. Different expert speakers approach an overall topic from diverse perspectives. Afterwards, there’s plenty of time for networking and discussion amongst attendees – with snacks and drinks, of course.
The events target mobile developers and decision makers. The goal is to dive deeper into relevant and already known topic areas, as well as to keep up to date with the constantly evolving and changing toolset of the mobile world.
In the meantime, we can look back to 14 previous #mdaw events. Topics so far included business apps, digital healthcare, Xamarin, user experience or a Refugee Hackathon. We’re organizing the events with technology partners like Microsoft, Oracle, IBM, the City of Vienna, and many others.
The community has grown to around 500 attendees, and we continue to get more and more developers on board with each event!
Together with Helmut Krämer, I’ve founded the community in 2013 and am proud that it’s still around. We’ve had a tremendous impact on the Austrian developer community so far. With the upcoming #mdaw15, we’re extending the reach geographically and host our first event outside of Vienna, to reach even more developers!