Today Microsoft is hosting a developer day, and one of the highlights they are showcasing is a new API called WinML. Artificial Intelligence and Machine Learning are two of the biggest trends in computing these days, but much of that compute is done in cloud datacenters, where custom servers have specialized hardware to improve the performance and lower the energy consumption. But as with everything in computing, the landscape is cyclical, and we’re once again looking at moving some of that compute back to the edge devices, such as PCs, tablets, and IoT.

WinML is a new set of APIs which will let developers harness the complete capabilities of any Windows 10 device to use pre-trained machine learning models, allowing AI tasks to be offloaded from the cloud, for several reasons.

The first reason is performance. Despite the massive computing power available in the cloud, we still live in a world where moving data to the cloud can be prohibitive in terms of cost, and speed. The latency of any network connection is orders of magnitude slower than local memory access, and working on massive datasets can be difficult without expensive, dedicated, high-bandwidth interconnects. Performing the compute tasks locally can significantly improve performance thanks to the lower latency, and offer up real-time results. Operational costs can be saved through reduction in network bandwidth, as well as less cloud-compute time.

Do to compliance and security concerns, many industries would not be able to leverage machine learning and AI to process datasets, so WinML provides them an easy option to move that work away from the cloud, meaning it’s much easier to ensure you are compliant with all necessary regulations.

An example provided by a Microsoft spokesperson was that of an industrial system where an image is taken of a product, and the system has to determine if that product is correct, or if it has a defect. This is an ideal workload for AI tools, such as Microsoft’s Computer Vision API. Moving this type of compute to an industrial PC, which may be offline, provides excellent performance without an ongoing cost, or the requirement of a cloud connection.

Microsoft’s new set of AI APIs offer several key benefits which should help developers integrate them into their products. Arguably the most important is that the API does all of the heavy lifting for the developer, so the developer doesn’t have to worry about what kind of hardware is available in any machine their app is going to run on. The WinML engine will leverage the hardware dynamically, and create code to get the maximum performance available from whatever hardware the device is running on.  The engine is built on Direct 3D, and if the system has a DX12 capable GPU, will utilize DX12 compute shaders dynamically. If you have a massive GPU with plenty of VRAM, the workload will be offloaded to the GPU. If a DX12 GPU is not available, or performance is an issue because of integrated graphics, the engine can also fall back to a CPU path, but the CPU path leverages all of the silicon as well, with support for things like AVX512. The code is generated at runtime, so it will always optimize for the hardware available, and the engine will get updated as well to take advantage of new silicon such as Intel’s Movidius VPU. Perhaps even more impressive is that the ML engine will even work on a new SnapDragon 835 based PC, or even IoT devices. If you have lots of performance available, it will leverage that, but if the system is a low-power device, it will still work.

A key point here is that the WinML API is meant for models that have already been trained. The training itself would be done ahead of time, and then the models and data supplied to the engine to perform the analysis and provide probabilities.

The new API will utilize the industry standard ONNX ML models, and ONNX is driven by Microsoft, Facebook, and Amazon, with support from all the key hardware players such as AMD, Intel, and NVIDIA. ONNX models can be trained using the Azure Machine Learning Workbench, and then the model will be able to be brought into Visual Studio which will create a wrapper for the engine to utilize. Visual Studio Preview 15.7 will support this, and earlier versions can use the MLGen tool to manually add it to projects, and ONNX models will be able to be directly exported from Azure.

Microsoft was candid enough to state that the idea for WinML really only gained traction around May of 2017, and work began on the new API in just August of last year, so in just a few months, they’ve gone from an idea to a new ML toolkit they are bringing to developers. That timeline is fairly impressive, but the response praised the capabilities built into DX12, as well as the high-quality drivers, which allowed the team to deploy this to scale in such a short time.

AI and ML are two of the big keywords in the development and hardware industry these days, and WinML and the AI Platform for Windows developers should help bring those capabilities to even more tasks.

Source: Microsoft

Comments Locked


View All Comments

  • shabby - Wednesday, March 7, 2018 - link

    Ai this, ai that, this word is already losing its meaning.
  • Stochastic - Wednesday, March 7, 2018 - link

    Pattern recognizer isn't as marketable.
  • close - Thursday, March 8, 2018 - link

    Neither is saying that it's using your processing power in a way that is some situations is quite similar to how Google uses people to teach their AIs image recognition with "training CAPTCHAs".
  • Holliday75 - Wednesday, March 7, 2018 - link

    Pre-programmed intelligence.
  • Gunbuster - Thursday, March 8, 2018 - link

    Cortana still cant answer "what time is sunset tonight", I'm not holding my breath for an "engaging and magical" experience from the gang in Redmond anytime soon.
  • peevee - Thursday, March 8, 2018 - link

    And why would anybody limit themselves to MS, Windows 10, DX12 and Win ML when you can pick up a multi-platform API like TensorFlow?
  • MrSpadge - Thursday, March 8, 2018 - link

    I guess because "it just works" and integrates nicely with their usual environment. Don't know if TensorFlow provides the same flexibility and ease of use as MS promises here.
  • MrSpadge - Thursday, March 8, 2018 - link

    It would be really helpful for the Win environment if people used this to make better software, leveraging the many core CPUs and powerful GPUs available.
  • linuxgeex - Thursday, March 22, 2018 - link

    Armchair response much?

    It just works? So you've developed some products with it already I see. Very good.

    It would be really helpful for Win ev. ppl if they used this to leverage CPU/GPU cores... um... Cuda/OpenCL etc have been around a long time for that. An AI API isn't going to magically make them start making better use of parallelism. If anything this is going to encourage lazy programmers to burn cycles pointlessly using AI for things that better optimisation algorithms would have done quicker, more accurately, and with way way way less power.

    What it will do a brilliant job of is monetizing the Windows subscriber base by targeting them with app/product recommendations and referrals, which is the real reason Redmond is in such a rush to land this on the desktop. Secondary to that is improving on-screen keyboard predictions and voice recognition, possibly improve touchscreen accuracy too by recognizing failure patterns and ignoring them. Tertiary to that is whatever developers want to do with it... more advertising, user monitoring, and perhaps a small amount of actual AI towards optimising UI's, resource usage, and gaming experiences.

Log in

Don't have an account? Sign up now