The AI system developed by DeepMind with Moorfields eye hospital and University College London is capable of referring patients with 94% accuracy. Photograph: Martin Godwin for the Guardian
A new machine-learning system is as good as the best human experts at detecting eye problems and referring patients for treatment, say scientists.
The groundbreaking artificial intelligence system, developed by the AI-outfit DeepMind with Moorfields eye hospital NHS foundation trust and University College London, was capable of correctly referring patients with more than 50 different eye diseases for further treatment with 94% accuracy, matching or beating world-leading eye specialists.
“The results of this pioneering research with DeepMind are very exciting and demonstrate the potential sight-saving impact AI could have for patients,” said Prof Sir Peng Tee Khaw, the director of the NIHR Biomedical Research Centre at Moorfields eye hospital and the UCL Institute of Ophthalmology.
The two-stage AI system takes a more human-like and intelligible approach to analysing the highly complex optical coherence tomography (OCT) scans of patient retinas. These are commonly used to triage patients with sight problems into four clinical categories: urgent, semi-urgent, routine and observation only.
Five separate machine-learning systems, trained using 877 clinical OCT scans, first create maps of the OCT scans. The five maps are then analysed by a second series of five machine-learning systems, trained on maps created from 14,884 OCT scans from 7,621 patients, which interpret the maps and each give a referral decision.
The referral decisions are combined into one result, with a confidence rating expressed as a percentage. The maps and any differing or ambiguous results can be shown visually to a clinician for their own interpretation and explanation of the referral result.
Most other AI-based systems essentially appear as a black box; data is fed in one end and the result is outputted from the other, with no way to check how the system came to its decision.
“The number of eye scans we’re performing is growing at a pace much faster than human experts are able to interpret them,” said Dr Pearse Keane, a consultant ophthalmologist at Moorfields eye hospital. “The AI technology we’re developing is designed to prioritise patients who need to be seen and treated urgently by a doctor or eye care professional. If we can diagnose and treat eye conditions early, it gives us the best chance of saving people’s sight.”
The two-stage approach also makes the systems more adaptable to different OCT machines, which produce images with differing characteristics. Only the mapping system would need to be retrained for different machines, leaving the existing referral system intact.
The next stage is to put the AI system through clinical trials and regulatory approval before it can be used in hospitals for patient referrals. If granted approval, the system will then be available for use across all of Moorfields’ sites for five years.
The researchers said the intelligible AI system could also be used to help train clinicians, and that Moorfields could use it for future non-commercial research efforts, as well as the de-identified dataset used to train it.
Experts said AI systems such as those created by the researchers had the potential to help clinicians treat more patients and make the NHS’s limited resources go further.
Robert Dufton, the chief executive at Moorfields Eye Charity, said: “The need for treatment for eye diseases is forecast to grow, in part because people are living longer, far beyond our ability to meet the demand using current practice.
“Artificial intelligence is showing the potential to transform the speed at which diseases can be diagnosed and treatments suggested, making the best use of the limited time of clinicians.”
Prints made by Tom White appear to humans as meaningless blobs, but to AI algorithms they look like specific objects and items. Photo: Ramesh Pathania
We live in a world that’s increasingly controlled by what might be called “the algorithmic gaze.” As we cede more decision-making power to machines in domains like health care, transportation, and security, the world as seen by computers becomes the dominant reality. If a facial recognition system doesn’t recognize the color of your skin, for example, it won’t acknowledge your existence. If a self-driving car can’t see you walk across the road, it’ll drive right through you. That’s the algorithmic gaze in action.
This sort of slow-burning structural change can be difficult to comprehend. But as is so often the case with societal shifts, artists are leaping headfirst into the epistemological fray. One of the best of these is Tom White, a lecturer in computational design at the University of Wellington in New Zealand whose art depicts the world, not as humans see it, but as algorithms do.
White started making this kind of artwork in late 2017 with a series of prints called “The Treachery of ImageNet.” The name combines the title of René Magritte’s famous painting of a pipe that isn’t a pipe, and ImageNet, a database of pictures that’s used across the industry to train and test machine vision algorithms. “It seemed like a natural parallel for me,” White tells The Verge. “Plus, I can’t resist a pun.”
To humans, the pictures look like haphazard arrangements of lines and blobs that lack any obvious immediate structure. But to algorithms trained to see the world on our behalf, they leap off the page as specific objects: electric fans, sewing machines, and lawnmowers. The prints are optical illusions, but only computers can see the hidden image.
1 of 6
An electric fan
White’s work has attracted a lot of attention in the machine learning community, and it’s getting its first major gallery show this month as part of an exhibition of AI artwork in India at Delhi’s Nature Morte gallery. White says he designs his prints to “see the world through the eyes of a machine” and make “a voice for the machine to speak in.”
That “voice” is actually a series of algorithms that White has dubbed his “Perception Engines.” They take the data that machine vision algorithms are trained on — databases of thousands of pictures of objects — and distill it into abstract shapes. These shapes are then fed back into the same algorithms to see if they’re recognized. If not, the image is tweaked and sent back, again and again, until it is. It’s a trial and error process that essentially ends up reverse-engineering the algorithm’s understanding of the world.
White compares the process to a “computational ouija board,” where neural networks “simultaneously nudge and push a drawing toward the objective.” He tells The Verge that this method gives him the control he wants out of the output, though it can take days to create a single image in this way, and he admits that the process is “kind of tedious.”
Unlike some artists who work with machine learning, White doesn’t pretend that his prints are the product of a some autonomous AI (a disingenuous narrative sometimes pushed by artists and promoters in order to create a feeling of technological mysticism). Instead, he’s up front about his role: he sets a number of starting parameters for his perception engines, like the colors and thickness of lines, and winnows the output, rejecting prints that he doesn’t find aesthetically pleasing. Although he is giving his algorithms a voice to speak in, he’s also making sure the results are pleasant to hear. “I think I am trying to free the algorithm so it can express itself, so people can relate to what it’s saying,” he says.
And what is it saying? Well, as with any art, different people hear different things.
Some see the imagery made by White and his peers as a bad omen, another sign that artificial intelligence is not only getting smarter but beginning to think creatively and take on roles reserved for humans. Karthik Kalyanaraman, one half of the curation team responsible for the Nature Morte exhibition, tells The Verge by email that he arranged the show to draw attention to the “inevitable” questions we face about the future of humanity. “Once so much of our labor (manual, mental, emotional, artistic) is replaced by machines, what is left for us to do?” he asks. “How will we define ourselves?”
Kalyanaraman suggests that art made with AI demonstrates that computers may deserve credit as creative actors. The type of machine learning used by White and his peers works by sifting through large amounts of data and then replicating the patterns it finds. Kalyanaraman suggests that this is similar to the process by which humans learn art, but that our “mysticism” surrounding the notion of creativity stops us from seeing the parallels. “If a machine can make humanly surprising, stylistically new kinds of art, I think it is foolish to say well it’s not really creative because it doesn’t have consciousness,” he says.
Others frame the question in more ruthless economic terms. Writing for contemporary art magazine frieze, Mike Pepi suggests the promotion of AI creativity is essentially propaganda for corporate interests. Pepi says that despite “utopian prognostication,” the development of artificial intelligence is ultimately about replacing human labor, including white-collar jobs that need creative abilities. Says Pepi: “If machine intelligence can conquer this uniquely human realm, the march to artificial general intelligence must be nigh, and the profits unimaginable.”
White stood next to his prints (including the cello, in orange) at the Nature Morte gallery. Photo: Ramesh Pathania
White says his motivation is primarily to deconstruct what we think of as machine perception. In other words: to explain the algorithmic gaze. Take the example of the cello print in White’s series “The Treachery of ImageNet.” If you know what you’re looking for, you can see shapes that represent the instrument (a cluster of straight parallel lines bracketed by curves). But there’s also a confusing shape looming behind it. White says these shapes are there because the algorithms were trained using pictures of cellos with cellists holding them. Because the algorithm has no prior knowledge of the world — no understanding of what an instrument is or any concept of music or performance — it naturally grouped the two together. After all, that’s what it’s been asked to do: learn what’s in the picture.
This sort of mistake is common in machine learning, and it demonstrates a number of important lessons. It shows how critical training data is: give an AI system the wrong data to learn from, and it’ll learn the wrong thing. It also demonstrates that no matter how “clever” these systems seem, they possess a brittle intelligence that only understands a slice of the world — and even that, imperfectly. White’s latest prints for the Nature Morte gallery, for example, are abstract smears of color designed to be flagged as “inappropriate content” by Google’s algorithms. The same algorithms used to filter what humans see around the world.
Still, White says that he doesn’t see his artwork as a warning. “I’m just trying to present the algorithms as they are,” he says. “But I admit it’s sometimes alarming that these machines we’re relying on have such a different take on how objects in the world are grounded.”
And despite the error-prone nature of algorithmic gaze, it can also do very beneficial things. Machine vision could make the world a safer place by steering cars safely on roads or save lives by speeding up medical diagnoses. But if we really want to use this technology for good, we need to understand it better. Looking at the world through an algorithm’s eyes might be the first step.
Applications and infrastructure evolve in lock-step. That point has been amply made, and since this is the AI regeneration era, infrastructure is both enabling AI applications to make sense of the world and evolving to better serve their needs.
As things usually go, the new infrastructure stack to power AI applications has been envisioned and given a name -- Infrastructure 3.0 -- before it is fully fledged. We set off to explore both the obvious, here and now, and the less obvious, visionary parts of this stack.
In order to keep things manageable, we will limit ourselves to "specialized hardware with many computing cores and high bandwidth memory" and call it AI chips for short. We take a look at how these AI chips can benefit data-centric tasks, both in terms of operational databases and analytics as well as machine learning (ML).
Let us commence on the first part of this journey with the low-hanging fruit: GPUs and FPGAs.
In order to keep things manageable, we will limit ourselves to "specialized hardware with many computing cores and high bandwidth memory" and call it AI chips for short. We take a look at how these AI chips can benefit data-centric tasks, both in terms of operational databases and analytics as well as machine learning (ML).
Let us commence on the first part of this journey with the low-hanging fruit: GPUs and FPGAs.
GPUs
Graphical Processing Units (GPUs) have been around for a while. Initially designed to serve the need for fast rendering, mainly for the gaming industry, the architecture of GPUs has proven a good match for machine learning.
Lest we forget, the new Nvidia GPUs actually bring improvements for graphics rendering. But, more importantly for our purposes, they pack Tensor Cores, the company's specialized architecture for machine learning, and introduces NGX. NGX is a technology which, as Nvidia puts it, brings AI into the graphics pipelines: "NGX technology brings capabilities such as taking a standard camera feed and creating super slow motion like you'd get from a $100,000+ specialized camera."
That may not be all that exciting if you are interested in general-purpose ML, but the capabilities of the new Nvidia cards sure are. Its prices, however, definitely reflect their high-end nature, ranging from US$2.5K to $10K.
GPUs can greatly accelerate workloads that can be broken down in parts to be executed in parallel, working in tandem with CPUs. Image: SQream.
But it takes more than a hardware architecture to leverage GPUs -- it also takes software. And this is where things have gone right for Nvidia, and wrong for the competition, such as AMD. The reason Nvidia is so far ahead in the use of GPUs for machine learning applications lies in the libraries (CUDA and cuDNN) needed to use GPUs.
Although there is an alternative software layer that can work with AMD GPUs, called OpenCL, maturity and support for it are not at par with Nvidia's libraries at this point. AMD is trying to catch up, and it also competes on the hardware front, but there is a bigger point to be made here.
In order to benefit from AI chips, the investment required goes beyond the hardware. A software layer that sits on top of these chips to optimize code running on them is required. Without it, they are practically unusable. But learning how to make use of this layer is also needed.
We already mentioned how GPUs are currently the AI chip of choice for ML workloads. Most popular ML libraries support GPUs -- Caffe, CNTK, DeepLearning4j, H2O, MXnet, PyTorch, SciKit, and TensorFlow to name just a few. In addition to learning the specifics of each library, building it for GPU environments is often needed too.
As for plain-old data operations and analytics -- one word: GPU databases. A new class of databases systems have been developed with the goal of utilizing GPU parallelism under the hood to bring the benefits of off-the-shelf hardware to mainstream application development. Some of the options in this space are BlazingDB, Brytlyt, Kinetica, MapD, PG-Strom, and SQream.
FPGAs
Field Programmable Gate Arrays (FPGAs) are not really new either -- they have been around since the 80s. The main idea behind them is that, as opposed to other chips, they can be reconfigured on demand. You may wonder how is this possible, how does this make them specialized, and what are they good for.
FPGAs can be simplistically thought of as boards containing low-level chip fundamentals, such as AND and OR gates. FPGA configuration is typically specified using a hardware description language (HDL). Using this HDL the fundamentals can be configured in a way that matches the requirements of specific tasks or applications, in essence mimicking application-specific integrated circuits (ASICs).
Having to reprogram your chips via HDL for every different application sounds complex. So again, the software layer is crucial. According to Jim McGregor, principal analyst with Tirias Research, "the toolset to build FPGAs is still ancient. Nvidia has done well with the CUDA language to leverage GPUs. With FPGA it's still kind of a black art to build an algorithm efficiently."
Intel is throwing its weight behind FPGAs, possibly as a way to make up for having being left behind in GPUs. But the FPGA software layer is yet not as mature as that of GPUs. Image: Intel
But that may be changing. Originally it was Intel who showed interest in FPGAs, acquiring Altera, one of the key FPGA manufacturers. It is possible that this is Intel's way of pushing into the AI chips world, which will be increasingly important, after having been left behind in the GPU battle. But, complexity aside, can FPGAs compete?
What this means in plain english is that Intel's FPGAs could compete with GPUs, as long as low precision data types are used. This may sound bad, but it is actually an emerging trend in DL. The rationale is to simplify calculations, while maintaining comparable accuracy.
That may well mean that there is a bright future in using FPGAs for ML. Today, however, things do not look that bright. In verification of McGregor's statement, there does not seem to be a single ML library that supports FPGAs out of the box. There is work under way to make using FPGAs possible with TensorFlow, but precious little else besides that.
Things are different when it comes to data operations and analytics however. Recently Intel presented some of the partners it works with for FPGA-accelerated analytics. Swarm64 looks like the most interesting among them, promising immediate speedup of up to 12 times for PostgreSQL, MariaDB, and MySQL. Other options are rENIAC, offering what it says is a times-13 accelerated version of Cassandra, and Algo-Logic, with its custom key-value store.
Hard choices, in the cloud and on-premise
As usual, there is an array of hard choices to be made with emerging technology, and hardware is no exception. Should you build your own infrastructure, or use the cloud? Should you wait until offerings become more mature, or jump onboard now and reap the early adopter benefits? Should you go for GPUs, or FPGAs? And then, which GPU or FPGA vendor?
When discussing GPU databases with fellow ZDNet contributor and analyst Tony Baer, for example, Baer opined that none of them have a future on their own. That is because, according to Baer, the economics of GPUs are such that only cloud providers will be able to accumulate them at scale, therefore GPU database vendors will be eventual targets for acquisition by cloud-based databases.
In fact, one such acquisition, that of Blazegraph by AWS, has already transpired. But while that does make sense, it's not the only plausible scenario. If we're talking about acquisitions, it's entirely possible that GPU databases could be acquired by non-cloud database vendors who will want to bring such capabilities to their products.
It is also possible that some GPU database vendors will come into their own. GPU databases may seem less mature compared to incumbents now, but the same could be said for many NoSQL solutions 10 years ago. GPU databases seem like a tempting option for everyday operations and analytics, although the question remains as to whether the cost of replacing existing systems is outweighed by the gains in performance.
Swarm64 and rENIAC, on the other hand, are FPGA offerings that promise to leave your existing infrastructure as untouched as possible, especially in the case of Swarm64. Although their maturity remains an open question, the idea of "simply" adding hardware to your existing database and getting a much better performance out of it sounds promising.
As far as the GPU versus FPGA question is concerned, GPUs seem to have a wider and more mature ecosystem, but FPGAs offer superior flexibility. It has also been suggested that FPGAs may offer a better performance/consumption ratio, and that going forward GPUs may have trouble keeping up with low precision data types, as they would have to redesign extensively to support this.
Which one works best for you - GPUs or FPGAs? Cloud, or on premise?
ktsimage, Getty Images/iStockphoto
In terms of what GPU or FPGA vendor to choose, the options are intertwined with the cloud or on premise question. GPUs are on offer on AWS, Azure, Google Cloud, all of which use Nvidia for their GPU-enabled instances. FPGAs, on the other hand, are on offer on AWS (EC2 F1 powered by Xilinx) and Azure (Project Brainwave powered by Intel), but not on Google Cloud.
For the million dollar question -- should you go cloud or build your own infrastructure -- the answer may be not that different from what applies in general: it depends.
If you use your infrastructure enough, perhaps it would make sense to invest in buying and installing, but for occasional use the cloud seems like a better fit. For other cases it might as well be mix-and-match.
Of course, we have not covered all options -- these are neither the only clouds, nor the only AI chips in town. This is a nascent area with many emerging players, and we will be revisiting it soon.
/cdn.vox-cdn.com/uploads/chorus_image/image/60965407/_MG_3406.0.jpg)
/cdn.vox-cdn.com/uploads/chorus_asset/file/12326541/electric_fan.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/12326601/TomWhite.jpg)
