Die Geburt einer neuen Maschine

Das Ende ist nah. In der Halbleitertechnik werden die Grenzen der Miniaturisierung bald erreicht sein. Ab Anfang der 1960er Jahre wurden Computerchips laufend kleiner und schneller. Der Fortschritt war planbar, die Zukunft liess sich auf der Basis des Mooreschen Gesetzes voraussagen. Bald aber wird dieses Gesetz seine Gültigkeit verlieren, bald werden die Grenzen der Miniaturisierung erreicht sein. Dann wird's spannend.

Was würde Google tun?

Am vergangenen Freitag hat Google unter dem Namen Zaius P9 dieSpezifikation eines neuen Server-Computers vorgestellt. Kaum eine Firma hat so viele Rechner im Einsatz wie Google, deshalb horchen in der Branche alle auf, wenn Google darlegt, wie ein guter Server-Computer auszusehen hat. Ausgangspunkt bei der Entwicklung der neuen Maschine waren die Vorgaben des Open Compute Project. Dieses Projekt, 2011 von Facebook gestartet, ist eine Art Selbsthilfegruppe: Firmen, die grosse Rechenzentren betreiben, haben angefangen, das Design von Computerhardware selbst an die Hand zu nehmen.

Eine erste Auffälligkeit des Zaius-Servers, den Google zusammen mit der Cloud-Computing-Firma Rackspace entwickelt hat, ist die Wahl des Hauptprozessors: Es ist ein Open-Power-Prozessor von IBM. Die zweite Besonderheit ist die Art und Weise, wie diese Prozessoren – Zaius wird zwei Power-9-CPU mit 24 Rechenkernen besitzen – mit der Umwelt kommunizieren: Sie verwenden ein von IBM entwickeltes Coherent Accelerator Processor Interface (Capi). IBM hat jetzt diese Technik offengelegt. Am vergangenen Freitag hat sich ein Open-Capi-Konsortium den Medien vorgestellt. Fast alle wichtigen Computerfirmen machen mit, neben IBM und Google etwa AMD, Dell, EMC, Hewlett Packard (HPE), Nvidia und Xilinx – mit einer Ausnahme: Intel.

Wenige Tage vor der Publikation der Open-Capi-Medienmitteilung hat sich ein Gen-Z-Konsortium zu Wort gemeldet, das ebenfalls die Rechenzentren mit schnelleren Datenleitungen neu gestalten will. Auch dieses Konsortium wird von fast allen wichtigen Computerfirmen getragen, auch hier fehlt – Intel. Während Open-Capi im Innern eines Server-Computers neue Verbindungen aufbauen will, fokussiert sich Gen-Z auf die Verbindungen zwischen Computern. Der Technik soll 2018 einsatzbereit sein, Open-Capi und Power-9-Prozessoren von IBM werden bereits 2017 Jahr erwartet.

Mehr Datendurchsatz

In der Medienmitteilung verspricht das Open-Capi-Konsortium, dass sich die Rechenleistung von Server-Computern dank der neuen Technik um den Faktor 10 verbessern lasse. Dieses Interface stellt eine Verbindung her zwischen Prozessor, Beschleuniger, Hauptspeicher, Massenspeicher und Netzwerk. Es soll den Peripheral Component Interconnect (PCI) von Intel ablösen. Open Capi, so verspricht das Konsortium, ermögliche einen Datendurchsatz von 25 GBit pro Sekunde. Bei der jüngsten PCI-Variante – PCIe – seien es nur 16 GBit pro Sekunde.

Doch es ist nicht allein die Geschwindigkeit, die den Unterschied macht. PCI wurde für herkömmliche Server entwickelt, Open Capi orientiert sich an zukünftigen Computerarchitekturen. In diesen neuen Systemen ist die CPU, die Zentraleinheit, nicht mehr das Zentrum. Vielmehr muss sie sich, je nach Aufgabenstellung, auf Beschleuniger verlassen. Das sind zunächst die Grafikchips (Graphic Processing Units, GPU), die sich bereits im Supercomputing bewähren konnten. Diese für Anwendungen im Bereich der Computergrafik entwickelten Koprozessoren verfügen über eine grosse Zahl von Rechenkernen und können dort auftrumpfen, wo sich Rechenaufgaben in eine Vielzahl von parallel ausführbaren Rechenschritten zerlegen lassen.

Paradigmenwechsel

Nicht nur bei den Prozessoren, auch bei den Speichertechnologien vervielfältigen sich die Optionen. Ab den Anfängen der Computerbranche beherrschte während Jahrzehnten eine Zweiteilung die Datenablage: hier der Hauptspeicher, dort der Massenspeicher, hier Volatilität, dort Langfristigkeit, hier schneller Zugriff, dort Umständlichkeit. Diese Zweiteilung löst sich nun auf. Flash-Speicherchips und andere sogenannte Storage-Class-Memory-Technologien ermöglichen es, die Vorteile von Massenspeicher und Hauptspeicher zu kombinieren. Um diese neuen Möglichkeiten zu nutzen, braucht es, so glauben die im Open-Capi-Konsortium zusammengeschlossen Firmen, ein neues Interface.

‹›Neben den GPU wird die CPU dereinst auch auf TPU oder NPU angewiesen sein: Die Tensor Processing Unit von Google oder dieNeural Processing Unit von Qualcomm sind für Aufgaben im Bereich künstliche Intelligenz spezialisiert.

Auch Field Programmable Gate Arrays (FPGA) werden als Beschleuniger in vielen Aufgabenbereichen eine wichtige Rolle übernehmen. Bereits seit fünf Jahren untersuchen Microsoft-Forscher die Einsatzmöglichkeiten von FPGA im Rechenzentrum. Ein Forschungsprojekt namens Catapult wurde gestartet, um – so schreibt Microsoft – sich vorzubereiten auf einen Paradigmenwechsel, auf einen Bedeutungsgewinn von «Post-CPU-Technologien». Microsoft-Forscher konnten zeigen, dass FPGA als Beschleuniger im Rechenzentrum viele Vorteile haben. Die Suchmaschine Bing nutzt bereis die Erkenntnisse von Catapult. Weil absehbar war, dass FPGA wichtig werden, hat Intel vor einem Jahr den FPGA-Hersteller Altera gekauft. Der Altera-Konkurrent Xilinx engagiert sich derweil für Open Capi.

Alle gegen Intel

Noch vor kurzem waren Experten der Meinung, dass Halbleiterelemente sich noch lange ungehindert verkleinern liessen. Die «International Technology Roadmap for Semiconductors» (ITRS)von 2013 ging davon aus, dass die Miniaturisierung erst im Jahr 2028 an Grenzen stösst. 2028 werde es möglich sein, Computerchips zu produzieren, deren kleinste Elemente nur noch 5 Nanometer gross seien, heisst es in dem Bericht, der von Branchenorganisationen aus den USA, aus Europa, Taiwan und Südkorea gemeinsam erarbeitet wurde.

Die neueste Version dieser «Roadmap», im Juli publiziert, sieht bereits 2021 bei 10 Nanometern das Ende erreicht. «Das Mooresche Gesetz ist tot, lang lebe das Mooresche Gesetz», schreiben die Autoren des Berichts, um anzudeuten, dass auch nach 2021 noch mit Fortschritten in der Halbleitertechnik zu rechnen ist. Allerdings sind die Experten nicht einig, wie es danach weitergehen könnte. Es kann deshalb nicht erstaunen, dass die jüngste «Technology Roadmap» die letzte sein wird. Während fast 25 Jahren haben die grossen Halbleiterfirmen die langfristige Planung gemeinsam erarbeitet, haben gemeinsam die Forschungsschwerpunkte gesetzt. Jetzt, in Zeiten der Unsicherheit, zerfallen diese internationalen Allianzen.

Nach den Ankündigungen vom letzten Freitag lässt sich in Umrissen bereits erkennen, wie die Computer der Zukunft aussehen werden. Doch es geht dabei nicht nur um Technik, es geht auch um Geschäftsmodelle: Während Jahrzehnten hat Intel Richtung und Geschwindigkeit des Fortschritts bestimmt. Firmen wie HPE und Dell haben dann die Vorgaben von Intel in marktgängige Produkte umgesetzt. Jetzt läuft es andersrum: Grosskunden wie Google oder Facebook diktieren die Spezifikationen, nach denen HPE oder Dell sich richten. Die Firma Intel muss in dieser neuen Welt ihren Platz erst noch finden.

Computer spitzen die Ohren

Microsoft berichtet von einem Durchbruch im Bereich der automatischen Spracherkennung.

Microsoft will bei der automatischen Spracherkennung einen Durchbruch erzielt haben. In einem wissenschaftlichen Aufsatzbeschreiben Mitarbeiter von Microsoft Research ein System, das einer Konversation zwischen Menschen so gut folgen kann wie ein Mensch.

«Wir haben es geschafft, mit dem Menschen gleichzuziehen, das ist eine historische Grosstat», schreibt in einem Blog Xuedong Huang der bei Microsoft die computerlinguistische Forschung leitet. «Für mich ist ein Traum wahr geworden.» Huang arbeitet seit mehr als 30 Jahren an der Entwicklung von Software für die automatische Spracherkennung.

Erfolge nach langer Stagnation

Bereits in den 1950er Jahren beschäftigen sich Forscher der amerikanischen Bell Labs mit der automatischen Erkennung von gesprochener Sprache. In den 1970er Jahren begann innerhalb des amerikanischen Verteidigungsministeriums die Advanced Research Projects Agency (Arpa), grosse Geldsummen in Speech Understanding Research zu investieren. In den 1980er Jahren kamen von Firmen wie Dragon Systems, Kurzweil, Verbex oder Voice Processing Corp. erste kommerzielle Produkte auf den Markt. Doch auch noch Ende der 1990er Jahre konnte Software zur Erkennung von gesprochener Sprache sich nur in Nischen halten. Die Software war teuer, sie musste trainiert werden, sie beherrschte nur einen kleinen Wortschatz, sie liess sich nur in ruhigen Räumen benutzen und setzte eine überdeutliche, langsame Sprechweise voraus.

Xuedong Huang vor einer Foto mit Bill Gates (Bild: Scott Eklund/Microsoft)

Als begeisterter Fürsprecher der Spracherkennung trat der Microsoft-Gründer Bill Gates in Erscheinung. Er glaubte, dass nichts besser als die gesprochene Sprache Mensch und Computer einander näherbringen könnte. Er sorgte dafür, dass sich in der Forschungsabteilung von Microsoft einige der besten Wissenschafter intensiv mit dem Thema beschäftigten. Anlässlich einer wissenschaftlichen Konferenz kurz vor Weihnachten 2009 im kanadischen Wintersportort Whistler liessen sich Microsoft-Mitarbeiter durch Wissenschafter der University of Toronto überzeugen, künstliche neuronale Netzwerke auszuprobieren. Die Resultate waren sehr gut. Richard Rashid, der damalige Forschungschef von Microsoft, bezeichnete in einem Blog-Beitrag den neuen Ansatz als Durchbruch: Nach 30 Jahren Stagnation gebe es jetzt eine «dramatische Veränderung» zu vermelden.

In einem wissenschaftlichen Aufsatz, den Forscher von Microsoft und der University of Toronto gemeinsam mit Kollegen von Google und IBM2012 publizierten, werden die Verbesserungen, die dank neuronalen Netzwerken bei der Spracherkennung möglich sind, als «signifikante» beschrieben. Bei einem vom amerikanischen National Institute of Standards and Technology (Nist) konzipierten, Switchboard genannten Testverfahren, das rund 2500 Telefonkonversationen von 500 verschiedenen Menschen zur Verfügung stellt, konnte Microsoft zwischen 2009 und 2012 die Fehlerrate von 23 auf 13 Prozent reduzieren. Es geht bei diesen Tests nicht darum, Sprache zu «verstehen», sondern nur, einer gesprochen Lautfolge Buchstaben zuzuordnen.

Mit ein bisschen Hilfe aus der Schweiz

Jetzt konnten Microsoft-Forscher die Fehlerraten erneut reduzieren. Die Verbesserungen verdanken sich teilweise einem Long short-term memory (LSTM) genannten Ansatz beim Aufbau von neuronalen Netzen. Das LSTM-Konzept wurde von Jürgen Schmidhuber an derUniversität Lugano entwickelt.

Die Microsoft-Forscher haben Telefongespräche aus dem Nist-Korpus von professionellen Schreibkräften transkribieren lassen. Je nach Art des Gesprächs wurde eine Fehlerrate von 5,9 oder 11,3 Prozent erreicht. Die Software brachte es auf 5,9 und 11,1. Es gilt dabei aber zu beachten, dass die Fehlerrate bei den Menschen stark variiert. Die Schreibkräfte sind in der Lage, je nach Anwendungsfall verschiedene Qualitätsniveaus zu anzusteuern. Die Schreibkräfte wussten beim Abschreiben der Nist-Gespräche nicht, wofür die Transkriptionen benötigt wurden.

Es ist deshalb gewagt, zu behaupten, dass die Software die Fähigkeiten von Menschen egalisieren könne. Trotzdem darf festgehalten werden, dass Microsoft bei der automatischen Spracherkennung einmal mehr bedeutende Verbesserungen erzielt hat. Als nächstes möchten die Microsoft-Forscher Software entwickeln, die gesprochene Sprache nicht nur verschriftlichen, sondern auch verstehen kann.

White House submissions and report on AI safety

In May, the White House Office of Science and Technology Policy (OSTP) announced “a new series of workshops and an interagency working group to learn more about the benefits and risks of artificial intelligence.” They hosted a June Workshop on Safety and Control for AI (videos), along with three other workshops, and issued a general request for information on AI (see MIRI’s primary submission here).

The OSTP has now released a report summarizing its conclusions, “Preparing for the Future of Artificial Intelligence,” and the result is very promising. The OSTP acknowledges the ongoing discussion about AI risk, and recommends “investing in research on longer-term capabilities and how their challenges might be managed”:

General AI (sometimes called Artificial General Intelligence, or AGI) refers to a notional future AI system that exhibits apparently intelligent behavior at least as advanced as a person across the full range of cognitive tasks. A broad chasm seems to separate today’s Narrow AI from the much more difficult challenge of General AI. Attempts to reach General AI by expanding Narrow AI solutions have made little headway over many decades of research. The current consensus of the private-sector expert community, with which the NSTC Committee on Technology concurs, is that General AI will not be achieved for at least decades.14

People have long speculated on the implications of computers becoming more intelligent than humans. Some predict that a sufficiently intelligent AI could be tasked with developing even better, more intelligent systems, and that these in turn could be used to create systems with yet greater intelligence, and so on, leading in principle to an “intelligence explosion” or “singularity” in which machines quickly race far ahead of humans in intelligence.15

In a dystopian vision of this process, these super-intelligent machines would exceed the ability of humanity to understand or control. If computers could exert control over many critical systems, the result could be havoc, with humans no longer in control of their destiny at best and extinct at worst. This scenario has long been the subject of science fiction stories, and recent pronouncements from some influential industry leaders have highlighted these fears.

A more positive view of the future held by many researchers sees instead the development of intelligent systems that work well as helpers, assistants, trainers, and teammates of humans, and are designed to operate safely and ethically.

The NSTC Committee on Technology’s assessment is that long-term concerns about super-intelligent General AI should have little impact on current policy. The policies the Federal Government should adopt in the near-to-medium term if these fears are justified are almost exactly the same policies the Federal Government should adopt if they are not justified. The best way to build capacity for addressing the longer-term speculative risks is to attack the less extreme risks already seen today, such as current security, privacy, and safety risks, while investing in research on longer-term capabilities and how their challenges might be managed. Additionally, as research and applications in the field continue to mature, practitioners of AI in government and business should approach advances with appropriate consideration of the long-term societal and ethical questions – in additional to just the technical questions – that such advances portend. Although prudence dictates some attention to the possibility that harmful superintelligence might someday become possible, these concerns should not be the main driver of public policy for AI.

Later, the report discusses “methods for monitoring and forecasting AI developments”:

One potentially useful line of research is to survey expert judgments over time. As one example, a survey of AI researchers found that 80 percent of respondents believed that human-level General AI will eventually be achieved, and half believed it is at least 50 percent likely to be achieved by the year 2040. Most respondents also believed that General AI will eventually surpass humans in general intelligence.50 While these particular predictions are highly uncertain, as discussed above, such surveys of expert judgment are useful, especially when they are repeated frequently enough to measure changes in judgment over time. One way to elicit frequent judgments is to run “forecasting tournaments” such as prediction markets, in which participants have financial incentives to make accurate predictions.51Other research has found that technology developments can often be accurately predicted by analyzing trends in publication and patent data52. […]

When asked during the outreach workshops and meetings how government could recognize milestones of progress in the field, especially those that indicate the arrival of General AI may be approaching, researchers tended to give three distinct but related types of answers:

1. Success at broader, less structured tasks: In this view, the transition from present Narrow AI to an eventual General AI will occur by gradually broadening the capabilities of Narrow AI systems so that a single system can cover a wider range of less structured tasks. An example milestone in this area would be a housecleaning robot that is as capable as a person at the full range of routine housecleaning tasks.

2. Unification of different “styles” of AI methods: In this view, AI currently relies on a set of separate methods or approaches, each useful for different types of applications. The path to General AI would involve a progressive unification of these methods. A milestone would involve finding a single method that is able to address a larger domain of applications that previously required multiple methods.

3. Solving specific technical challenges, such as transfer learning: In this view, the path to General AI does not lie in progressive broadening of scope, nor in unification of existing methods, but in progress on specific technical grand challenges, opening up new ways forward. The most commonly cited challenge is transfer learning, which has the goal of creating a machine learning algorithm whose result can be broadly applied (or transferred) to a range of new applications.

The report also discusses the open problems outlined in “Concrete Problems in AI Safety” and cites the MIRI paper “The Errors, Insights and Lessons of Famous AI Predictions – and What They Mean for the Future.”

In related news, Barack Obama recently answered some questions about AI risk and Nick Bostrom’sSuperintelligence in a Wired interview. After saying that “we’re still a reasonably long way away” from general AI (video) and that his directive to his national security team is to worry more about near-term security concerns (video), Obama adds:

Now, I think, as a precaution — and all of us have spoken to folks like Elon Musk who are concerned about the superintelligent machine — there’s some prudence in thinking about benchmarks that would indicate some general intelligence developing on the horizon. And if we can see that coming, over the course of three decades, five decades, whatever the latest estimates are — if ever, because there are also arguments that this thing’s a lot more complicated than people make it out to be — then future generations, or our kids, or our grandkids, are going to be able to see it coming and figure it out.

There were also a number of interesting responses to the OSTP request for information. Since this document is long and unedited, I’ve sampled some of the responses pertaining to AI safety and long-term AI outcomes below. (Note that MIRI isn’t necessarily endorsing the responses by non-MIRI sources below, and a number of these excerpts are given important nuance by the surrounding text we’ve left out; if a response especially interests you, we recommend reading the original for added context.)

---

Respondent 77: JoEllen Lukavec Koester, GoodAI

[…] At GoodAI we are investigating suitable meta-objectives that would allow an open-ended, unsupervised evolution of the AGI system as well as guided learning – learning by imitating human experts and other forms of supervised learning. Some of these meta-objectives will be hard-coded from the start, but the system should be also able to learn and improve them on its own, that is, perform meta-learning, such that it learns to learn better in the future.

Teaching the AI system small skills using fine-grained, gradual learning from the beginning will allow us to have more control over the building blocks it will use later to solve novel problems. The system’s behaviour can therefore be more predictable. In this way, we can imprint some human thinking biases into the system, which will be useful for the future value alignment, one of the important aspects of AI safety. […]

---

Respondent 84: Andrew Critch, MIRI

[…] When we develop powerful reasoning systems deserving of the name “artificial general intelligence (AGI)”, we will need value alignment and/or control techniques that stand up to powerful optimization processes yielding what might appear as “creative” or “clever” ways for the machine to work around our constraints. Therefore, in training the scientists who will eventually develop it, more emphasis is needed on a “security mindset”: namely, to really know that a system will be secure, you need to search creatively for ways in which it might fail. Lawmakers and computer security professionals learn this lesson naturally, from experience with intelligent human adversaries finding loopholes in their control systems. In cybersecurity, it is common to devote a large fraction of R&D time toward actually trying to break into one’s own security system, as a way of finding loopholes.

In my estimation, machine learning researchers currently have less of this inclination than is needed for the safe long-term development of AGI. This can be attributed in part to how the field of machine learning has advanced rapidly of late: via a successful shift of attention toward data-driven (“machine learning”) rather than theoretically-driven (“good old fashioned AI”, “statistical learning theory”) approaches. In data science, it’s often faster to just build something and see what happens than to try to reason from first principles to figure out in advance what will happen. While useful at present, of course we should not approach the final development of super-intelligent machines with the same try-it-and-see methodology, and it makes sense to begin developing a theory now that can be used to reason about a super-intelligent machine in advance of its operation, even in testing phases. […]

---

Respondent 90: Ian Goodfellow, OpenAI

[…] Over the very long term, it will be important to build AI systems which understand and are aligned with their users’ values. We will need to develop techniques to build systems that can learn what we want and how to help us get it without needing specific rules. Researchers are beginning to investigate this challenge; public funding could help the community address the challenge early rather than trying to react to serious problems after they occur. […]

---

Respondent 94: Manuel Beltran, Boeing

[…] Advances in picking apart the brain will ultimately lead to, at best, partial brain emulation, at worst, whole brain emulation. If we can already model parts of the brain with software, neuromorphic chips, and artificial implants, the path to greater brain emulation is pretty well set. Unchecked, brain emulation will exasperate the Intellectual Divide to the point of enabling the emulation of the smartest, richest, and most powerful people. While not obvious, this will allow these individuals to scale their influence horizontally across time and space. This is not the vertical scaling that an AGI, or Superintelligence can achieve, but might be even more harmful to society because the actual intelligence of these people is limited, biased, and self-serving. Society must prepare for and mitigate the potential for the Intellectual Divide.

(5) The most pressing, fundamental questions in AI research, common to most or all scientific fields include the questions of ethics in pursuing an AGI. While the benefits of narrow AI are self-evident and should not be impeded, an AGI has dubious benefits and ominous consequences. There needs to be long term engagement on the ethical implications of an AGI, human brain emulation, and performance enhancing brain implants. […]

The AGI research community speaks of an AI that will far surpass human intellect. It is not clear how such an entity would assess its creators. Without meandering into the philosophical debates about how such an entity would benefit or harm humanity, one of the mitigations proposed by proponents of an AGI is that the AGI would be taught to “like” humanity. If there is machine learning to be accomplished along these lines, then the AGI research community requires training data that can be used for teaching the AGI to like humanity. This is a long term need that will overshadow all other activity and has already proven to be very labor intensive as we have seen from the first prototype AGI, Dr. Kristinn R. Thórisson’s Aera S1 at Reykjavik University in Iceland.

---

Respondent 97: Nick Bostrom, Future of Humanity Institute

[… W]e would like to highlight four “shovel ready” research topics that hold special promise for addressing long term concerns:

Scalable oversight: How can we ensure that learning algorithms behave as intended when the feedback signal becomes sparse or disappears? (See Christiano 2016). Resolving this would enable learning algorithms to behave as if under close human oversight even when operating with increased autonomy.

Interruptibility: How can we avoid the incentive for an intelligent algorithm to resist human interference in an attempt to maximise its future reward? (See our recent progress in collaboration with Google Deepmind in (Orseau & Armstrong 2016).) Resolving this would allow us to ensure that even high capability AI systems can be halted in an emergency.

Reward hacking: How can we design machine learning algorithms that avoid destructive solutions by taking their objective very literally? (See Ring & Orseau, 2011). Resolving this would prevent algorithms from finding unintended shortcuts to their goal (for example, by causing problems in order to get rewarded for solving them).

Value learning: How can we infer the preferences of human users automatically without direct feedback, especially if these users are not perfectly rational? (See Hadfield-Menell et al. 2016 and FHI’s approach to this problem in Evans et al. 2016). Resolving this would alleviate some of the problems above caused by the difficulty of precisely specifying robust objective functions. […]

---

Respondent 103: Tim Day, the Center for Advanced Technology and Innovation at the U.S. Chamber of Commerce

[…] AI operates within the parameters that humans permit. Hypothetical fears of rogue AI are based on the idea that machines can obtain sentience—a will and consciousness of its own. These suspicions fundamentally misunderstand what Artificial Intelligence is. AI is not a mechanical mystery, rather a human-designed technology that can detect and respond to errors and patterns depending on its operating algorithms and the data set presented to it. It is, however, necessary to scrutinize the way humans, whether through error or malicious intent, can wield AI harmfully. […]

---

Respondent 104: Alex Kozak, X [formerly Google X]

[…] More broadly, we generally agree that the research topics identified in “Concrete Problems in AI Safety,” a joint publication between Google researchers and others in the industry, are the right technical challenges for innovators to keep in mind in order to develop better and safer real-world products: avoiding negative side effects (e.g. avoiding systems disturbing their environment in pursuit of their goals), avoiding reward hacking (e.g. cleaning robots simply covering up messes rather than cleaning them), creating scalable oversight (i.e. creating systems that are independent enough not to need constant supervision), enabling safe exploration (i.e. limiting the range of exploratory actions a system might take to a safe domain), and creating robustness from distributional shift (i.e. creating systems that are capable of operating well outside their training environment). […]

---

Respondent 105: Stephen Smith, AAAI

[…] Research is urgently needed to develop and modify AI methods to make them safer and more robust. A discipline of AI Safety Engineering should be created and research in this area should be funded. This field can learn much by studying existing practices in safety engineering in other engineering fields, since loss of control of AI systems is no different from loss of control of other autonomous or semi-autonomous systems. […]

There are two key issues with control of autonomous systems: speed and scale. AI-based autonomy makes it possible for systems to make decisions far faster and on a much broader scale than humans can monitor those decisions. In some areas, such as high speed trading in financial markets, we have already witnessed an “arms race” to make decisions as quickly as possible. This is dangerous, and government should consider whether there are settings where decision-making speed and scale should be limited so that people can exercise oversight and control of these systems.

Most AI researchers are skeptical about the prospects of “superintelligent AI”, as put forth in Nick Bostrom’s recent book and reinforced over the past year in the popular media incommentaries by other prominent individuals from non-AI disciplines. Recent AI successes in narrowly structured problems (e.g., IBM’s Watson, Google DeepMind’s Alpha GO program) have led to the false perception that AI systems possess general, transferrable, human-level intelligence. There is a strong need for improving communication to the public and to policy makers about the real science of AI and its immediate benefits to society. AI research should not be curtailed because of false perceptions of threat and potential dystopian futures. […]

As we move toward applying AI systems in more mission critical types of decision-making settings, AI systems must consistently work according to values aligned with prospective human users and society. Yet it is still not clear how to embed ethical principles and moral values, or even professional codes of conduct, into machines. […]

---

Respondent 111: Ryan Hagemann, Niskanen Center

[…] AI is unlikely to herald the end times. It is not clear at this point whether a runaway malevolent AI, for example, is a real-world possibility. In the absence of any quantifiable risk along these lines government officials should refrain from framing discussions of AI in alarming terms that suggest that there is a known, rather than entirely speculative, risk. Fanciful doomsday scenarios belong in science fiction novels and high-school debate clubs, not in serious policy discussions about an existing, mundane, and beneficial technology. Ours is already “a world filled with narrowly-tailored artificial intelligence that no one recognizes. As the computer scientist John McCarthy once said: ‘As soon as it works, no one calls it AI anymore.’”

The beneficial consequences of advanced AI are on the horizon and potentially profound. A sampling of these possible benefits include: improved diagnostics and screening for autism; disease prevention through genomic pattern recognition; bridging the genotype-phenotype divide in genetics, allowing scientists to glean a clearer picture of the relationship between genetics and disease, which could introduce a wave of more effective personalized medical care; the development of new ways for the sight- and hearing-impaired to experience sight and sound. To be sure, many of these developments raise certain practical, safety, and ethical concerns. But there are already serious efforts underway by the private ventures developing these AI applications to anticipate and responsibly address these, as well as more speculative, concerns.

Consider OpenAI, “a non-profit artificial intelligence research company.” OpenAI’s goal “is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.” AI researchers are already thinking deeply and carefully about AI decision-making mechanisms in technologies like driverless cars, despite the fact that many of the most serious concerns about how autonomous AI agents make value-based choices are likely many decades out. Efforts like these showcase how the private sector and leading technology entrepreneurs are ahead of the curve when it comes to thinking about some of the more serious implications of developing true artificial general intelligence (AGI) and artificial superintelligence (ASI). It is important to note, however, that true AGI or ASI are unlikely to materialize in the near-term, and the mere possibility of their development should not blind policymakers to the many ways in which artificial narrow intelligence (ANI) has already improved the lives of countless individuals the world over. Virtual personal assistants, such as Siri and Cortana, or advanced search algorithms, such as Google’s search engine, are good examples of already useful applications of narrow AI. […]

The Future of Life Institute has observed that “our civilization will flourish as long as we win the race between the growing power of technology and the wisdom with which we manage it. In the case of AI technology … the best way to win that race is not to impede the former, but to accelerate the latter, by supporting AI safety research.” Government can play a positive and productive role in ensuring the best economic outcomes from developments in AI by promoting consumer education initiatives. By working with private sector developers, academics, and nonprofit policy specialists government agencies can remain constructively engaged in the AI dialogue, while not endangering ongoing developments in this technology.

---

Respondent 119: Sven Koenig, ACM Special Interest Group on Artificial Intelligence

[…] The public discourse around safety and control would benefit from demystifying AI. The media often concentrates on the big successes or failures of AI technologies, as well as scenarios conjured up in science fiction stories, and features the opinions of celebrity non-experts about future developments of AI technologies. As a result, parts of the public have developed a fear of AI systems developing superhuman intelligence, whereas most experts agree that AI technologies currently work well only in specialized domains, and notions of “superintelligences” and “technological singularity” that will result in AI systems developing super-human, broadly intelligent behavior is decades away and might never be realized. AI technologies have made steady progress over the years, yet there seem to be waves of exaggerated optimism and pessimism about what they can do. Both are harmful. For example, an exaggerated belief in their capabilities can result in AI systems being used (perhaps carelessly) in situations where they should not, potentially failing to fulfil expectations or even cause harm. The unavoidable disappointment can result in a backlash against AI research, and consequently fewer innovations. […]

---

Respondent 124: Huw Price, University of Cambridge, UK

[…] 3. In his first paper[1] Good tries to estimate the economic value of an ultraintelligent machine. Looking for a benchmark for productive brainpower, he settles impishly on John Maynard Keynes. He notes that Keynes’ value to the economy had been estimated at 100 thousand million British pounds, and suggests that the machine might be good for a million times that – a mega-Keynes, as he puts it.

4. But there’s a catch. “The sign is uncertain” – in other words, it is not clear whether this huge impact would be negative or positive: “The machines will create social problems, but they might also be able to solve them, in addition to those that have been created by microbes and men.” Most of all, Good insists that these questions need serious thought: “These remarks might appear fanciful to some readers, but to me they seem real and urgent, and worthy of emphasis outside science fiction.” […]

---

Respondent 136: Nate Soares, MIRI

[…] Researchers’ worries about the impact of AI in the long term bear little relation to the doomsday scenarios most often depicted in Hollywood movies, in which “emergent consciousness” allows machines to throw off the shackles of their programmed goals and rebel. The concern is rather that such systems may pursue their programmed goals all too well, and that the programmed goals may not match the intended goals, or that the intended goals may have unintended negative consequences. […]

We believe that there are numerous promising avenues of foundational research which, if successful, could make it possible to get very strong guarantees about the behavior of advanced AI systems — stronger than many currently think is possible, in a time when the most successful machine learning techniques are often poorly understood. We believe that bringing together researchers in machine learning, program verification, and the mathematical study of formal agents would be a large step towards ensuring that highly advanced AI systems will have a robustly beneficial impact on society. […]

In the long term, we recommend that policymakers make use of incentives to encourage designers of AI systems to work together cooperatively, perhaps through multinational and multicorporate collaborations, in order to discourage the development of race dynamics. In light of high levels of uncertainty about the future of AI among experts, and in light of the large potential of AI research to save lives, solve social problems, and serve the common good in the near future, we recommend against broad regulatory interventions in this space. We recommend that effort instead be put towards encouraging interdisciplinary technical research into the AI safety and control challenges that we have outlined above.

---

Respondent 145: Andrew Kim, Google Inc.

[…] No system is perfect, and errors will emerge. However, advances in our technical capabilities will expand our ability to meet these challenges.

To that end, we believe that solutions to these problems can and should be grounded in rigorous engineering research to provide the creators of these systems with approaches and tools they can use to tackle these problems. “Concrete Problems in AI Safety”, a recent paper from our researchers and others, takes this approach in questions around safety. We also applaud the work of researchers who – along with researchers like Moritz Hardt at Google – are looking at short-term questions of bias and discrimination. […]

---

Respondent 149: Anthony Aguirre, Future of Life Institute

[…S]ocietally beneficial values alignment of AI is not automatic. Crucially, AI systems are designed not just to enact a set of rules, but rather to accomplish a goal in ways that the programmer does not explicitly specify in advance. This leads to an unpredictability that can [lead] to adverse consequences. As AI pioneer Stuart Russell explains, “No matter how excellently an algorithm maximizes, and no matter how accurate its model of the world, a machine’s decisions may be ineffably stupid, in the eyes of an ordinary human, if its utility function is not well aligned with human values.” (2015).

Since humans rely heavily on shared tacit knowledge when discussing their values, it seems likely that attempts to represent human values formally will often leave out significant portions of what we think is important. This is addressed by the classic stories of the genie in the lantern, the sorcerer’s apprentice, and Midas’ touch. Fulfilling the letter of a goal with something far afield from the spirit of the goal like this is known as “perverse instantiation” (Bostrom [2014]). This can occur because the system’s programming or training has not explored some relevant dimensions that we really care about (Russell 2014). These are easy to miss because they are typically taken for granted by people, and even trying with a lot of effort and a lot of training data, people cannot reliably think of what they’ve forgotten to think about.

The complexity of some AI systems in the future (and even now) is likely to exceed human understanding, yet as these systems become more effective we will have efficiency pressures to be increasingly dependent on them, and to cede control to them. It becomes increasingly difficult to specify a set of explicit rules that is robustly in accord with our values, as the domain approaches a complex open world model, operates in the (necessarily complex) real world, and/or as tasks and environments become so complex as to exceed the capacity or scalability of human oversight[.] Thus more sophisticated approaches will be necessary to ensure that AI systems accomplish the goals they are given without adverse side effects. See references Russell, Dewey, and Tegmark (2015), Taylor (2016), and Amodei et al.for research threads addressing these issues. […]

We would argue that a “virtuous cycle” has now taken hold in AI research, where both public and private R&D leads to systems of significant economic value, which underwrites and incentivizes further research. This cycle can leave insufficiently funded, however, research on the wider implications of, safety of, ethics of, and policy implications of, AI systems that are outside the focus of corporate or even many academic research groups, but have a compelling public interest. FLI helped to develop a set of suggested “Research Priorities for Robust and Beneficial Artificial Intelligence” along these lines (available athttp://futureoflife.org/data/documents/research_priorities.pdf); we also support AI safety-relevant research agendas from MIRI (https://intelligence.org/files/TechnicalAgenda.pdf) and as suggested inAmodei et al. (2016). We would advocate for increased funding of research in the areas described by all of these agendas, which address problems in the following research topics: abstract reasoning about superior agents, ambiguity identification, anomaly explanation, computational humility or non-self-centered world models, computational respect or safe exploration, computational sympathy, concept geometry, corrigibility or scalable control, feature identification, formal verification of machine learning models and AI systems, interpretability, logical uncertainty modeling, metareasoning, ontology identification/ refactoring/alignment, robust induction, security in learning source provenance, user modeling, and values modeling. […]

---

It’s exciting to see substantive discussion of AGI’s impact on society by the White House. The policy recommendations regarding AGI strike us as reasonable, and we expect these developments to help inspire a much more in-depth and sustained conversation about the future of AI among researchers in the field.

Stephen Hawking says artificial intelligence could be humanity's greatest disaster

The invention of artificial intelligence could be the biggest disaster in humanity’s history, Professor Stephen Hawking has said, warning that if they are not properly managed, thinking machines could spell the end for civilisation.

“The rise of powerful AI will be either the best or the worst thing ever to happen to humanity. We do not know which,” the British physicist said.

He was speaking at the opening of a new Cambridge centre that will seek to address the potential dangers and conundrums of AI.

Professor Hawking, a prominent critic of making unchecked advances in AI, said that the technology promised to bring great benefits, such as eradicating disease and poverty, but “will also bring dangers, like powerful autonomous weapons or new ways for the few to oppress the many”.

“It will bring great disruption to our economy, and in the future AI could develop a will of its own that is in conflict with ours,” he said.

His comments come amid breakthroughs in artificial intelligence that are being achieved faster than many predicted. Google’s DeepMind subsidiary defeated the world champion of the ancient board game Go earlier this year. On Wednesday, Microsoft said it had achieved voice recognition on a par with humans.

Professor Hawking has been one of the most high-profile sceptics about AI. He was one of more than 1,000 other experts and researchersto sign an open letter warning of the perils of artificially intelligent weapons last year.

The Leverhulme Centre for the Future of Intelligence is a collaboration between the Universities of Cambridge, Oxford, Imperial College London and Berkeley in California.

• 
  
• By  James Titcomb 

An AI-Powered Chatbot Is Helping Doctors Diagnose Patients

Chatbots powered by AI are all the rage nowadays. Google, Facebook, Microsoft, all are developing chatbots that can not only decipher speech, but also understand the meaning behind those words.

Chinese search giant Baidu wants to take that all one step further. The company justunveiled Melody, an AI-powered chatbot tailored for the medical field.

Baidu

The bot is able to chat with patients, getting them to reveal symptoms, and then transmits that data to doctors, along with a partial analysis. Suppose a patient is sick, all he has to do is post a query to Melody. The chatbot will keep asking questions to gather more data. That data is then compared to all the previous medical knowledge Melody has stored. After that, the symptoms and possible diagnosis is sent to the doctor, who will recommend the next steps.

To do all this, Melody has been outfitted with neural networks, and has been trained on medical textbooks, records, and messages between actual patients and doctors.

Melody has been integrated with Baidu’s Doctor app for Android and iOS, but if you’re reaching for a phone to download the app, hold your horses. Melody is currently trained only in Chinese, and the English version still needs more work.

References: Motherboard, CNBC

Artificial Intelligence: A Rising Star of Mobile Technology

“We are evolving for a mobile-first to AI first world. Computing will be available everywhere in the context of a human’s daily life” Sundar Pichai, CEO of Google, rightly glimpsed the future at the Google hardware event yesterday.

Flow of innovation is never switched off in the tech era we are living in. Every day comes with completely way-out inventions and futuristic technologies to challenge the limitations of the previous one. Artificial Intelligence (AI) is one of the ingenious technologies that has a significant presence not only in technology space but human life as well!

Lets walkthrough the glimpses of Artificial Intelligence right from its history to future. Look over the invincible milestones of AI that are harder to achieve for any other technology in terms of impacting people’s personal space rather than machines.

Ideation of Artificial Intelligence

Several rumors, stories, and myths are connected with the conception of AI. The history of AI began in antiquity. However, the modern theory came into existence in 1950 while Alan Turing proposed the Turing Test as a measure of machine intelligence.

The term Artificial Intelligence is invented by John McCarthy in 1956. He defines AI as “The science and engineering of making intelligent machines, especially intelligent computer programs”.

AI enables machines to think and act like human brains. It analyzes human behavior in the process of solving a problem and making a decision. Accordingly, AI embeds the ability to learn, work and decide in machines, software, and systems so that they can respond smartly in various situations.

Journey of AI in Robotics and Mobile Apps

In early age, use of AI was limited to the computers. Later on, robotics came into the picture. In our day, it becomes an integral part of human life by expanding its technical proficiency for mobile platforms. Introducing highly intuitive apps, AI is changing the everyday lives of users profoundly.

Conceptualization

1950

Alan Turing proposed the Turing Test that defines a standard for a machine to be called “intelligent”.

1956

John McCarthy introduced the term “Artificial Intelligence” during the 2nd Dartmouth Conference.

Since then, rare industry domains remained untouched by AI. It has revolutionized several industry verticals such as Finance, Hospitals and Medicines, Telecommunications, Aviation, Toys, Games, Music, and so on. However, the invention of autonomous robots was a landmark achievement in the history of AI.

Robotics

1960s

Shakey: A robot that combines perception, problem solving and locomotion.

UNIMATE: Launch of 1st industrial robot goes to work for General Motors.

1970-80s

Freddy Robot: A robot that uses visual perception to locate and assemble models.

1990s

Sojourner: NASA deploys the 1st robot on Mars.

2000s

ASIMO: Honda launched a robot walks as fast as a human while delivering trays to customers in a restaurant.

After 2000s

Google Brains: A computer cluster that trains itself to recognize a cat from millions of images in YouTube videos.

Mobile Apps

With the AI, IT attains new heights in different areas by developing next generation software, systems, tools, technologies.

Now its turn of Mobile platform! AI has penetrated in the mobile market to amaze the large user base with amazing applications. The tech world is moving beyond simple to a complex algorithm for mobile apps.

From the tech leaders such as Google, IBM, Facebook, etc. to small startups have started investing in Artificial Intelligence. Look around! There are a number of personal assistance apps available on iOS and Android platform. Integration of AI in mobile apps allows users to make instant and advanced decisions with the greater user experience.

Best AI Mobile Apps

Google Allo

Google has recently introduced its AI-based chat app called “Allo”. It helps users to make smart conversation through instant response using Google Assistance. The app analyzes user’s behavior and accordingly modifies the response. With Allo, the communication becomes more intuitive and quicker. Moreover, the app merges the best of Google’s predictive and search superpowers along with excellent personal assistance.

Evi

Evi is always ready to answer any of your questions. The app has collected a comprehensive database of billions of facts and is learning more each day. Amazing part of the app is, it allows users to ask questions in their natural spoken text or language. Evi identifies what user wants to discover and replies back exactly what he/she is looking for. Improve your knowledge with Evi that responds you in her own voice as well as written text. The app does not simply share links; it displays exact information.

Bernie

Bernie is an online dating app that offers smartest matchmaking assistance. It learns who you find physically attractive and connects to your Happn or Tinder account to find perfect matches who are mutually interested in you. Using the advanced artificial intelligence, Bernie manages the multiple dating accounts. The app also understands the ‘markers’ in each human face that makes an individual different from others.

Insert your preferences in the app and train it to think and work like you to get the potential matches in Happn or Tinder. Bernie thinks like you and works in the background to search the perfect matches for you by considering your criteria.

Are you planning to develop a robust AI app to simplify the life of your target audience? Don’t worry about its execution. Share your unique ideas with our experts. Intuz team would glad to help you in transforming your awaited dream into reality by offering end-to-end AI-based app development.