How to apply deep learning to real world problems

The rise of artificial intelligence in recent years is grounded in the success of deep learning. Three major drivers caused the breakthrough of (deep) neural networks: the availability of huge amounts of training data, powerful computational infrastructure, and advances in academia. Thereby deep learning systems start to outperform not only classical methods, but also human benchmarks in various tasks like image classification or face recognition. This creates the potential for many disruptive new businesses leveraging deep learning to solve real-world problems.

At Berlin-based Merantix, we work on these new business cases in various industries (currently automotive, health, financial and advertising).

It easier than ever before to train a neural network. However, it is rarely the case that you can just take code from a tutorial and directly make it work for your application. Interestingly, many of the most important tweaks are barely discussed in the academic literature but at the same time critical to make your product work.

Therefore I thought it would be helpful for other people who plan to use deep learning in their business to understand some of these tweaks and tricks. In this blog post I want to share three key learnings, which helped us at Merantix when applying deep learning to real-world problems:

• Learning I: The Value Of Pre-Training
• Learning II: Caveats Of Real-World Label Distributions
• Learning III: Understanding Black Box Models

LEARNING I: THE VALUE OF PRE-TRAINING

In the academic world of machine learning, there is little focus on obtaining datasets. Instead, it is even the opposite: in order to compare deep learning techniques with other approaches and ensure that one method outperforms others, the standard procedure is to measure the performance on a standard dataset with the same evaluation procedure. However, in real-world scenarios, it is less about showing that your new algorithm squeezes out an extra 1% in performance compared to another method. Instead it is about building a robust system which solves the required task with sufficient accuracy. As for all machine learning systems, this requires labeled training from which the algorithm can learn from.

• By Rasmus Rothe, founder of AI research lab Merantix, Berlin

Facebook using artificial intelligence to combat terrorist propaganda

Facebookhas spoken for the first time about the artificial intelligence programmes it uses to deter and remove terrorist propaganda online after the platform was criticised for not doing enough to tackle extremism. 

The social media giant also revealed it is employing 3,000 extra people this year in order to trawl through posts and remove those that break the law or the sites' community guidelines.

It also plans to boost it's "counter-speech" efforts, to encourage influential voices to condemn and call-out terrorism online to prevent people from being radicalised. 

In a landmark post titled "hard questions", Monika Bickert, Director of Global Policy Management, and Brian Fishman, Counterterrorism Policy Manager explained Facebook has been developing artificial intelligence to detect terror videos and messages before they are posted live and preventing them from appearing on the site. 

The pair state: "In the wake of recent terror attacks, people have questioned the role of tech companies in fighting terrorism online. We want to answer those questions head on."

Explaining how Facebook works to stop extremist content being posted the post continues: "We are currently focusing our most cutting edge techniques to combat terrorist content about ISIS, Al Qaeda and their affiliates, and we expect to expand to other terrorist organizations in due course.

"When someone tries to upload a terrorist photo or video, our systems look for whether the image matches a known terrorism photo or video. This means that if we previously removed a propaganda video from ISIS, we can work to prevent other accounts from uploading the same video to our site.

"We have also recently started to experiment with using AI to understand text that might be advocating for terrorism." Facebook also detailed how it is working with other platforms, clamping down on accounts being re-activated by people who have previously been banned from the site and identifying and removing clusters of terror supporters online.   

The social media platform, which is used by billions of people around the world, also explained it employs thousands of people to check posts and has a dedicated counter-terrorism team.

"Our Community Operations teams around the world — which we are growing by 3,000 people over the next year — work 24 hours a day and in dozens of languages to review these reports and determine the context. This can be incredibly difficult work, and we support these reviewers with onsite counseling and resiliency training," it said. 

Facebook came under pressure from ministers after a number of recent terror attacks for failing to do more tackle and remove extremist posts. 

Amber Rudd, the Home Secretary, said earlier this year: "Each attack confirms again the role that the internet is playing in serving as a conduit, inciting and inspiring ­violence, and spreading extremist ­ideology of all kinds,” she writes.

“But we can’t tackle it by ourselves … We need [social media companies] to take a more proactive and leading role in tackling the terrorist abuse of their platforms.”

Artificial Intelligence and the Future of Work: An Interview With Moshe Vardi

by Tucker Davey

“The future of work is now,” says Moshe Vardi. “The impact of technology on labor has become clearer and clearer by the day.”

Machines have already automated millions of routine, working-class jobs in manufacturing. And now, AI is learning to automate non-routine jobs in transportation and logistics, legal writing, financial services, administrative support, and healthcare.

Vardi, a computer science professor at Rice University, recognizes this trend and argues that AI poses a unique threat to human labor.

Initiating a Policy Response

From the Luddite movement to the rise of the Internet, people have worried that advancing technology would destroy jobs. Yet despite painful adjustment periods during these changes, new jobs replaced old ones, and most workers found employment. But humans have never competed with machines that can outperform them in almost anything. AI threatens to do this, and many economists worry that society won’t be able to adapt.

“What people are now realizing is that this formula that technology destroys jobs and creates jobs, even if it’s basically true, it’s too simplistic,” Vardi explains.

The relationship between technology and labor is more complex: Will technology create enough jobs to replace those it destroys? Will it create them fast enough? And for workers whose skills are no longer needed – how will they keep up?

To address these questions and consider policy responses, Vardi will hold a summit in Washington, D.C. on December 12, 2017. The summit will address six current issues within technology and labor: education and training, community impact, job polarization, contingent labor, shared prosperity, and economic concentration.

Education and training

A 2013 computerization study found that 47% of American workers held jobs at high risk of automation in the next decade or two. If this happens, technology must create roughly 100 million jobs.

As the labor market changes, schools must teach students skills for future jobs, while at-risk workers need accessible training for new opportunities. Truck drivers won’t transition easily to website design and coding jobs without proper training, for example. Vardi expects that adapting to and training for new jobs will become more challenging as AI automates a greater variety of tasks. 

Community impact

Manufacturing jobs are concentrated in specific regions where employers keep local economies afloat. Over the last thirty years, the loss of 8 million manufacturing jobs has crippled Rust Belt regions in the U.S. – both economically and culturally.

Today, the fifteen million jobs that involve operating a vehicle are concentrated in certain regions as well. Drivers occupy up to 9% of jobs in the Bronx and Queens districts of New York City, up to 7% of jobs in select Southern California and Southern Texas districts, and over 4% in Wyoming and Idaho. Automation could quickly assume the majority of these jobs, devastating the communities that rely on them.

Job polarization

“One in five working class men between ages 25 to 54 without college education are not working,” Vardi explains. “Typically, when we see these numbers, we hear about some country in some horrible economic crisis like Greece. This is really what’s happening in working class America.”

Employment is currently growing in high-income cognitive jobs and low-income service jobs, such as elderly assistance and fast-food service, which computers cannot automate yet. But technology ishollowing out the economy by automating middle-skill, working-class jobs first.

Many manufacturing jobs pay $25 per hour with benefits, but these jobs aren’t easy to come by. Since 2000, when millions of these jobs disappeared, displaced workers have either left the labor force or accepted service jobs that often pay $12 per hour, without benefits.

Truck driving, the most common job in over half of US states, may see a similar fate.

Source: IPUMS-CPS/ University of Minnesota Credit: Quoctrung Bui/NPR

Contingent labor

Increasingly, communications technology allows firms to save money by hiring freelancers and independent contractors instead of permanent workers. This has created the Gig Economy – a labor market characterized by short-term contracts and flexible hours at the cost of unstable jobs with fewer benefits. By some estimates, in 2016, one in three workers were employed in the Gig Economy, but not all by choice. Policymakers must ensure that this new labor market supports its workers.

Shared prosperity

Automation has decoupled job creation from economic growth, allowing the economy to grow while employment and income shrink, thus increasing inequality. Vardi worries that AI will accelerate these trends. He argues that policies encouraging economic growth must also support economic mobility for the middle class.

Economic concentration

Technology creates a “winner-takes-all” environment, where second best can hardly survive. Bing search is quite similar to Google search, but Google is much more popular than Bing. And do Facebook or Amazon have any legitimate competitors?

Startups and smaller companies struggle to compete with these giants because of data. Having more users allows companies to collect more data, which machine-learning systems then analyze to help companies improve. Vardi thinks that this feedback loop will give big companies long-term market power.

Moreover, Vardi argues that these companies create relatively few jobs. In 1990, Detroit’s three largest companies were valued at $65 billion with 1.2 million workers. In 2016, Silicon Valley’s three largest companies were valued at $1.5 trillion but with only 190,000 workers.

Work and society

Vardi primarily studies current job automation, but he also worries that AI could eventually leave most humans unemployed. He explains, “The hope is that we’ll continue to create jobs for the vast majority of people. But if the situation arises that this is less and less the case, then we need to rethink: how do we make sure that everybody can make a living?”

Vardi also anticipates that high unemployment could lead to violence or even uprisings. He refers to Andrew McAfee’s closing statement at the 2017 Asilomar AI Conference, where McAfee said, “If the current trends continue, the people will rise up before the machines do.”

This article is part of a Future of Life series on the AI safety research grants, which were funded by generous donations from Elon Musk and the Open Philanthropy Project.

Artificial Intelligence gains momentum with news media

Artificial Intelligence (AI), a computer’s ability to replicate the human thought process and solve problems, is hard at work in today’s news media. And for those not already leveraging AI, the time is now. In the new report, Artificial Intelligence: News Media’s Next Urgent Investment, Martha L. Stone, CEO of the World Newsmedia Network, in association with the INMA, explores how AI is being applied in today’s news industry.

Stone explains that AI has three main forms of behavior: natural language processing; predictive analytics; and machine learning/neural networks. Publishers can apply all three forms to address a wide range of news challenges.

Natural Language Processing

The first form describes the way in which computers understand the natural language process (NLP). It allows for automatic creation of articles (“robo-journalism”). Both the Associated Press and the BBC use Wordsmith, an NLP automated database, to create huge volumes of stories within seconds.

Natural language processing enables speech recognition and is used in devices like Apple’s Siri, Amazon’s Alexa, or Google’s Home. This type of AI process allows CNN, The New York Times, The Washington Post, the Chicago Tribune, Quartz, the Huffington Post, and others to offer “flash” new briefings. Users signal these audio devices to inform them of the day’s news.

Media companies use sentiment analysis, a subset of NLP, to identify opinions in social media and blogs. The sentiment analysis sorts through comments about people, brands, etc. by analyzing both positive and negative words used in online discussions.

NLP also powers the recommendation engines used by many news publishers. Story recommendations help increase both traffic and user engagement. Chat apps and bots can also be used to drive traffic. However, they are especially good at repetitive tasks such as answering specific questions and offering data alerts. A successful example is The Washington Post’s Facebook Messenger feed bot. Users ask the bot questions and responds to overall news inquiries by suggesting links to other relatable news stories.

Predictive Analysis

The second form of AI, predictive analytics, allows analysts to predict trends and behaviors based on a subset of data. Predictive models are often used to target advertising, subscription, or membership offerings. The analytics identify consumer patterns and project the potential the outcome. The Financial Times uses predictive analytics to correlate revenue to content usage and conversion rate to engagement.

Schibsted, one of the biggest news media publishers in Europe, uses predicted analytics to identity the gender of their users. Using predictive analytics, Schibsted’s accuracy of gender prediction grew from 15% – 20% to 100%. Demographic assignments are extremely important in serving personalize content and advertising. Likewise, The Weather Channel uses weather trends to help predict the optimal time for advertising. A cold front or snow storm approaching is a perfect time to advertise hot breakfast foods or batteries.

Machine Learning

The third form of AI is machine learning, which is essentially computers learning to make decisions. Computers identify patterns and apply new logic based on the results. Algorithms allow publishers to make predictions on data, including consumer usage patterns and personal preferences. The New York Times uses machine learning to help identify content for readers. Pinterest uses machine learning to identify relevant user-generated content for their users.

Personalization is a great way to utilize machine learning. These practices include recommendations of text and video, location-specific content, segment-based personalization (identifying users or specific products or a demographic, etc.) and newsletter recommendations. Of course, there are concerns that personalization bubbles leave little room for new content discovery.

Artificial Intelligence offers support and efficiency in real-time using sentiment and machine based audience analytics. It presents the news media with a way to connect with consumers and provide relevancy. Importantly, the use AI technologies has direct and positive impact on revenue and customer engagement.

Artificial Intelligence: Open Source and Standards Bodies Drive Opportunities

Artificial intelligence (AI) and machine learning (ML) skillsets are now becoming a crucial way for technology-focused workers to differentiate themselves from the pack. Moreover, from Elon Musk’s OpenAI organization to Google’s sweeping new open AI initiatives announced at the recent Google I/O conference, investment in AI is driving many new opportunities. For technologists who straddle the arenas of open source and AI, opportunities are looking particularly promising.

At Google’s recent developer conference, the company introduced a project called AutoML from its Google Brain artificial intelligence research group.  It is designed to help automate many of the toughest aspects of designing machine learning and AI tools. Google is looking to grow the number of developers able to leverage machine learning by reducing the expertise required and is aiming to drum up community involvement.

As The Verge recently noted, the company’s AI initiatives “attract talent to Google and help make the company’s in-house software the standard for machine learning.” The bottom line is that AI and machine learning talent is very in-demand talent.

Organized Responses to the Promise of AI

Powerful consortiums are taking shape to help drive the future of open artificial intelligence. Partnership on AI is one of the most notable.  According to its founders: “We are at an inflection point in the development and application of AI technologies. The upswing in AI competencies, fueled by data, computation, and advances in algorithms for machine learning, perception, planning, and natural language, promise great value to people and society… We are excited about the prospect of coming together to collaborate on addressing concerns, rough edges, and rising challenges around AI, as well as to work together to pursue the grand opportunities and possibilities of the long-term dream of mastering the computational science of intelligence. It is our intention that the Partnership on AI will be collaborative, constructive, and work openly with all.”

More than 20 companies have joined Partnership on AI. The organizations range from Facebook to Intel to Salesforce and SAP. Many of these companies are actively contributing open source AI and machine learning projects to the community.

Meanwhile, Elon Musk’s OpenAI is creating new types of opportunities, including the release of open source tools. “We seek to broadcast our work to the world as papers,blog posts, software, talks, and tutorials,” the organization reports, and OpenAI is also hiring. 

Most recently, OpenAI has delivered an open toolkit for training robots via virtual reality. It has also open sourced a toolkit called Universe, which is middleware that can help AI agents solve arbitrary tasks and learn as they solve problems.

Building Out Your Skillset

So how can you gain skills that can become valuable as AI and machine learning advance? Coursera offers a popular class focused on machine learning, taught by a Stanford University expert. Udacity also offers free courses on AI, and has a notable course on deep learningdeveloped with one of the principal scientists at Google. The course shows you how to train and optimize basic neural networks, convolutional neural networks, and long short term memory networks. It also introduces Google’s open source AI tools.

One of the more popular online courses on AI is found on the edX platform. The course is offered in conjunction with Columbia University and taught by a Columbia professor. The course covers building intelligent agents, open source AI tools, machine learning and more. Check out more free courses in this area, rounded up by the Hackearth blog.

There are also many good online tutorials focused on AI and machine learning. Here, you can find many of them for TensorFlow, Google’s flexible and popular open source framework that can be applied to image recognition tasks, neural networking, and more. You can also find many tutorials for H2O.ai’s popular AI and machine learning tools here.

To learn more about the promise of machine learning and artificial intelligence, watch a video featuring David Meyer, Chairman of the Board at OpenDaylight, a Collaborative Project at The Linux Foundation.

Are you interested in how organizations are bootstrapping their own open source programs internally? You can learn more in the Fundamentals of Professional Open Source Management training course from The Linux Foundation.Download a sample chapter now!

High-speed light-based systems could replace supercomputers for certain ‘deep learning’ calculations

Low power requirements for photons (instead of electrons) may make deep learning more practical in future self-driving cars and mobile consumer devices

June 14, 2017

(a) Optical micrograph of an experimentally fabricated on-chip optical interference unit; the physical region where the optical neural network program exists is highlighted in gray. A programmable nanophotonic processor uses a field-programmable gate array (similar to an FPGA integrated circuit ) — an array of interconnected waveguides, allowing the light beams to be modified as needed for a specific deep-learning matrix computation. (b) Schematic illustration of the optical neural network program, which performs matrix multiplication and amplification fully optically. (credit: Yichen Shen et al./Nature Photonics)

A team of researchers at MIT and elsewhere has developed a new approach to deep learning systems — using light instead of electricity, which they say could vastly improve the speed and efficiency of certain deep-learning computations.

Deep-learning systems are based on artificial neural networks that mimic the way the brain learns from an accumulation of examples. They can enable technologies such as face- and voice-recognition software, or scour vast amounts of medical data to find patterns that could be useful diagnostically, for example.

But the computations these systems carry out are highly complex and demanding, even for supercomputers. Traditional computer architectures are not very efficient for calculations needed for neural-network tasks that involve repeated multiplications of matrices (arrays of numbers). These can be computationally intensive for conventional CPUs or even GPUs.

Programmable nanophotonic processor

Instead, the new approach uses an optical device that the researchers call a “programmable nanophotonic processor.” Multiple light beams are directed in such a way that their waves interact with each other, producing interference patternsthat “compute” the intended operation.

The optical chips using this architecture could, in principle, carry out dense matrix multiplications (the most power-hungry and time-consuming part in AI algorithms) for learning tasks much faster, compared to conventional electronic chips. The researchers expect a computational speed enhancement of at least two orders of magnitude over the state-of-the-art and three orders of magnitude in power efficiency.

“This chip, once you tune it, can carry out matrix multiplication with, in principle, zero energy, almost instantly,” says Marin Soljacic, one of the MIT researchers on the team.

To demonstrate the concept, the team set the programmable nanophotonic processor to implement a neural network that recognizes four basic vowel sounds. Even with the prototype system, they were able to achieve a 77 percent accuracy level, compared to about 90 percent for conventional systems. There are “no substantial obstacles” to scaling up the system for greater accuracy, according to Soljacic.

The team says is will still take a lot more time and effort to make this system useful. However, once the system is scaled up and fully functioning, the low-power system should find many uses, especially for situations where power is limited, such as in self-driving cars, drones, and mobile consumer devices. Other uses include signal processing for data transmission and computer centers.

The research was published Monday (June 12, 2017) in a paper in the journal Nature Photonics (open-access version available on arXiv).

The team also included researchers at Elenion Technologies of New York and the Université de Sherbrooke in Quebec. The work was supported by the U.S. Army Research Office through the Institute for Soldier Nanotechnologies, the National Science Foundation, and the Air Force Office of Scientific Research.

---

Abstract of Deep learning with coherent nanophotonic circuits

Artificial neural networks are computational network models inspired by signal processing in the brain. These models have dramatically improved performance for many machine-learning tasks, including speech and image recognition. However, today’s computing hardware is inefficient at implementing neural networks, in large part because much of it was designed for von Neumann computing schemes. Significant effort has been made towards developing electronic architectures tuned to implement artificial neural networks that exhibit improved computational speed and accuracy. Here, we propose a new architecture for a fully optical neural network that, in principle, could offer an enhancement in computational speed and power efficiency over state-of-the-art electronics for conventional inference tasks. We experimentally demonstrate the essential part of the concept using a programmable nanophotonic processor featuring a cascaded array of 56 programmable Mach–Zehnder interferometers in a silicon photonic integrated circuit and show its utility for vowel recognition.

Researchers decipher how faces are encoded in the brain

Only 205 neurons required per face; findings also have artificial intelligence applications

This figure shows eight different real faces that were presented to a monkey, together with reconstructions made by analyzing electrical activity from 205 neurons recorded while the monkey was viewing the faces. (credit: Doris Tsao)

In a paper published (open access) June 1 in the journal Cell, researchers report that they have cracked the code for facial identity in the primate brain.

“We’ve discovered that this code is extremely simple,” says senior author Doris Tsao, a professor of biology and biological engineering at the California Institute of Technology and senior author. “We can now reconstruct a face that a monkey is seeing by monitoring the electrical activity of only 205 neurons in the monkey’s brain. One can imagine applications in forensics where one could reconstruct the face of a criminal by analyzing a witness’s brain activity.”

The researchers previously identified the six “face patches” — general areas of the primate and human brain that are responsible for identifying faces — all located in the inferior temporal (IT) cortex. They also found that these areas are packed with specific nerve cells that fire action potentials much more strongly when seeing faces than when seeing other objects. They called these neurons “face cells.”

Previously, some experts in the field believed that each face cell (a.k.a. “grandmother cell“) in the brain represents a specific face, but this presented a paradox, says Tsao, who is also a Howard Hughes Medical Institute investigator. “You could potentially recognize 6 billion people, but you don’t have 6 billion face cells in the IT cortex. There had to be some other solution.”

Instead, they found that rather than representing a specific identity, each face cell represents a specific axis within a multidimensional space, which they call the “face space.” These axes can combine in different ways to create every possible face. In other words, there is no “Jennifer Aniston” neuron.

The clinching piece of evidence: the researchers could create a large set of faces that looked extremely different, but which all caused the cell to fire in exactly the same way. “This was completely shocking to us — we had always thought face cells were more complex. But it turns out each face cell is just measuring distance along a single axis of face space, and is blind to other features,” Tsao says.

AI applications

“The way the brain processes this kind of information doesn’t have to be a black box,” Chang explains. “Although there are many steps of computations between the image we see and the responses of face cells, the code of these face cells turned out to be quite simple once we found the proper axes. This work suggests that other objects could be encoded with similarly simple coordinate systems.”

The research also has artificial intelligence applications. “This could inspire new machine learning algorithms for recognizing faces,” Tsao adds. “In addition, our approach could be used to figure out how units in deep networks encode other things, such as objects and sentences.”

This research was supported by the National Institutes of Health, the Howard Hughes Medical Institute, the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech, and the Swartz Foundation.

* The researchers started by creating a 50-dimensional space that could represent all faces. They assigned 25 dimensions to the shape–such as the distance between eyes or the width of the hairline–and 25 dimensions to nonshape-related appearance features, such as skin tone and texture.

Using macaque monkeys as a model system, the researchers inserted electrodes into the brains that could record individual signals from single face cells within the face patches. They found that each face cell fired in proportion to the projection of a face onto a single axis in the 50-dimensional face space. Knowing these axes, the researchers then developed an algorithm that could decode additional faces from neural responses.

In other words, they could now show the monkey an arbitrary new face, and recreate the face that the monkey was seeing from electrical activity of face cells in the animal’s brain. When placed side by side, the photos that the monkeys were shown and the faces that were recreated using the algorithm were nearly identical. Face cells from only two of the face patches–106 cells in one patch and 99 cells in another–were enough to reconstruct the faces. “People always say a picture is worth a thousand words,” Tsao says. “But I like to say that a picture of a face is worth about 200 neurons.”

Caltech | Researchers decipher the enigma of how faces are encoded

---

Abstract of The Code for Facial Identity in the Primate Brain

Primates recognize complex objects such as faces with remarkable speed and reliability. Here, we reveal the brain’s code for facial identity. Experiments in macaques demonstrate an extraordinarily simple transformation between faces and responses of cells in face patches. By formatting faces as points in a high-dimensional linear space, we discovered that each face cell’s firing rate is proportional to the projection of an incoming face stimulus onto a single axis in this space, allowing a face cell ensemble to encode the location of any face in the space. Using this code, we could precisely decode faces from neural population responses and predict neural firing rates to faces. Furthermore, this code disavows the long-standing assumption that face cells encode specific facial identities, confirmed by engineering faces with drastically different appearance that elicited identical responses in single face cells. Our work suggests that other objects could be encoded by analogous metric coordinate systems.