Newsweek Japan: The democratization of artificial-intelligence tools moves forward at Recruit

Translated from an article by Tsuruaki Yukawa

We are now in an age when all of a company’s employees must know how to use AI tools. Alon Halevy, who quit Google to become head of Recruit’s AI research center, was surprised to find that Recruit had not made any AI tools available to its employees. Will Japanese tech companies be able to catch up?

Recently, Microsoft adopted the catchphrase “AI for everyone,” referring to a democratization of AI. Microsoft and other US tech companies have begun to believe — and act on — the premise that just as the productivity of businesses improved remarkably when everyone became able to use a PC, so too every businessman of the future will need to be able to use AI. In Japan, Recruit has begun deploying AI-related tools for its ordinary employees, and positive results are already coming in, such as the fact that in half a year, more than 4,000 AI predictive models have been completed.

The three levels of AI implementation

It is said that there are three levels of AI implementation at businesses. Level 1 is when a business hires one AI engineer. Level 2 is when the company sets up an organizational unit specializing in AI. And Level 3 is when every single employee is capable of using AI freely. If the person in charge of something, someone who is fully aware of the needs on the ground, is able to make use of AI freely, then the productivity of the company should take a jump upward.

Top-level US technology companies Google and Facebook are said to have already reached Level 3. It is reported that half of the in-house engineers at these companies are assigned to the development and maintenance of in-house AI infrastructure and tools.

Last fall, I reported on the AI research center that Recruit had set up in Silicon Valley. Alon Halevy, who quit Google and became head of Recruit’s AI research center, was initially surprised to find that Recruit was not making any AI tools available to its employees.

Related article: What it means that a former Google research superstar is taking over as head of AI research at Recruit

I also reported that the AI research center was hurrying to develop and deploy tools for data preprocessing and data integration.

DataRobot, an incredibly powerful tool

Recruit has also moved forward quickly in deploying DataRobot, a tool that is capable of developing predictive models in an almost completely automatic fashion. Predictive models are formulas that statistically process past data in order to discern trends and predict the future.

For example, it is possible to construct a model that, by statistically processing weather data and daily ice-cream sales, can predict by how much sales of ice cream will increase if the temperature increases by a certain number of degrees. Similarly, by combining the titles of the books that a person purchases with the person’s attribute data, a predictive model is able to predict what sorts of books should be recommended to what sorts of persons in order to increase book sales.

Up until recently, constructing such predictive models was the work of specialists in statistical processing called “data scientists.” But with the DataRobot tool, if you simply provide the data, the tool will develop the predictive model automatically almost entirely on its own, making for a tool of incredible power.

Related article: How far along is AI? Predictions regarding 10 promising AI-related technologies and the maturity of their markets

In the US, a growing number of patients who have been discharged from hospitals are experiencing relapses of their symptoms and being hospitalized again. This is said to be turning into a societal problem, since hospitals are being accused of possibly kicking out patients who are not completely cured. I was permitted to watch while DataRobot created a model to predict re-hospitalization rates by examining which measures, taken in the case of patients with certain test results, lead to lower re-hospitalization rates.

The data that was input consisted of patient attributes such as age, sex, type of illness, and symptoms, as well as data on what treatment they had received at the hospital, and records on readmissions within 30 days, etc.

The program is simple to operate. Just drag and drop the data files with a mouse, and DataRobot will automatically do the statistical processing and recognize any data patterns there may be among the rehospitalized patients. For instance, it can tell whether, for elderly male patients with lifestyle diseases, X% of them end up being readmitted after only having been treated with medication. In other words, the program can predict that if patients with those same attributes are only given medication, then X% of them are likely to be readmitted to the hospital.

The program tries out this sort of statistical processing using a variety of different statistical methods. The speed with which the predictive model can be created is influenced by the number of cloud-based servers that are used. The demonstration that I observed used 20 servers, so I watched as calculations took place, algorithm by algorithm for a variety of statistical algorithms, of how precise each algorithm would be. And it was very impressive to watch. I was amazed by how DataRobot carried out in just a few minutes calculations that looked like they would take months for a human expert to process.

More than 4,000 predictive models in half a year

Recruit has made this incredibly powerful tool available to all its employees. According to Ko Ishiyama, former executive manager of the RIT Promotion Office at Recruit, although employees were given only simple training when the program was deployed, in the roughly half a year since its deployment, 4,855 predictive models have been created. And he says that approximately 80% of these models have been developed by ordinary employees.

In addition, the AI research center in Silicon Valley joined with a development team at Recruit headquarters to jointly build a data-discovery system to search for where data is located — and what kinds of data exist — within companies in the Recruit group, completing the task in just half a year. Ishiyama says that they ended up with a system that is like the systems owned by companies such as Google and Apple, systems that are the most advanced in the world. He adds that with this system, data that formerly might take months to obtain by means of complicated procedures can now be obtained in minutes, and that the number of people using data has ballooned from several hundred to several thousand. In short, the democratization of AI has taken a sudden jump forward.

Related article: Artificial intelligence exacerbates economic disparities and poverty

Using pre-hire data to predict post-hire performance

Ishiyama says that the predictive models created by ordinary employees contain many things that data specialists would not have thought of. For instance, during job-hunting season, people send the company their résumés and the results of an aptitude test called “SPI.” SPI, which is used by a lot of companies, measures factors such as drive, motivation, emotions, ability to relate socially, and language skills.

Ishiyama says that when these pre-hire data were input into DataRobot along with data that indicated the employee’s performance after hiring, a model was completed that can predict with considerable accuracy how a person will perform after hiring based on pre-hire data.

Although it is difficult to choose what data to use as an indicator of post-hire performance, the ordinary employees who created this model had the novel idea of choosing performance-based pay. Ishiyama says that once the model had finished learning, when additional pre-hire data was loaded into it, the model guessed the post-hire salary with roughly 80% accuracy. For businesses, being able to figure out, before hiring, which personnel will perform well is a matter of vital importance.

Big companies receive large quantities of job applications, and during the document-screening phase they throw away the documents of applicants who are graduating from less-than-famous universities. This practice is never made public, and is no more than a rumor, but I have heard this same rumor from a number of different HR people. I think it is because this practice remains secretly in place that society continues to have the bad habit of placing an emphasis on educational background. If it becomes possible to detect and hire, with greater accuracy, personnel who are likely to perform well after being hired, regardless of the university that they graduated from, then this also ought to change education in Japan.

AI is no more than a tool, but it is an incredibly powerful tool

This predictive model is also useful when a company wants to revise its strategy. The company simply needs to hire personnel whose data are similar to the pre-hire data of employees who are making major contributions under the new strategy. These sorts of predictive models are being generated at Recruit one after another, and this is likely to lead to predictive models being combined, and to the pursuit of new data in order to revise strategies. The democratization of AI is likely to become a driving force that changes companies and society.

When personal computers first began to be widely used, one often heard people say that the personal computer was no more than a tool. And when the Internet first began to be widely used, one often heard people say that the Internet was no more than a tool. There were even people who detested personal computers and the Internet. Nowadays, one often hears people say that AI is no more than a tool, and that AI is irrelevant for their company.

It is quite true that AI is no more than a tool. Simply deploying AI will not improve a company’s results. Nevertheless, tools are important. If one’s rival begins using jet fighters, one can hardly afford to keep on fighting using bamboo spears. We are coming to a time when, just as the businessman of today is expected to be able to use a personal computer, the businessman will henceforth be expected to be able to use AI. Whether we like it or not, what cannot be denied is that the democratization of AI is moving forward at an incredible pace.