João Graça
The President’s Council of Advisors on Science and Technology predicts that U.S. companies will spend upward of $100 billion on AI R&D per year by 2025. Much of this spending today is done by six tech companies — Microsoft, Google, Amazon, IBM, Facebook and Apple, according to a recent study from CSET at Georgetown University. But what if you’re a startup whose product relies on AI at its core?
Can early-stage companies support a research-based workflow? At a startup or scaleup, the focus is often more on concrete product development than research. For obvious reasons, companies want to make things that matter to their customers, investors and stakeholders. Ideally, there’s a way to do both.
Before investing in staffing an AI research lab, consider this advice to determine whether you’re ready to get started.
Compile the right research team
Assuming it’s your organization’s priority to do innovative AI research, the first step is to hire one or two researchers. At Unbabel, we did this early by hiring Ph.D.s and getting started quickly with research for a product that hadn’t been developed yet. Some researchers will build from scratch and others will take your data and try to find a pre-existing model that fits your needs.
From there, you’ll need to hire research engineers or machine learning operations professionals. Research is only a small part of using AI in production. Research engineers will then release your research into production, monitor your model’s results and refine the model if it stops predicting well (or otherwise is not operating as planned). Often they’ll use automation to simplify monitoring and deployment procedures as opposed to doing everything manually.
None of this falls within the scope of a research scientist — they’re most used to working with the data sets and models in training. That said, researchers and engineers will need to work together in a continuous feedback loop to refine and retrain models based on actual performance in inference.
Choose the problems you want to solve
The CSET research cited above shows that 85% of AI labs in North America and Europe do some form of basic AI research, and less than 15% focus on development. The rest of the world is different: A majority of labs in other countries, such as India and Israel, focus on development.
What’s the difference? While Google’s X division may have the capital to focus on moonshots, most startups can only invest in innovation that provides them a competitive advantage or improves their product.
Applied research is often used to solve a specific problem or to aid in the development of a product. This type of research often uses existing, open-source machine learning models with a company’s data sets. This practice may be applied to cut costs, increase efficiencies, measure the success of a product or develop features. Applied research becomes easier with APIs — you can effectively take what comes out of academia, apply it to your product and measure its success based on business KPIs.
Another form of development-focused research is called fundamental research. This is done when you need to invent something for a problem that hasn’t been solved yet or build a data set that doesn’t exist. Our research team focused on solving existing quality estimation problems by inventing a completely new metric for machine translation quality. We open-sourced this development to give back to the community and prevent others from having to reinvent the wheel.
While this effort wasn’t tied to specific product features, it has helped both our team and our customers understand if our product is working as intended and how it can improve. Having a clearly defined business benefit helps with both executive- and board-level alignment.
Use research as a talent funnel
Research shows that demand for AI is back up after a temporary COVID-19 slowdown, accompanied by a major talent shortage — nearly 40% of companies cite a lack of technical expertise as a top reason for not deploying AI technologies.
If you’re starting an AI research lab, it can be helpful to recruit someone who has a dual role within an AI-focused academic institution and within the company. It also helps to be close to a good academic center for access to talent. For example, if you’re in proximity to a major university, you can work with master’s and Ph.D. students and offer summer internships that could convert into full-time opportunities. This opens up access to talent within the university and conversely helps give access to careers students may not have considered.
This practice is common among some of the world’s leading corporate AI research labs. For example, Facebook recruited top AI researcher Yann LeCun eight years ago, and he remains both VP and chief AI scientist at the company and a professor at New York University. Collaborations between the company and the university are common, including a recent development to improve the speed of MRI scans.
Having this type of relationship can help build better bridges and avoid the perception of a “brain drain” of university talent. Remember that your research should ultimately support the company, the university, and its students. As seen in the Facebook example above, opportunities to do joint research projects are immense. For example, we are working on a multilingual conversational agent project with Carnegie Mellon University in Portugal and other academic partners. This type of collaboration benefits everyone involved.
Set expectations on the investment and time to ROI
Most startups and scaleups are used to fast-paced resourcing and R&D cycles designed specifically for product development. However, there is no rushing academic-style research. It’s a rigorous, methodical process that takes time to be successful. Does your culture support the idea of bringing long-term research to reach state of-the-art outcomes? Or are your goals too aggressive?
It can help to lay out the case for a research function to the board early in the process. If the leadership team isn’t fully convinced of the value, you can start small by hiring only one researcher and trying to outsource some of the research to a community (there are existing communities like Kaggle, or in our case, there’s a robust natural language processing community that supports projects with universities).
If you’re working with a university, you can apply for a joint research grant, support a master’s or Ph.D. student, or work on an ongoing project over the course of a year that pairs students with employees at the company.
Investmentwise, an AI lab should be considered in the same category as R&D. It can be easier to get the leadership team or board to consider what they’re willing to invest if you can justify whether your project worked and had an impact on the organization. Some companies are more prone to research because of their DNA, but others will face more of an uphill battle explaining why they’re proposing research and aligning expectations with reality.
Start by really understanding your product as well as which KPIs would be changed if a piece of research were to work. You can use that process to prioritize different research. To prove your concepts, make sure to create datasets that are good proxies to the product KPIs at hand. That way, if anyone asks how to map the value of research back to a product KPI, you’ll have all of the documentation in place to do so.
The power of an AI lab
Setting up and sustaining an AI lab can be costly and can easily become a distraction. For example, one of the biggest cost centers is computing power (such as on-premise or cloud-based GPUs). Anticipating these costs as well as seeking ongoing alignment between research and product is key.
Research has to explicitly include a roadmap and process for incorporating research outcomes into the product. A research arm will fail if expectations are not aligned between the product team and the lab.
With the proper alignment and commitment, you will attract top researchers who will feel fulfilled in their career path, and you can make major advancements you’d never expect from developing a product alone.
These simple metrics will tell you if your startup is ready to scale
Comment