Armen Aghajanyan: Armenia needs more GPUs

Armen Aghajanyan is a research scientist at Meta AI. He collaborates with the YerevaNN team, leads student research groups and works on projects developing the field of Machine Learning in Armenia, including the creation of relevant infrastructure for it.

Itel.am talked to him about the AI revolution-caused challenges, work at Meta AI and the Armenian tech ecosystem development initiatives.

Tell us about your background. How did you get into the tech industry?

I grew up in the tech family. When I was young, maybe 12 years old, I started programming. Then at the age of 16 I had some Stanford lectures about using Machine Learning to drive a car and that was before anyone was looking at this as specific as it may be.

I actually got interested in online courses available when I was in high school. I wrote a publication, and I got really lucky, Misha Bilenko, a guy from Microsoft reached out to me and we talked and he offered me an internship in Microsoft. The internship was great.  

After high school I started working at a startup, and part time I was studying at Washington University. Then I got a nice offer from a team that was working with language models before Open AI existed, this was pretty much earlier. In 2018 I joined Meta (Facebook at that time) in a peer research team. During these years we had many publications, I also met a lot of outstanding professors, worked with them and attended many conferences. It is going to be almost 5 years since I have worked in the AI research team here in Meta.

You’re focused on Meta AI projects. Tell us about your projects that you're working on these days.

When ChatGPT came out, around that time we knew that Meta (Facebook) was following it all as well, but behind the space. We skipped the text only, but focused on multiple modalities at once (text, image, audio, video, code, etc.). We started to lay the foundation around the world for this.

And we presented CM3Leon (pronounced “Chameleon”), a retrieval-augmented, token based, decoder-only multi-modal language model capable of generating and infilling both text and images. It sets a new state-of-the-art for text-to-image generation. It can be fine-tuned to perform quite complex image + text tasks, including visual question answering, text-guided image editing, image generation given a human pose.

What do you know about the Armenian tech ecosystem? Tell us about the collaborations with YerevaNN and with the Armenian students, engineers.

I met Hrant Khatchatrian, co-founder of YerevaNN in ACL 2019 conference that was held in Florence. We had just a brief talk and exchanged our contacts.  

In that time for me Hrant was the central point of all Armenian tech, he is the one who knows almost everyone in the tech community. Through him I started to get to know and understand what technical capabilities, systems there are in Armenia. In general I was surprised more positively.

There are strong engineers, but there are not a lot of them. It is nice to see the internal drive they have. There is a balance between men and women, which is also good. But the problem is the lack of opportunities realistically and this makes growth hard.

Talking specifically about Machine Learning and the fundamentals on it, capacities in Armenia that needs to be done. YerevaNN is the first and fantastic step, but there are very significant problems. The most fundamental is the lack of computers, GPUs. One of my interns has more GPUs than all of Armenia, believe me. I think this is strategic misjudgement.

So we have a couple of plans to raise a capital to at least try to build out this type of computing infrastructure in Armenia.

Since the launch of ChatGPT, we’ve started to work in this very high compute regime. So the question is whether you want YereavNN to stay in a very academic niche group or to become one of the top ten lab in the world in the next five years.

As I said, Armenia in general lacks machine learning engineers. Part of this is that the field is new and there are no formal programs in institutes, and there is not an actual degree that you can get.

There is a growing interest in AI / ML and especially Generative AI. Obviously it is transforming industries and possibilities. How can Armenia win this race and become competitive and offer competitive global products?

From my understanding I think it does transform as the Internet did when it first came online. We are going to start seeing two things happen. One thing is that certain fields will explode very quickly in terms of capabilities. For example, programmers now have a model that can help them program etc. Biology, robotics, defense and the other industries will fundamentally be changed. It will change the way that we’re living.

Another thing is the nontrivial impact it has on the economy. One of the scary things which will happen in the next couple of years is that even GPT4 has the capability to remove part of the workforce.

The worst thing that could happen to the tech scene in Armenia is that paradoxically it can grow quickly to become a service based economy like in case of India. You can earn money, live a good life doing a service job, but this is actually a very negative thing, because Armenia doesn’t have enough population to support a service based economy and right now it doesn't have the capacity for deep tech.

So the key focus for anyone should be to try to get into deep tech in one way or another: whether it is like machine learning or biotech. The deep tech is going to be a fundamental separator of Armenia with any other country. The way like GPT4 is going most of the service stuff is going to disappear very soon and we’re going to have a population that does not have a job because there is not actual deep talent, so this is one of the biggest worries for the tech community.

Now we can't say that Armenia should be the part of this game but it can’t be far behind.

How to counter those threats? Is there a need for regulations?

Regulations are not going to work. Recently Israel and Japan said that they have some AI laws and it says you can train your model on whatever you want and that is very smart. Because if I’m going to start an AI company, I will go to a place that defends me from possible loss. But strong regulations are not necessarily going to work if there is an existential threat to the economy, the regulations will be left in. The progress can’t be stopped, historically we are adapting to the progress. The benefit of Armenia with such a small population is that we can adapt quickly in the same way as Israel has done.

As I said we have plans to raise capital to build computing infrastructures, GPUs in Armenia. It has been an ongoing process for a couple of months since this February. It will start with local capacity projects, training people. It will give a chance to Armenian specialists to understand how to train large language models like ChatGPT. Generative AI is important and we can speculatively teach people in Armenia and when capacity comes we can start doing something cool.

Nune Grigoryan talked to Armen Aghajanyan