Welcome
I am a research scientist with broad interests in machine learning and artificial intelligence. My work on large scale learning and stochastic gradient algorithms has received attention in the recent years. I am also known for the DjVu document compression system. I joined Facebook AI Research in March 2015.
Use the sidebar to navigate this site.
News
Joining the Flatiron Institute
After eleven exciting years at FAIR. I joined the Flatiron Institute at the Simons Foundation.
What is the most important letter in “AI”?
Much of the agitation these days is on letter “A” of “Artificial”. Because an artificial intelligence is one that he can own or sell, the Schumpeterian entrepreneur sees a chance to rewire our society, to “found a private kingdom”, “the nearest approach to medieval lordship possible to modern man”, to “prove oneself superior to others”, and also to experience the “joy of creating” and of “exercising one’s energy and ingenuity.” (Schumpeter, The Theory of Economic Development, 1934). We read this in the news every day.
However, the real prize is with letter “I” of “Intelligence”. We cannot overstate the role of intelligence in the development of mankind over many millennia. Yet we remain unable to precisely describe how intelligence works, or even define it without loopy concepts. However, we now have machines that display some level of intelligence. We do not understand precisely how they work, but we know much more about the operation of these machines than we know about our cognitive processes. We can use them as models. Any salient discussion of the nature of artificial intelligence is also a discussion of the nature of intelligence, and eventually a discussion that impacts all forms of human activity or knowledge. Such an outcome has a much bigger impact and lasts much longer than any business venture
My conclusion was that understanding AI is more important than just building it.
The Flatiron Institute focuses on computational methods for both mathematics and empirical sciences/ Therefore it provides a double opportunity to better understand AI, first because AI is an instance of computational mathematics, second because we can observe how AI will transform computational method for empirical sciences and maybe draw useful conclusions.
The Fiction Machine
This SIAM News article by Bernhard and I summarizes the argument of Borges and AI and make additional points about the curse of alignment and the machine within the machine.
Two lessons from ICLR 2025
Machine learning conferences nowadays are too large for my enjoyment. I made the trip to Singapore for two posters and a talk in the associative memory workshop. I spent my time listening to the morning keynotes, walking briskly in the poster room, and catching up with friends and colleagues from both industry and academia.
Fifteen minutes before the morning keynote
On my way back, I met Kyunghyun Cho in the airport. We had a drink over what we had learned. Cho always has great insights; he has invented attention mechanisms; he dines with Korean stars. Therefore, I know what I must do when he tells me “You should tweet that!”
Two weakly related points.
Borges and AI
Léon Bottou and Bernhard Schölkopf https://arxiv.org/abs/2310.01425
We started this work mid-2022. AI was already turning into a mainstream topic. Both as a scientist and a member of the society, I was troubled by the ambient confusion between the actual AI technology and the AI of our dreams or nightmares. We seem unable to grasp this technology and its impact without referring to an AI mythology that maybe starts with Homer's golden maiden and was popularized by modern science fiction.
Therefore we decided to instead interpret the advances of AI using a very different lens: the fiction of Jorge Luis Borges, whose subtly ironical stories illuminate how language works and relates to reality. This intellectual exercise turned into a very fruitful exercise, one that has reframed our outlook on AI:
- It clarifies the relation between AI and language models, or fiction machines.
- It explains how humans perceive these technologies, searching for vindications that comfort our preconceptions, vainly attempting to purify the fiction machine, or trusting this modern Pythia over our own reason.
- It also explains how fiction machines should be seen as tools to construct theories for both real and imagined worlds. The ability to create fictional stories —so-called “hallucinations”— is crucially important. For instance, to understand a factual story, say a historical battle, we must be able to imagine how different circumstance or decisions would have changed the events. This provides a new meaning to Pat Winston's claim about the centrality of story making and story telling.
- And finally, it shows the importance how understanding the world through the right story. For instance, understanding the weather patterns through the mood of the Gods only went so far. Yet it took centuries to readjust.