AI models are being adapted to work with the building blocks of biology. At the JP Morgan Healthcare Conference, Nvidia and its collaborators presented two projects: Nucleotide Transformer, which learns from genomic sequences, and ProT-VAE, which aims to generate proteins with specific functions.
The projects approach biology from different directions. One seeks to apply a single model across genomic tasks; the other uses machine learning to help design new proteins. Their results suggest ways AI could support research, while leaving questions about how broadly the methods will work.
A language model approach to genomic sequences
Nucleotide Transformer was developed by InstaDeep, recently acquired by Biontech, the Technical University of Munich, and Nvidia. The team trained different model sizes using data from up to 174 billion nucleotides from different species on Nvidia’s Cambridge-1 supercomputer.
The approach echoes large language models: train models at different scales on a large body of data, using substantial computing power. In tests across 19 benchmarks, the Nucleotide Transformer achieved performance equivalent to or better than models built specifically for the tasks in 15 of them.
That result matters because a model that can generalize across tasks may be useful beyond a single narrowly defined analysis. The article says researchers expect the transformer could help translate DNA sequences into RNA and proteins. That is a prospective use, rather than a claim that the model already performs every step of that process in practice.
InstaDeep CEO Karim Beguir described the results as evidence that genomics foundation models can generalize across tasks. He compared the development to adaptable foundation models in natural language processing, while pointing to the challenges of drug discovery and human health as areas where the work may be applied.
ProT-VAE aims to generate proteins
ProT-VAE addresses a different task. Whereas models such as AlphaFold and ESMFold predict protein structures from sequences, ProT-VAE is designed to infer functions from sequences and generate new proteins intended to perform a specified function.
Evozyne developed the model using Nvidia’s BioNeMo platform. Its architecture places a variational autoencoder network between a pre-trained protein transformer encoder and decoder. The VAE is trained for a particular protein family, while the system can draw on representations learned by the ProtT5 transformer from millions of protein sequences.
The goal is to narrow the search through an enormous range of possible proteins. The source notes that the number of proteins that can be made from naturally occurring amino acids exceeds the number of protons in the visible universe. Machine learning guided protein engineering offers a way to focus that search around a desired function.
A PAH protein variant offers an early test
To evaluate ProT-VAE, the researchers engineered, among other things, a variant of the human PAH protein. The source explains that mutations of the PAH gene can limit its activity and cause metabolic disorders, including effects on mental development and epilepsy.
According to the researchers, the model designed a variant with 51 mutations, 85 percent sequence similarity, and 2.5-fold improved function. These figures describe the reported protein engineering result; they do not establish a treatment or show that the variant has been used clinically.
The reported development timeline also changed: a process that had taken months to years was reduced to weeks with ProT-VAE. That could make protein engineering work faster, but the source frames the model as a platform for future machine learning guided directed evolution campaigns and the design of synthetic proteins with “super-natural” function.
What the advances could mean
Together, the projects illustrate two possible roles for generative AI in biology. A genomic model may provide reusable representations across several sequence tasks, while a protein generation model may help researchers propose candidates around a function they want to achieve.
The evidence presented is promising but specific: benchmark comparisons for Nucleotide Transformer and a reported PAH variant for ProT-VAE. The next step implied by the work is to establish how well these approaches extend to further tasks and protein families. For now, they point to ways AI might help researchers explore biological possibilities that would otherwise be difficult to search systematically.