You will be shown an excerpt from a computer science scientific research paper, followed by a question about the paper. Please answer the question. Do not include any text in your response other than the answer.

Context: Next, we describe the LMs and their variants used in NILM. We experiment with four language models, based on both Transformer and RNN architectures. BERT small.This is the bert-base-uncased model with 12 transformer encoder layers and the dimension of the representations is 768. BERT tokenizer is based on the WordPiece model Wu et al. (2016). BERT large.This is the bert-large-uncased model which has 24 transformer encoders and representations have 1024 dimensions. DeBERTa.This is a transformer based language model and its tokenizer is built using Byte Pair Encoding Sennrich et al. (2016). We consider the DeBERTa base model. It has 12 transformer encoder layers and representations have 768 dimensions. ELMO.This is an LSTM based language model Peters et al. (2018).It has 3 layers and the output representations have 1024 dimensions. 

Question: What is the major one structural difference between ELMO model and others (BERT small, BERT large, DeBERTa)
ELMO is LSTM based language model, but BERT and DeBERTa is transformer based language model.