Skip to content

Recurrent Neural Networks

Recurrent Neural Networks (RNNs) are a class of neural networks designed to process sequential data by maintaining and updating an internal hidden state at each time step. This architecture enables the network to capture dependencies and patterns across time or across the sequence.

Key architectures

Standard RNN: At each time step, the hidden state is computed from the previous hidden state and the current input via a non-linear activation function (typically tanh). Standard RNNs suffer from vanishing gradient problems when sequences are long, limiting their ability to learn long-range dependencies.

Long Short-Term Memory (LSTM): Introduced by Hochreiter & Schmidhuber (1997), LSTMs address the vanishing gradient problem through a gating mechanism: input gates, forget gates, output gates, and a memory cell. This architecture enables the network to selectively retain or discard information over long sequences.

Gated Recurrent Unit (GRU): Proposed by Cho et al. (2014), GRUs simplify LSTMs by using only update and reset gates (no separate memory cell), making them computationally cheaper while retaining the ability to learn long-range dependencies.

Bidirectional variants: In many NLP tasks, context from both directions (past and future) is valuable. Bidirectional RNNs process the sequence in both directions and concatenate the outputs, enabling the network to access full context.

Applications in misinformation and fake news detection

  • Text classification: RNNs capture sequential linguistic patterns in headlines and articles
  • Rumor detection: LSTMs model rumor propagation patterns in social media sequences
  • Stance detection: Bidirectional RNNs with attention capture dependencies between comment and target text
  • Clickbait detection: Character and word embeddings fed to RNNs learn patterns of misleading headlines

Key papers