Activation Function Explorer

Compare activation functions used in deep learning. Plot Sigmoid, Tanh, ReLU, Leaky ReLU, ELU, and GELU on the same chart; toggle derivatives and tune parameters interactively.

Visualization Controls Guide
Activation Curves
Sigmoid Tanh ReLU Leaky ReLU ELU GELU
Solid lines: activation f(x). If enabled, dashed lines: derivative f′(x). Adjust domain and parameters on the right.
Functions
Parameters
Leaky α0.01
ELU α1.00
Domain
x min-6.0
x max6.0
Samples400
How to use & What to observe
Quick start
  • Toggle functions (left) and derivatives (′) to compare shapes and slopes.
  • Adjust domain (x min/max) to zoom in around 0 where most differences matter.
  • Tune parameters: Leaky α (negative slope), ELU α (negative saturation), GELU exact vs tanh approximation.
Key differences
  • Sigmoid: outputs (0,1); saturated ends → vanishing gradients.
  • Tanh: outputs (−1,1); zero-centered but still saturates.
  • ReLU: 0 for x<0, linear for x≥0; no saturation for positives but “dead” for negatives.
Variants
  • Leaky ReLU: lets a small negative slope (α) to avoid dead units.
  • ELU: smooth negative saturation controlled by α.
  • GELU: smooth, probabilistic gating; exact (erf) vs fast tanh approximation.
Search terms: “sigmoid vs tanh vs relu graph”, “activation functions explained visually”.

Support This Free Tool

Every coffee helps keep the servers running. Every book sale funds the next tool I'm dreaming up. You're not just supporting a site — you're helping me build what developers actually need.

500K+ users
200+ tools
100% private
☕ One-time support Buy me a coffee 📚 Learn & support 9-Book Bundle - $9 Stay updated Follow @anish2good
Privacy Guarantee: Private keys you enter or generate are never stored on our servers. All tools are served over HTTPS.

About This Tool & Methodology

This explorer computes activation values and derivatives directly in your browser and plots them using Chart.js. Functions include Sigmoid, Tanh, ReLU, Leaky ReLU (α configurable), ELU (α configurable) and GELU (exact and tanh approximation). Inputs are clamped to avoid overflow in exp‑based functions for numerical stability.

Learning Outcomes

  • Recognize activation shapes and ranges (e.g., Sigmoid/Tanh saturation vs ReLU sparsity).
  • Relate derivatives to gradient flow and vanishing gradients.
  • Compare Leaky/ELU/GELU as remedies for dead ReLUs and improved learning dynamics.
  • Understand how α (leak/ELU) shifts curvature and affects optimization.

Authorship & Review

  • Author: 8gwifi.org engineering team
  • Reviewed by: Anish Nath (tools maintainer)
  • Last updated: 2025-11-19

Trust & Privacy

  • All calculations and plots run locally in your browser (no data upload).
  • Share links, if any, only encode selected options; remove parameters to keep sessions private.

Sources & References

Quick Access