Questions: Capsule Networks

Question 1

A CNN trained on faces correctly classifies a normal face image. When shown an image with two eyes positioned below the mouth (spatially scrambled), the CNN still outputs high confidence for 'face.' What architectural feature of CNNs explains this failure?

• A. CNNs use too few layers to detect complex features like faces
• B. Max pooling discards spatial position information, so the CNN detects that face parts are present but cannot verify their geometric arrangement
• C. CNNs use ReLU activations which cannot represent negative spatial relationships
• D. Softmax output layers normalize confidence scores in a way that ignores spatial order

Answer: (B)

Max pooling is specifically designed to detect whether a feature is present somewhere in a region while being invariant to exactly where. This is useful for translation tolerance, but it means pooled representations lose precise spatial information. After several pooling layers, the network knows it saw eyes and a mouth, but the spatial relationship between them has been summarized away. Capsule networks were designed to address exactly this limitation by preserving instantiation parameters (position, orientation) throughout the network.

Question 2

In a capsule network, a 'mouth capsule' outputs a vector with length 0.95 and a specific orientation. What does each component represent?

• A. The length represents the capsule's learning rate; the orientation represents the error gradient direction
• B. The length represents the probability that a mouth is present; the orientation encodes instantiation parameters like position, size, and tilt
• C. The length represents the number of training examples containing mouths; orientation encodes the class label
• D. Both length and orientation together represent the capsule's confidence score, similar to a scalar neuron

Answer: (B)

This is the key architectural difference between capsules and conventional neurons. A scalar neuron outputs a single number (e.g., 'mouth detected with 0.95 confidence') that encodes only presence. A capsule's vector encodes two things simultaneously: its length encodes the probability that the entity exists, while its direction encodes how it exists — position, orientation, size, texture. This richer representation is what enables routing by agreement to verify geometric consistency between parts and wholes.

Question 3

In routing by agreement, connections between lower-level capsules (parts) and higher-level capsules (wholes) are strengthened when the part capsules' predictions about the parent capsule are geometrically consistent with each other.

• T. True
• F. False

Answer: True

This is the core mechanism of routing by agreement. Each part capsule makes a prediction about where and how a parent capsule should appear, based on the learned spatial transformation. If the nose capsule predicts a face at position (x, y) and the eye capsule predicts a face at approximately the same position and orientation, these predictions 'agree,' and the routing algorithm strengthens those connections, activating the face capsule. Disagreement weakens connections. This geometric verification is what allows capsule networks to reject spatially scrambled inputs that CNNs wrongly accept.

Question 4

Capsule networks are more computationally efficient than CNNs because routing by agreement eliminates the need for multiple convolutional layers.

• T. True
• F. False

Answer: False

The opposite is true. Routing by agreement requires multiple iterative passes between capsule layers — each iteration updates coupling coefficients based on agreement — making capsule networks significantly more computationally expensive than equivalently sized CNNs. This computational overhead has been a major barrier to scaling capsule networks to large images and datasets, limiting their practical adoption despite the theoretical elegance of the approach.

Question 5

Why does a capsule's vector output achieve viewpoint equivariance more structurally than a CNN's pooling-based approach?

Model answer: When an object rotates, all part capsules update their instantiation parameters (position, orientation) in a geometrically consistent way. Routing by agreement then still finds consensus among the updated predictions, activating the correct parent capsule. The recognition is tied to geometric relationships rather than specific pixel patterns, so novel viewpoints are handled by the same mechanism as familiar ones. CNNs, lacking explicit pose representation, achieve viewpoint tolerance mainly through data augmentation — training on many rotated examples — which is brittle and requires much more data.

Equivariance is different from invariance: equivariance means the output changes in a predictable way when the input transforms, while invariance means the output doesn't change at all. CNNs use pooling to achieve approximate invariance, discarding spatial information. Capsule networks are equivariant — when the input transforms, the capsule vectors update correspondingly, allowing the network to recognize the object in any pose it hasn't necessarily seen before.