Package me.damoebe.architectures.transformer.mha

Class EDHead

java.lang.Object

me.damoebe.architectures.transformer.mha.Head

me.damoebe.architectures.transformer.mha.EDHead

All Implemented Interfaces:

Backpropagation

public class EDHead
extends Head

The Encoder-Decoder Head class

Field Summary

Fields inherited from class me.damoebe.architectures.transformer.mha.Head

biases, inputEmbeddingAmount, inputEmbeddingSize, learningRate, masked, weights

Constructor Summary

Constructors

Constructor	Description
EDHead(int inputEmbeddingAmount, int inputEmbeddingSize, double learningRae)	The main constructor of the decoder Head Identical to the super constructor

Method Summary

Modifier and Type	Method	Description
protected List<double[][]>	generateQKVMatrices(Sequence[] inputEmbeddings)	Calculates the QKVMatrices based on the decoder and encoder input embeddings.

Methods inherited from class me.damoebe.architectures.transformer.mha.Head

backPropagate, getInputEmbeddingAmount, getInputEmbeddingSize, getOutput, insertInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

EDHead

public EDHead(int inputEmbeddingAmount,
 int inputEmbeddingSize,
 double learningRae)

The main constructor of the decoder Head Identical to the super constructor

Parameters:

inputEmbeddingAmount - The amount of input embeddings

inputEmbeddingSize - The size of each input embedding

Method Details

generateQKVMatrices

protected List<double[][]> generateQKVMatrices(Sequence[] inputEmbeddings)
                                        throws Exception

Calculates the QKVMatrices based on the decoder and encoder input embeddings.

Overrides:

generateQKVMatrices in class Head

Parameters:

inputEmbeddings - The input Embedding lists -> here it has to contain 2 embedding lists. [0] is the decoder input and [1] is the encoder input

Returns:

The query, key and value matrices which will be used to calculate the attention.

Throws:

Exception