Package me.damoebe.architectures.transformer.mha

Klasse Head

java.lang.Object

me.damoebe.architectures.transformer.mha.Head

Bekannte direkte Unterklassen:

EDHead

public class Head
extends Object

The Decoder and Encoder Head Object class

Feldübersicht

Felder

Modifizierer und Typ	Feld	Beschreibung
protected List<Double>	biases	All biases of the head in one list, (size = inputEmbeddingSize*inputEmbeddingAmount*3) -> used to calculate the queries, keys and values.
protected final int	inputEmbeddingAmount	The amount of embeddings that are allowed to be inserted.
protected final int	inputEmbeddingSize	The vector dimension (size) that each inserted embedding must have
protected final boolean	masked
protected List<Double>	weights	All weights of the head in one list, (size = inputEmbeddingSize*inputEmbeddingAmount*3) -> used to calculate the queries, keys and values.

Konstruktorübersicht

Konstruktoren

Konstruktor	Beschreibung
Head(int inputEmbeddingAmount, int inputEmbeddingSize, boolean masked)	The main constructor of the decoder Head Identical to the super constructor

Methodenübersicht

Modifizierer und Typ	Methode	Beschreibung
protected List<List<double[]>>	generateQKVMatrices(Sequence[] inputEmbeddingLists)	Generates a list of query, key and value matrices based on the heads weights and biases.
int	getInputEmbeddingAmount()
int	getInputEmbeddingSize()
double[][]	getOutput()	Generates an output of this head by using the current attention, which is multiplied with the current value-vectors
void	insertInput(Sequence... inputEmbeddingLists)	Generates an output for one input embedding list, regarding the decoder head attention rules.
static double[][]	multiplyMatrices(double[][] matrix1, double[][] matrix2)	Multiplies two matrices
void	updateQKVWeights(double[][] deltas)	Updates all QKV weights based on delta matrix from MultiHeadAttention weights

Von Klasse geerbte Methoden java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Felddetails

weights

protected List<Double> weights

All weights of the head in one list, (size = inputEmbeddingSize*inputEmbeddingAmount*3) -> used to calculate the queries, keys and values.

biases

protected List<Double> biases

All biases of the head in one list, (size = inputEmbeddingSize*inputEmbeddingAmount*3) -> used to calculate the queries, keys and values.

inputEmbeddingSize

protected final int inputEmbeddingSize

The vector dimension (size) that each inserted embedding must have

inputEmbeddingAmount

protected final int inputEmbeddingAmount

The amount of embeddings that are allowed to be inserted.

masked

protected final boolean masked

Konstruktordetails

Head

public Head(int inputEmbeddingAmount,
 int inputEmbeddingSize,
 boolean masked)

The main constructor of the decoder Head Identical to the super constructor

Parameter:

inputEmbeddingAmount - The amount of input embeddings

inputEmbeddingSize - The size of each input embedding

Methodendetails

insertInput

public void insertInput(Sequence... inputEmbeddingLists)
                 throws Exception

Generates an output for one input embedding list, regarding the decoder head attention rules.

Parameter:

inputEmbeddingLists - The input Embedding lists -> here it can only contain one embedding-list

Löst aus:

Exception - if the inserted input embeddings don't fit the expectations

generateQKVMatrices

protected List<List<double[]>> generateQKVMatrices(Sequence[] inputEmbeddingLists)
                                            throws Exception

Generates a list of query, key and value matrices based on the heads weights and biases.

Parameter:

inputEmbeddingLists - The input Embedding lists -> here it can only contain one embedding-list

Gibt zurück:

a list of query, key and value matrices

Löst aus:

Exception

updateQKVWeights

public void updateQKVWeights(double[][] deltas)

Updates all QKV weights based on delta matrix from MultiHeadAttention weights

Parameter:

deltas - The delta from the mha of this head

getOutput

public double[][] getOutput()

Generates an output of this head by using the current attention, which is multiplied with the current value-vectors

Gibt zurück:

The output for the input Embeddings as a matrix.

multiplyMatrices

public static double[][] multiplyMatrices(double[][] matrix1,
 double[][] matrix2)

Multiplies two matrices

Parameter:

matrix1 - The first matrix needs to have as many columns as the second matrix has rows

matrix2 - The second matrix needs to have as many rows as the first matrix has columns

Gibt zurück:

The result matrix of the multiplication

getInputEmbeddingSize

public int getInputEmbeddingSize()

getInputEmbeddingAmount

public int getInputEmbeddingAmount()