Klasse Head
java.lang.Object
me.damoebe.architectures.transformer.mha.Head
-
Feldübersicht
FelderModifizierer und TypFeldBeschreibungAll biases of the head in one list, (size = inputEmbeddingSize*inputEmbeddingAmount*3) -> used to calculate the queries, keys and values.protected final intThe amount of embeddings that are allowed to be inserted.protected final intThe vector dimension (size) that each inserted embedding must haveprotected final booleanAll weights of the head in one list, (size = inputEmbeddingSize*inputEmbeddingAmount*3) -> used to calculate the queries, keys and values. -
Konstruktorübersicht
KonstruktorenKonstruktorBeschreibungHead(int inputEmbeddingAmount, int inputEmbeddingSize, boolean masked) The main constructor of the decoder Head Identical to the super constructor -
Methodenübersicht
Modifizierer und TypMethodeBeschreibunggenerateQKVMatrices(Sequence[] inputEmbeddingLists) Generates a list of query, key and value matrices based on the heads weights and biases.intintdouble[][]Generates an output of this head by using the current attention, which is multiplied with the current value-vectorsvoidinsertInput(Sequence... inputEmbeddingLists) Generates an output for one input embedding list, regarding the decoder head attention rules.static double[][]multiplyMatrices(double[][] matrix1, double[][] matrix2) Multiplies two matricesvoidupdateQKVWeights(double[][] deltas) Updates all QKV weights based on delta matrix from MultiHeadAttention weights
-
Felddetails
-
weights
-
biases
-
inputEmbeddingSize
protected final int inputEmbeddingSizeThe vector dimension (size) that each inserted embedding must have -
inputEmbeddingAmount
protected final int inputEmbeddingAmountThe amount of embeddings that are allowed to be inserted. -
masked
protected final boolean masked
-
-
Konstruktordetails
-
Head
public Head(int inputEmbeddingAmount, int inputEmbeddingSize, boolean masked) The main constructor of the decoder Head Identical to the super constructor- Parameter:
inputEmbeddingAmount- The amount of input embeddingsinputEmbeddingSize- The size of each input embedding
-
-
Methodendetails
-
insertInput
Generates an output for one input embedding list, regarding the decoder head attention rules.- Parameter:
inputEmbeddingLists- The input Embedding lists -> here it can only contain one embedding-list- Löst aus:
Exception- if the inserted input embeddings don't fit the expectations
-
generateQKVMatrices
Generates a list of query, key and value matrices based on the heads weights and biases.- Parameter:
inputEmbeddingLists- The input Embedding lists -> here it can only contain one embedding-list- Gibt zurück:
- a list of query, key and value matrices
- Löst aus:
Exception
-
updateQKVWeights
public void updateQKVWeights(double[][] deltas) Updates all QKV weights based on delta matrix from MultiHeadAttention weights- Parameter:
deltas- The delta from the mha of this head
-
getOutput
public double[][] getOutput()Generates an output of this head by using the current attention, which is multiplied with the current value-vectors- Gibt zurück:
- The output for the input Embeddings as a matrix.
-
multiplyMatrices
public static double[][] multiplyMatrices(double[][] matrix1, double[][] matrix2) Multiplies two matrices- Parameter:
matrix1- The first matrix needs to have as many columns as the second matrix has rowsmatrix2- The second matrix needs to have as many rows as the first matrix has columns- Gibt zurück:
- The result matrix of the multiplication
-
getInputEmbeddingSize
public int getInputEmbeddingSize() -
getInputEmbeddingAmount
public int getInputEmbeddingAmount()
-